Practice · Deepen

OUTCOMES · ALONE, AS A LEVEL, AS A DEPARTMENT

Outcomes

You already grade against a standard. It lives in your head, in your rubric, in the comment you write for the fourth time on a Tuesday night. An outcome is that standard, written down once, in a place Canvas can count. Here is the moment it pays off.

WHY YOU WOULD BOTHER

It is a Tuesday in November and you are on paper 41 of 60. Since September you have typed some version of the evidence is here, the reasoning is not — tell me why it supports your claim maybe thirty times. You could write it in your sleep. You are not failing to notice the thing.

Now answer two questions. Are they better at it than they were in September — or did you get tired and start marking it more gently? And the students who still cannot do it: are they the same ones as in October, or a different four?

You cannot answer either one. Not for want of attention — you paid attention thirty times. But the noticing went into 60 comment boxes and nowhere else, so there is nothing to count. Writing it down once, as an outcome, is the whole difference between I have a strong sense of how this class is doing and I can show you.

And if the three colleagues teaching the other sections write the same sentence down, you get a question none of you can ask alone: is it our freshmen who cannot do this, or is it my room? Those are very different problems and they have very different fixes.

One caveat before you spend a minute on this. Writing outcomes down does not by itself teach anyone anything — the research on that is unusually blunt and it is in §11. What it buys you is a question worth answering. §9 is what you do with the answer, and it is the only section here that changes anything for a student.

NONE OF THIS IS AN EXPECTATION

Not required, not an initiative, not an evaluation criterion. Nobody is checking. This is a draft — idea starters, not finished practice — and none of it has been run at Regis Jesuit yet. You know your subject and I do not. What I can tell you is where the buttons are, what Canvas does with what you type, and what the research does and does not actually support.

SET IT UP 1 The picture · 2 Do it alone · 3 On a quiz · 4 How far you take it · 5 As a level · 6 The permission

READ IT, ACT ON IT 7 The two gradebooks · 8 The reports · 9 It came back low — now what

BEFORE YOU COMMIT 10 Where it breaks · 11 What the evidence says · 12 Where AI helps

Pick one thing. Not six.

Two you can do by yourself this week. Two need the people teaching your level. One needs a department meeting. One is deciding against it, which is a real answer. Each tile tells you where it goes.

ALONE One outcome, on a rubric you already use Since September you have written some version of the evidence is here, the reasoning is not on thirty papers. Write that one sentence down as an outcome, hang it on the rubric you already use, and Canvas starts keeping count of it. Nothing else about your grading changes. COSTS 25 minutes, once. GETS YOU By April you can see whether it is still the same students — instead of a strong sense that it probably is. How to do it → §2 Do it alone Goes deeper: Enhanced Rubrics — what changed	ALONE Turn on the Learning Mastery Gradebook Your gradebook says Marcus has a 79 and you cannot tell what is wrong. The mastery view says Marcus can do three of your four things and not the fourth — and so can nine other students, which is a Thursday reteach rather than 10 conversations. COSTS Two minutes. Genuinely. GETS YOU “Who has not got this yet” instead of “who has a low average.” How to do it → §7 The two gradebooks
WITH YOUR LEVEL Four shared outcomes, one shared rubric Four of you teach the level. All four rubrics say analysis and all four of you mean something slightly different by it — which is why comparing your grades has never told anyone anything. One meeting ends that. It is also the oldest finding in grading research: §11. COSTS One 80-minute meeting, plus somebody with the permission. GETS YOU Scores from four rooms that can honestly sit in one table. How to do it → §5 As a level Goes deeper: Designing as a team	WITH YOUR LEVEL Align a shared item bank You already pull from the same bank. Tag each question once with what it actually tests, and from then on every quiz any of you gives comes back saying which idea they missed rather than which question number — with nobody doing anything extra, ever again. COSTS An afternoon once, then nothing. GETS YOU Quiz data you can act on without reading the quiz. How to do it → §3 On a quiz Goes deeper: Building it together
AS A DEPARTMENT Read one semester’s results together The export comes back and one outcome is low in every section. That is not 11 rooms of weak students — that is the unit, or the sequence, or the task. It is the only finding that can change what the department does next fall, and nothing except this will surface it. COSTS One email, then half a meeting. GETS YOU The conversation this whole page is for. How to do it → §9 Now what Goes deeper: Quiz Data Coach	OR NOT AT ALL Read the limits and decide against it There is no dashboard, and the evidence that this raises achievement is thinner than anyone selling it will tell you. The level view is a spreadsheet I run and a conversation somebody leads. If that is less than you were picturing, find out now and not in November. COSTS Six minutes. GETS YOU A decision you do not have to revisit. How to do it → §10 Where it breaks

PART ONE

Set it up

About 25 minutes of clicking, once you have decided what you want to measure. Sections 1–6. Nothing here needs permission except §6, which is one email.

1 The whole thing in one picture

Five steps. The right column is the part that surprises people.

1	An outcome is written at the department level	Needs a permission most teachers do not have. That is §6.
2	Each teacher pulls it into their own course — Find, then Add	This does not happen by itself. What lands is a link, not a copy — the only reason 11 sections are comparable.
3	It rides on a rubric — or on quiz questions	A rubric is the only way to attach an outcome to a regular assignment. There is no other path.
4	Your Learning Mastery Gradebook fills up	Your course only. Off until you switch it on. This is where Canvas stops.
5	The level view — an export I run for you	Not a dashboard. A spreadsheet, once a semester, with course and section on every row.

↑ back to top

2 Do it alone — one Academic Support block

Nothing here needs permission from anyone. All of it, in your own course, in about 25 minutes. Every step links to Canvas’s own guide in case a button has moved.

1 Write the outcome

Outcomes → + Create. Name, friendly name, description. Then the part that matters: the ratings and the point where mastery sits.

Regis Jesuit has no house-wide mastery scale. The ratings you type belong to this outcome alone. Fine solo. With a level, agree the scale before anyone writes outcome one.

Canvas: create an outcome for a course → · Canvas: use the outcomes page →

2 Put it on a rubric

Rubrics → open one or Create New Rubric → Create From Outcome → pick it → Import. Then Create Rubric or Save Rubric. It is not saved until you click that.

Older rubric screen? The link reads Find Outcome and you finish with Update Rubric. Same thing underneath. It goes away in December 2026.

Canvas: align an outcome with a rubric (Enhanced) → · Canvas: add a rubric to a course →

3 Decide whether it counts toward the grade

Same dialog, one checkbox: Use this criterion for scoring. Leave it unchecked and the outcome is still assessed and still recorded as mastery — it just does not move the grade.

Unchecked is usually what people want the first time: real data, no argument about points.

4 Attach the rubric to an assignment

Assignment → Find Rubric → yours → Add, then Save Rubric. Two of the three checkboxes decide everything: Use this rubric for assignment grading (or the rubric is advisory), and Don’t post to Learning Mastery Gradebook, which must stay unchecked.

Checking that second box is what silently throws the mastery data away.

5 Turn on the gradebook that reads it

Settings → Feature Options → Learning Mastery Gradebook. Then Grades → the Learning Mastery tab.

Canvas: use the Learning Mastery Gradebook →

6 Check your work — the tab nobody uses

Outcomes → Alignments. Anything not attached to real work is collecting nothing. Two minutes here saves a semester of diligent assessment against an outcome that was never wired up.

Canvas: view all aligned items within an outcome →

What makes an outcome worth writing — open if you want the argument

Most outcomes fail the way most rubric criteria fail: they name the packaging instead of the thing. Writes a well-organized essay is a wish. Explains why the evidence supports the claim, not just that it does is an outcome — two teachers reading the same paragraph would agree on it, and a student could self-assess against it.

Four to six per level, not 20. An outcome you assess three times a year tells you something; one you assess once tells you almost nothing, and Canvas’s calculation methods assume repeated evidence. If your list will not fit on one rubric, it is a curriculum map, not a set of outcomes.

And pair them with student work. The argument that students calibrate to a standard far better from an annotated example than from criteria alone goes back to Sadler (1989). Worth saying plainly: that paper is a theoretical account, not a classroom trial, and nothing has been tested here. The practical version is hard to argue with anyway — an outcome without an exemplar is an outcome only you can apply.

↑ back to top

3 On a quiz, not just an essay

Three ways, and they stack. The third pays you back for work you have already done.

One question at a time	Quiz Options → Build → edit a question → Align to Outcomes → Confirm Alignments. Canvas: align an outcome to a New Quizzes question →	Precise, slow. For the six questions that actually matter on a unit test.
The whole quiz at once	Build → the Outcomes button → select → Align Selected.	Fast and blunt. Says the test is about that standard, without saying which question proves it.
In the item bank — do this one	Align the item in the bank. The alignment travels into every quiz that ever draws it, for every teacher sharing the bank. Building it together →	Align once and a level’s whole quiz program reports mastery forever.

THE TRAP, AND IT IS SILENT

A bank aligned to outcomes a course has not imported produces no data and no warning. So if a level shares a bank, every teacher still runs Find → Add in their own course. Miss it and your quizzes look fine and report nothing.

↑ back to top

4 How far you take it

The same feature, four times over. What moves you up is how many people your answer is good for.

FOUNDATIONS You have one One outcome, on a rubric, with something in the Learning Mastery Gradebook. You have stopped re-deciding what “proficient” means every time you grade. Serves: you.	ENHANCE It is deliberate Four to six outcomes, each assessed more than once, on an essay and a quiz. Clean Alignments tab. You can answer “how do you know” by pointing at something. Serves: you, and whoever takes your sections next year.
DEEPEN The level shares it Outcomes live at the department, everyone has imported the same ones, one rubric is shared. Different rooms, same standard. This is the rung the page is aimed at. Serves: everyone teaching the level.	INNOVATE It changes what you teach The department reads a semester together and something moves — a unit gets more time, a sequence flips, a shared task gets rewritten. This is §9, and it is the only rung the research says actually matters. Serves: the students you have not met yet.

↑ back to top

5 Doing it as a level — the whole sequence

The same work, in the order it has to happen, across one semester. Left is one teacher. Right is what changes when a level does it together. Nothing in the first two rows needs anyone’s permission.

WHEN	ALONE	WITH YOUR LEVEL
BEFORE THE TERM One 80-minute block	Decide the four to six things you want to be able to say about a student at the end of the course. Write them as sentences, not headings. On paper.	Same conversation, out loud, everyone in the room. Also agree the rating scale and where mastery sits. Nothing in Canvas enforces it later, so it is decided here or it is not decided. Have the person with the department permission in the room.
WEEK ZERO 25 minutes	Outcomes → + Create. Build one rubric with Create From Outcome. Attach it to a real assignment.	One person writes them at the department, in a group named for the level — or imports the whole set from one file, which is §12. Everyone else runs Outcomes → Find → Add in their own course — five minutes each, and the step levels forget. Canvas: import outcomes into a course → · Designing as a team
SAME DAY Two minutes	Turn on the Learning Mastery Gradebook, and decide about the student-facing one. That is §7.	Everyone turns it on, and the level makes one decision about the student view rather than four — students in the same level talk to each other.
BEFORE YOU GRADE Two minutes	Check Outcomes → Alignments. Anything aligned to nothing collects nothing. Fix it now, not in November.	Everyone checks their own. One clean Alignments tab and three unchecked produces a level comparison built on one section’s data.
DURING THE TERM No extra work	Grade as you already do. Assess each outcome more than once — twice is the minimum for the number to mean anything.	Do not edit the outcomes. Changing one changes every rubric using it — except rubrics already used on a student, which freeze. Write it down, change it between terms.
END OF TERM Half a meeting	Read your Learning Mastery Gradebook and the New Quizzes outcomes report. Then §9.	Ask me for the department export, put it on the table, and find the outcome every section missed. Then §9 — which is the only part of this that changes a student’s experience.

↑ back to top

6 The permission you need — and how to ask

Everything in §2 and §3 you can already do today, in your own course, with no permission from anyone. The one thing you cannot do alone is write an outcome at the department level — the one everybody can pull from. That is all this section is about.

Your chair can already do it

Every academic department has a chair who can build outcomes at the department level today. They may not know — the permission has sat there unused.

Ask them first. Shorter conversation than emailing me.

Or ask for the role yourself

One person per department is a bottleneck, and the people who know a level best teach it. Outcome Designer exists for level leads and anyone a department wants writing outcomes. As many people as you need can hold it.

Email me exactly this: “Please give [name] the Outcome Designer role at the [department] sub-account.” No form, no rationale, no meeting, no limit.

What it grants: write, edit and import outcomes, and build rubrics — in your department only. What it does not: anyone’s course content, grades or roster; any other department; any setting. If a colleague asks what you can now see of their course, the answer is nothing you could not see before.

What happens after it is granted, and how to organize several builders

On your screen: an Admin link appears in your left-hand menu, going to your department, where you will find Outcomes and Rubrics. Nothing else.

Three timing notes. Canvas emails you automatically when it is granted. The permission can take half an hour to take effect — not broken. And a brand-new department outcome can take up to 48 hours to appear in the New Quizzes picker, so do not write outcomes the morning you plan to build the quiz.

How many people hold it, and who writes what, is your department’s business. Nothing here is a rule. The one thing worth knowing: several people writing into the same department without talking tends to produce near-identical outcomes, and results split across duplicates do not add up to much. The common fix is a named group per level, with whoever leads that level writing inside it — write in your own group, import from anyone’s.

And if three levels turn out to have written their own version of the same standard, that is not a mess. It usually means the department has a shared standard it has never named, which is worth 10 minutes of a meeting.

↑ back to top

PART TWO

Read it, then do something

Ten minutes at the end of a unit, and half a meeting at the end of a term. Sections 7–9. §9 is the one that matters — everything before it only produces the number.

7 The two gradebooks — and who sees what

Two separate switches, both off until you turn them on, both in Settings → Feature Options. One is for you. One is a teaching decision. (Thinking of Mastery Paths? Start here — different feature, same word.)

FOR YOU

Learning Mastery Gradebook

Students down the side, your outcomes across the top. Each cell is a mastery judgment built from every aligned assignment and quiz question — not an average.

Turn it on: Settings → Feature Options → Learning Mastery Gradebook, then Grades → Learning Mastery.

Canvas guide →

THE QUESTION IT ANSWERS
Not “who is failing.” Which idea has this class not got yet — and which four students have not got it while everyone else has.

Give it a term. With one assignment behind it the grid looks thin and unconvincing.

FOR STUDENTS

Student Learning Mastery Gradebook

Adds a Learning Mastery tab to the student’s own Grades page. They see their standards, not the class’s.

Turn it on: a separate switch — turning yours on does not turn theirs on.

THE DECISION UNDERNEATH IT
This changes what a student thinks a grade is. Done well, near mastery on one standard and mastery on three gives them somewhere specific to go instead of a B-minus and a shrug. Done carelessly, it is a second scoreboard to be anxious about.

If you turn it on, say something. Five minutes in class is the difference between a useful tool and 30 emails. Start of a term, never week nine. (A third switch, Student Grades Page Learning Mastery Default, makes it the tab they land on. Leave it off until the rest works.)

Why the mastery number is not an average — open if a student asks

Each outcome carries a calculation method, chosen when it is written: weighted average, decaying average, a number of times the student must hit mastery, most recent, highest, or plain average. The default weights recent evidence more heavily than old evidence.

That is the real difference between the two gradebooks. A regular grade says here is everything you did, added up. A mastery score says here is what you can do now. A student lost in September and fluent in November has a mediocre average and a strong mastery score, and the second is the truer sentence.

Which means the two columns will disagree, visibly, in front of students. Decide which sentence you are willing to defend before you turn on the student view. One caution: changing an outcome’s calculation method after students have been assessed does not fully reapply to earlier work until it is graded again.

↑ back to top

8 The reports — and what each one is for

Four you run yourself, one you ask me for. Running the wrong one is how people conclude the data is useless.

REPORT	ANSWERS	USE IT FOR
Learning Mastery Gradebook Yours · Grades → Learning Mastery	Where every student sits on every outcome, right now.	Tomorrow. A column mostly below mastery is a reteach. Four students in an otherwise fine column are four conversations — see §9.
Outcomes → Alignments Yours · course Outcomes page	Which outcomes are attached to real work — and which to nothing.	Coverage, before you grade. An outcome with one alignment in a semester is a claim you cannot support. Canvas guide →
New Quizzes outcomes report Yours · the quiz → Reports	Per-outcome results on one assessment. CSV. Needs three submissions to build.	The same-day read after a test. Pair it with item analysis — a weak-looking outcome may be one badly written question. Quiz Data Coach.
One student’s outcomes report Yours · People → student → Options	Every attempt behind one student’s mastery score.	The parent meeting. Replaces “you have a 78” with “you can do three of these four things, and here is the one to work on.” Canvas guide →
Department outcome results export Ask me · one email	Every student, every outcome, every score — with course and section on every row.	The end-of-term department meeting. One outcome low in one section is a conversation about practice. Low in every section is the department’s problem. Both are answered in §9.

Read it like a teacher, not a statistician. Three students out of 30 is not a pattern. A section of 18 and a section of 31 do not compare on raw counts. And year one mostly measures how consistently four people applied a new rubric — worth knowing, and worth saying out loud in the meeting. The number tells you where to look. You still have to look.

↑ back to top

9 It came back low. Now what?

This is the only section on the page that changes anything for a student. Everything before it produces a number. A number does not teach anybody.

The research is unusually blunt about this — no grading system improves learning by itself, because changing how you record a score changes neither what you teach nor how you teach it (§11). Whatever gain there is comes from what you do on this page. So: the number came back low. Read the shape of it first, because the shape tells you whose problem it is.

THE DATA LOOKS LIKE THIS

Low in every section

Not the students. Four teachers in four rooms did not independently fail at the same thing in the same week. It is the task, the sequence, or the time the unit gets.

THE MOVE
Take it to the department meeting, not to your class. Pick one change: more time on it, move it earlier so it gets spaced practice across the term, or rewrite the task. One, not three — or next year you will not know which one worked.

THE DATA LOOKS LIKE THIS

Low in your section only

Practice, not curriculum. This is the most uncomfortable finding and the most useful one, and it is why the shared standard was worth the meeting.

THE MOVE
Go to the colleague whose section did fine and ask what their students did — the activity, the sequence, the example. Not the philosophy. Red/White means you see your students every other day, so a change you make lands twice a week and you will know by the next unit.

THE DATA LOOKS LIKE THIS

Fine, except four students

Not a reteach. Reteaching the class costs 30 students a lesson to fix a problem four of them have.

THE MOVE
Four five-minute conversations in Academic Support, 9:55–10:20. Twenty-five minutes, no study halls, everybody free — you can do all four in one sitting without touching class time. It is the most underused block in the building.

THE DATA LOOKS LIKE THIS

Low, but item analysis blames one question

A measurement problem, not a learning problem. A single badly worded question can drag an outcome under mastery for a whole class and it looks exactly like a teaching failure.

THE MOVE
Fix the question in the bank, not the unit. Then the fix travels to every teacher sharing it. Quiz Data Coach is built for telling these two apart.

ONE SHAPE THAT IS NOT A FINDING AT ALL

Low, and you only assessed it once. You do not have a result, you have a single data point — and Canvas’s mastery calculations assume repeated evidence, so the number is thin by design. Assess it a second time before you change anything. Half of what looks like a finding in a first semester is this.

THE ONE RULE, IF YOU REMEMBER NOTHING ELSE

Change one thing, and write down what you changed. Not three. Otherwise next semester’s number cannot tell you anything either, and you will have done all of this twice for nothing. A sentence in a shared doc is enough — “spring 2027: moved the evidence unit from March to October.” That sentence is what turns a number into a finding.

↑ back to top

PART THREE

Before you commit a level

Roughly six minutes, read once. Sections 10–12. If you only read one, read §11 — it is the honest account of what this does and does not buy you.

10 Where this breaks

None of these is a reason not to start. All of them are reasons to promise less.

There is no cross-section dashboard	No screen in Canvas — at any permission level, including mine — puts four sections side by side. The level view is an export somebody runs and a conversation somebody leads. Better you hear it now than build a semester expecting a dashboard that does not exist.
Nothing enforces a shared scale	We do not run house-wide mastery scales, so every outcome carries its own ratings. Consistency across a level is a promise people keep, not a setting. Agree it once and write it down somewhere that is not a person’s memory.
Editing an outcome mid-flight is messy	Change an outcome and every rubric using it changes — except any rubric already used on a student, which keeps the old version. Change outcomes between terms.
“Mastery” means two unrelated things	Mastery Paths branches students by assignment score and has nothing to do with outcomes. The Learning Mastery Gradebook is the outcomes one. If someone in a meeting says “mastery,” ask which.
The evidence is thinner than the enthusiasm	This one has its own section. §11.

↑ back to top

11 What the evidence actually says

Three findings, because one study is not an evidence base. Each one gets what it actually supports, what it does not, and where it touches this page. If you want to check any of it, every source is linked at the end and none of it is behind our library wall.

The short version: the case for writing a shared standard down is a century old and very strong. The case that doing so raises achievement is weak. The case that it raises achievement when paired with a real change in what students get to do is one good trial. Promise the first. Work on the third. Do not claim the second.

FINDING 1

Teachers grading the same work disagree wildly — and have since 1912

Starch and Elliott gave the same two English exam papers to 142 high school teachers and got marks spread across 30 to 40 points on a 100-point scale. The following year they repeated it with 138 mathematics teachers and a single geometry paper, where the disagreement was wider still. Starch traced about 5.4 points of the error to three causes, and one of them was simply that teachers held different standards. Brookhart and colleagues reviewed a century of follow-up work in Review of Educational Research and the finding has never gone away.

WHAT IT MEANS	“All four of our rubrics say analysis and we each mean something different” is not a joke about your department. It is the oldest and best-replicated finding in grading research, and it is the reason two teachers’ grades have never been comparable.
HOW YOU BUILD IT HERE	This is the entire argument for §5: write the standard down once, at the department, and agree the ratings before anyone writes outcome one. That is the specific error source Starch named.
WHERE IT BREAKS	This is about reliability, not learning. It says shared standards make scores comparable. It says nothing about whether comparable scores help anyone teach better. That is Finding 2.

FINDING 2

Changing your gradebook does not, by itself, improve learning

Linka and Guskey reviewed the standards-based grading evidence in Theory Into Practice and put it flatly: “No grading system by itself improves student learning. Why would we expect it to? Changing grading does not alter the curriculum (i.e., what is taught) or instruction (i.e., how it is taught).” Their conclusion as of 2022 was that no evidence indicated that standards-based grading improves achievement, and that the approach “remains largely unstudied.”

WHAT IT MEANS	If this page stopped at §8 it would be paperwork with a nicer spreadsheet. Any teacher who suspects that is right, and should say so in the meeting.
HOW YOU BUILD IT HERE	It is why §9 exists and why it is the only section that claims to change anything. Build the outcomes if you want the question. Budget the time for the answer.
WHERE IT BREAKS	“No evidence of benefit” in a thinly studied field is not the same as “evidence of no benefit” — their own complaint is that nobody has studied it properly. And their review predates Finding 3.

FINDING 3

The strongest trial that did find a gain paired the standards with a second attempt

Kramer and colleagues ran a cluster randomized controlled trial across 29 schools — 14 treatment, 15 control — in ninth grade algebra and geometry. The PARLO program did two things at once: rated students not yet proficient / proficient / high performance against learning outcomes, and let them re-assess for full credit after further study. Result: 0.33 standard deviations on the end-of-course test, statistically significant at p < .014 — roughly a typical student moving from the 50th to the 63rd percentile.

WHAT IT MEANS	The two halves came as a package, and the trial cannot separate them. One half is a way of recording a score. The other half is a real change to what a student is allowed to do after failing at something. It is not hard to guess which half did the work.
HOW YOU BUILD IT HERE	If your level takes this on, the re-assessment policy is the thing to argue about — not the rating scale. Deciding whether a student can have another go, and what they have to do to earn it, is the decision with evidence behind it.
WHERE IT BREAKS	One trial, one named program, ninth grade mathematics, one subject area. The effect was larger for students who were already more motivated. And at least one advocacy site is already summarizing it as “gold-standard study confirms standards-based grading accelerates math learning,” which is more than the study says. Read it yourself before you quote it.

SOURCES — CHECK THEM

Brookhart, S. M., Guskey, T. R., Bowers, A. J., McMillan, J. H., Smith, J. K., Smith, L. F., Stevens, M. T., & Welsh, M. E. (2016). A century of grading research: Meaning and value in the most common educational measure. Review of Educational Research, 86(4), 803–848. Full text →
Linka, L. J., & Guskey, T. R. (2022). Is standards-based grading effective? Theory Into Practice, 61(4). Full text →
Kramer, S., Posner, M., Browman, A., Lawrence, J., & Krier, K. (2024). The impacts of a standards-based grading system emphasizing formative assessment, feedback, and re-assessment: A mixed methods, cluster randomized control trial in ninth grade mathematics classrooms. Journal of Research on Educational Effectiveness. Summary →
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144. (A theoretical account, not a classroom trial — see §2.)
Starch, D., & Elliott, E. C. (1912). Reliability of the grading of high school work in English. School Review, 20, 442–457; and (1913) the mathematics replication. Both are discussed in Brookhart et al. above.

↑ back to top

12 Where AI helps, and where it does not

§11 is the reason this section is shaped the way it is. The argument in the room is the part with evidence behind it. Everything downstream of the argument is typing, and typing is worth handing off.

Worth handing over

Turning agreed outcomes into rating descriptors a student could self-assess against · drafting rubric rows so they read in one voice · producing the outcomes import file, which is the three-step workflow that follows · sorting an existing item bank by which outcome each question actually tests, so you align in one pass instead of 200 · drafting the questions themselves.

Not worth handing over

Deciding what the level should be able to do. Ask a model for “outcomes for freshman English” and you get competent, generic, standards-flavored prose that four teachers will nod at and none will use, because nobody argued about it. Finding 1 in §11 is that teachers genuinely mean different things by the same word — the argument is where that gets discovered. Automate it away and you have a shared document without a shared standard, which is worse than nothing because it looks like progress.

Writing a level’s outcomes with AI — the workflow that actually leverages this system

The payoff is at step 2: Outcome Designer includes the import permission, so a whole level’s set can go in as one file instead of being typed one at a time.

STEP 1

Argue it in the room first

Four to six sentences, in your own words, agreed out loud — plus the rating scale and where mastery sits. No AI in this step. This is §5, and it is the part the research says is load-bearing.

Bring paper. An 80-minute block does this; a 45 does not.

STEP 2

Have it build the import file

Give the model your agreed sentences and your scale, then point it at Canvas’s own outcomes CSV specification → and ask for a conforming file. Required columns are vendor_guid, object_type and title; mastery_points, calculation_method and ratings are optional but you want them. Ratings go last, as alternating points-then-label pairs. The header row is mandatory.

Then Outcomes → Import at your department. That button is the CSV uploader — here it is finally the one you want.

STEP 3

Read them for one failure

An outcome that names the assignment rather than the capability. Completes the evidence paragraph worksheet is a task; explains why the evidence supports the claim is an outcome. A model produces the first kind constantly, because tasks are what it can see.

Also check the descriptors are things a 14-year-old could self-assess against. If they are not, they are for you, not for them.

THE SAME CHECK, FOR QUESTIONS

If you use Item Bank & Question Bank Expander to fill a bank, an AI will cheerfully write a question that tests whether a student remembers the wording of your outcome rather than whether they can do it. Those look perfectly aligned and measure nothing. One check: could a student answer this by having memorized the outcome, without ever practicing the skill? If yes, throw it out and ask again. Then Raider Quiz Builder Assistant and Raider Quiz Builder package it, and you align in the bank — §3.

WHAT CANVAS’S OWN AI DOES AND DOES NOT DO HERE

Stuck on where a button is? Canvas LMS Support Assistant for RJHS answers Canvas questions in our context. Sign in with your school account — it is scoped to RJHS, so a personal ChatGPT login will not open it.

The AI rubric generator you may have seen in a webinar is not switched on here. Canvas ships one, it builds rubric criteria inside an assignment, and I checked our account and course feature flags on August 25 — we do not have it. Worth knowing before you go hunting for the button. If a department wants it, that is a request to make, not a setting I can flip. And note what it would and would not do: it generates rubric criteria, which are not outcomes and do not report to the Learning Mastery Gradebook. The outcome has to exist first.

Canvas’s AI question generation for New Quizzes is switched on here. I have not written up its click-path yet, so this page will not pretend to. If you want to try it, ask and I will sit down with you and we will document it properly.

Where to go next. Assess by Design covers the same machinery from the item-bank side. Write an assessment worth trusting is the part that has to be right first — an outcome on a task that does not measure what you think it does just gives you confident wrong numbers. And if you would rather do this with someone, book a session — happy to sit with a level for a block and build the first four together.

↑ back to top

LAST VERIFIED · AUGUST 25, 2026 · Click-paths checked against the linked Instructure guides and against our own Canvas. Feature flags read from our account the same day. Every study in §11 checked at source, not quoted from memory. Canvas moves; if a button has moved, tell me and I will fix the page.