Can we receive the presentation deck?

Rubrics are particularly fun! :)

no

That’s it

yes

I’ve always heard “analytic” rubric contrasted with “holistic” rubric (which is like your original checklist, looks at student work holistically as opposed to breaking out and analyzing the specific components of the student work.

RE: Structured Observation Guide - this also seems like a great tool for creating meaningful conversation among faculty with regard to what they look for and value.

Also for when specific creative things might look different person to person but still fulfill the category well

is there a way to get copies of those different examples including structured observation guide?

Yes Miriam. I will check with Linda to see what she can share with us

Thank you Jarek. I would find that useful as well.

It also makes grading a LOT faster! (At least for me)

Linda, given the difficulty and time-consuming nature of grading, can you speak to the advantages of using rubrics for grading over “auto-grading” via a quiz in Canvas or similar?

Can rubrics be used outside of learning outcomes, like administrative outcomes?

I also always have to double check my rubrics that is I grade all in the "adequate" row, the score actually adds up to a passing grade on the assignment.

If you have an odd number, I have read, it is too easy to choose the middle / average level. As opposed to four or an even-numbered columns where you have to make a choice.

I will admit that I have used "appropriate" in rubrics, particularly where I am attempting to evaluate responses in health sciences settings. Some health interventions are appropriate and others are not. I would welcome a suggestion for better language, though.

I think that I have also used it for engineering tech projects as well.

I have started using "under-developed" "emerging" and "well-developed" -- but I'm not confident with those descriptors.

Amy, what is intended? Safe? All steps followed? Culturally competent? Use the adjective that’s needed for that event-product.

I like those suggestions, Andrea.

For an Agriculture school we used these descriptors, fwiw: emerging, shooting up, blooming, bearing fruit.

^^ Love this!

No, I think it was branching out, not shooting up.

I often use four levels : “Excellent, Adequate, Shows Effort, Needs Work.”

I use "Beginning, Intermediate, and Exemplary"

Is there a consensus for whether rubrics should build left to right or right to left? (It may be pedantic, but both approaches seem to be out there)

Transparent Assignment Design!

I also use four levels - Novice, Developing, Approaching, and Proficient

Excellent book!

I liked the option this past spring for "pass" and "high pass" for course grades. Now thinking it might be good for an assignment rubric.

I like to have the highest achievement level on the left, because that is where most of my students will start reading.

And I don't necessarily expect that they will read the whole rubric. So I want them to read what I want them to do, not what I DON'T want them to do.

@Amy, excellent points!

I use “Emergent, Proficient, Accomplished” for program assessment, and a 4-point scale for course level assessment “Emergent, Fairly Proficient, Mostly Proficient, Accomplished”

I like the principle of putting the important thing on the side that the students will read first. That said, some students' first language reads RIGHT to LEFT. So in the Arab world or Israel, the high proficiency category should maybe be on t he right. (Just saying)

@Amy & @Katherine, good point. I hadn't thought about them not reading all the way. I was sort of mentally checking off the levels left to right when looking at an assignment. Maybe that's helping me more than the students, though. Thanks!

Yes, good point, Katherine.

One of the things that can help them see the other levels is how it is used…for instance, having students do a mid-point self-assessment using it would help them develop metacognition of where they are and also help them refocus on the higher level understandings-behaviors they need to get to

And subcultures.

I think it’s good practice to share the rubric up front and even discuss it with students. This is an equity practice as well.

+1 re subcultures.

I have also had my students use the rubric to create a checklist they use in peer review.

do not assume all kids like McDonald's

I agree share up front-be transparent- also students can help with the rubric!

Yes!

Linda, can you talk about concretizing more subjective outcomes like creative or interesting?

YES!

YESSSS!!!

(Re grading is not editing

@Andrea, great question-- I am interested in this, too

Great presentation!

This is what I use for Reflection Assignments.

Thank you so much for this presentation! Would it be possible to share these PowerPoint with us for reference?

@Andrea -- I hope Linda answers your question too. But may I also recommend Essentials of Creativity Assessment by Kaufman, Plucker & Baer

What if we don't have demographic data on our students while we have them?

thank you so much, fantastic!

@Katherine, thank you!

Thanks, Katherine - I’ll check that out

Super presentation! Focused, clear, with great examples.

Thanks Linda --and everyone. Gotta run to next zoom meeting

Thank you for the talk - very interesting indeed

Yes, I’ve seen that! It’s wonderful.

I use it in classes

Thank you; I guess a follow-up question (if there is time after others’) is how you would persuade faculty to spend the time creating and using a rubric or similar, as opposed to just doing their multiple choice quizzes….

How do you determine individual performance on student learning outcomes in group projects?

(Ha ha)

You can have an individual component that’s graded, but that does not solve the problem of the time to grade individual work

@Felicia - I think that the key is to highlight that the Rubric is a TEACHING tool (when given to and discussed with students) that is not just about making the process easier for faculty - but also about improving the learning for students . . .

Love it, Tate!

@Tate -- right on! It's an important piece of transparent teaching design.

Linda, what strategies do you use to reflect or check how well your rubrics worked in your courses? What do you look for in determining if your rubric was effective?

I always say, “there’s a reason we write it in Word and not in stone.”

I attach the rubric I use to student assignments, and also give it to them in advance when assessing pre-final drafts. I even have students apply rubrics in group work when workshopping drafts.

Esp the first time you use a rubric or assignment! Always wind up tweaking for the next use

@Tate! Yes. I think it boils down to thinking of students first…

Here is a rubric I use for Discussion Board post in Blackboard.

Question: Follow up on what Linda said about grading group projects rather than individual projects.... How do we determine individual performance on the learning outcomes if we give group projects?

Do you differentiate an assessment rubric (like VALUE rubrics) from a grading rubric?

CA: IF you want to know they participated in gathering the knowledge, can have each st keep a list of what they contribute to the group and ALL sign off that yes, that person contributed that. If want to see individual learning, a SHORT summary paper (1pg) from each can also be graded, or a small quiz can be give after on the topic for accountability and assessment

how effective are holistic rubrics for program learning outcomes?

Question: How do can rubrics be leveraged to support equitable learning outcomes for students? What's the relationship to culturally competent teaching strategies? (Apologies if this has already been addressed -- came in late from another meeting about Ensuring Learning!)

@Andrea - thanks

I recommend Mark Wise, Rubric Repair: 5 changes that get results from Cult of Pedagogy. 1. measure what matters 2. weigh criteria appropriately 3. check your math 4. use positive language 5. provide lots of models. https://www.cultofpedagogy.com/rubric-repair/

@Rafael Salas has a great question!

@Stacy - There were two AALHE webinars (Karen Singer-Freeman was a co-presenter on both) about assessments of student learning to support equity and rubrics were discussed in at least one if not both. She will also be presenting at IUPUI Assessment Institute (starts this Sunday)

How do you respond to schools that insist that grading be separated from program-level assessment even when using rubrics?

Thank you for the information, @Sarah! I'll have to check it out.I think helping to be really explicit about how purposeful curriculum and assessment design can help instructors who care about teaching in a more equitable way to embrace these strategies as a way to move the needle on equity.

Can you share that source with all of us?

I can recommend Megan Oakleaf on rubric validation (generally) and inter-rater reliability

Work smarter, not harder...

Thank you!

Thank you so much for this session!

Thanks!

Thank you!!

Thanks, to Linda, Jarek, and all