Sunday, December 28, 2014

Meaningful feedback in a timely fashion - the Wombat's biggest weakness & An All Time Thirsty!

I scrolled through my ratings on the site that shan't be named, going all the way back to the beginning.  I know it's a bi-modal pile of nonsense written by non-representative students:  the ax grinders and the super-keaners.  But... as I wrap up my 10th year of teaching, I thought there might be enough data to yield something meaningful in terms of a pattern.  And there was.  

There are intermittent "I learned everything I needed for my upper division classes from her" and "the MCATs will be a breeze if you take her class - but you have to do your part".  An occasional "she doesn't explain anything", and one fight that broke out between raters arguing about whether another rating was supposed to say "she has a lisp" or "she has a limp".  But in looking over the whole archive, there is a distinct pattern.  Most of them say the same two things, one good, one bad, in each rating.  The gist of the good is that I'm the "only professor who cares", or "who tries".  And the bad is:  

She's really disorganized with grading.  

So - now that the Christmas chaos has left my house, and I have a moment to freshen up my syllabi, I wanted to poll the commiserates for strategies to improve the quality and efficiency of my feedback.

  • How do you grade?  
  • How do you assign work?  
  • Lots of little things that can be graded in minutes?  
  • A few gigantic assessments that take time to evaluate, but only have to be done a few times?   Rubrics?  
  • How do we feel about multiple choice?  [I have a knee-jerk aversion to MC, but maybe there's an appropriate way to add MC (in moderation, of course)].  
  • Automating anything with Blackboard?  

Q: What are your tricks?  Why am I perennially failing to keep up with grading?  

Thanks in advance, 
The gimpy wombat with a speech impediment.  

16 comments:

  1. Fascinating. As a longtime writing instructor I use a 4 point rubric on all large assignments, with narrative descriptions of what qualifies as argument for a "4" paper, or documentation for a "3" paper. Students learn the rubric. They get it stapled to every major essay, and the questions are light once I teach the rubric with sample essays.

    I do half of my grades from small 1-2 point assignments, where 1 is doing OK work, satisfying the objectives, and 2 is A level work. Again, student samples show them the difference between meeting guidelines and really nailing them.

    I used to to do 5 major essays and 5 minor writing tasks per semester, but as my students have weakend I'm down to 4 & 4.

    I hope this topic takes off!

    ReplyDelete
  2. You should display the good in the same sized font as the bad. Don't sell yourself short.

    You're teaching chemistry, right? I have 4 or 5 MC tests each semester and they have to show their work for calculations to get any credit. That's a decent trade off between ease of grading and making sure students know what they are doing. The students like it too. I grade my own exams so saving time is important to me.

    Weekly homework is graded online by the textbook publisher's program. It's shitty but they cheat on the homework anyway. It's an easy A for those who make an attempt.

    Lab reports are a pain in the ass. The TAs use a rubric but it still takes them a long time to grade. I'm not happy with our requirements for lab reports so I'll probably change them next year.

    ReplyDelete
  3. Wombat, when it comes to feedback. I've just stopped.

    Sort of ...

    Long suspicious of the "Feedback, more feedback, for the love of Dog I can't breathe without feedback" comments on student evaluations, an assignment return oversight revealed the truth.

    So, a few terms ago, more than a little fatigued after a night of grading, I cut-and-pasted a "Your graded paper will be uploaded within 24 hours" notice into the grade book comments... and then (honestly) forgot to do it.

    Lo and behold, there were no wails of despair unleashed across the face of the known world.The sun came up, birds sang, life - as we know it - continued. I got one ... ONE! ... EMail saying the promised attached file couldn't be found. I zipped off a quick "Oops, my bad" reply and attached that paper.

    Then for the next assignment, this time deliberately, I said I attached feedback and didn't for anyone but the one student who had previously asked. Halle-fucking-lujah!

    So, my new system - for the first couple of weeks, I will hunker down and provide full bore feedback for everyone, "forget" to do it around Week 3, find the one or two actually interested students, continue to provide substantive feedback for them, and reduce everyone else to a few canned comments that can be batch uploaded to the comment box in the online grade book.

    I used to invest untold hours to download files, open them individually, use Track Changes, embed comment bubbles, lather, rinse, return and repeat. Now I batch download the submissions, scan them using the Windows preview window, starting at the references because 75% will have added nothing to the assigned readings, paste the canned "Failure to include sources beyond the assigned reading is a sign of extreme moral terpitude, seek immediate help before monkeys start flying from your anal region" comment (Hey, they're not reading it anyway!), assign a B- grade, and move to the next one.

    This system has reduced review time for 75% of the class to 30-45s/submission. For the 20% for whom the reference page scan reveals they actually added a couple of sources, I'll spend twice that time, so I can add a "Interesting point you got from the Jones article" comment to see if they have actually started paying attention to feedback (which they usually haven't), give them an A- and move on.

    That leaves me much calmer for the 5% who actually do review the feedback. Knowing that these comments are going to be reviewed, I am in a significantly calmer mindset and do not feel stressed when investing several minutes to the review of their work.

    Epilogue: Recently I had the same student take different classes in back-to-back terms. Despite it being pointed out, s/he kept making the same very noticeable but easily fixable mistake on the first page of every paper. Come the new class, the mistake was still happening so my brief grade book comment started getting quite a bit more testy. A few weeks into the second term, the student finally noticed and sent a message claiming that "No other instructor" ("Every Other Instructor's" cousin?) had included feedback in the grade book (where it is supposed to go) so s/he didn't know to look there.

    Apparently you and I, dear Wombat, are the only people still providing feedback at all!

    ReplyDelete
    Replies
    1. This is brilliant, A&S. I've long said that my grading would be much, much easier if I could find a way to identify those students who would actually make use of my feedback, spend 50%-100% more time on those students' work than I currently do on each student's work, and simply provide a grade for the rest. You may have found a way to actually do it.

      My one fear would be that my students have a tendency to become conscientious (or at least curious about the reasoning behind their grades) at the very last minute. I can imagine receiving quite a few queries about the missing file an hour or two before the revised version is due. Still, I think there's a way to make this work -- the sort of short, canned comments you mention, and/or a rubric, plus "let me know if you have any questions" might well do the trick.

      Delete
  4. I probably put far more time and effort into grading than I should. It -is- discouraging that so little of it is apparently even read at all. Still, it beats the nonsense that I put up with as an undergraduate, such as getting a paper back with a grade of C and the only other thing written on it being: "You have an interesting perspective, but you're wrong."

    I do use multiple choice in the larger classes. Jacques Barzun didn't like it, but all of his students had ACT scores over 29, and I doubt he often had a class of more than 15 of them. Multiple choice is clear and unambiguous, and since everyone is so eager to contest everything, it serves them right.

    Still, I do read and grade writing in many classes, even large ones. I find that a 3-point scale works well for shorter (1-3 sentence) answers. Here, 3 means essentially perfect, 2 means not quite perfect (e.g. off within a factor of 10), 1 means the basic idea is wrong (e.g. off by a factor of over 1000), and 0 only for nothing written. This helps grading to go quickly, and reduces ambiguity and arguments a lot.

    I find that one-page papers, done 3-4 times per semester in addition to other, more quantitative assignments, work well in large classes. I also find that about half the errors that people make in these are the same 6 or 7 errors, so I had stamps made up for them. They look authoritative, and they speed up grading quite a bit. They include:

    "Spelling error"

    "Grammar error"

    "Omit unnecessary words. See The Elements of Style, by Strunk & White."

    "Avoid colloquialisms or cliches."

    "Avoid opinion: let the facts tell the story."

    "How do you know? Please cite a reference."

    "This isn't college-level writing. See the Writing Center for how to improve your writing."

    "It's = contraction of it is. Its = posessive of it. Please avoid this error."


    And of course, for quantitative courses:

    "These digits aren't significan't. Don't write them, they're probably wrong."

    ReplyDelete
  5. Side note: Who would have ever predicted it actually is more time consuming to return assignments digitally?

    I pine for the days where - yes, it was a pain to lug a class set of papers around - but on return day, you put them on a desk, said "Pick yours up on the way out" and any left in the pile were filed circularly before leaving the room. Using one university's "customized" version of Black(hole)board, it takes 12 separate clicks/screen manipulations to attach one file

    So after slogging through the actual provision of feedback, tack on another hour to just return the papers. And heaven help you if you forgot to glimpse at the student name before scrolling to the submit button and mistakenly attach Smith_John's paper to Smith_Jane's grade..

    [RGM - Has there always been a 4,096(!) character limit on comments? I had to remove this from my previous comment. Maybe I had less to say in bygone days?]

    ReplyDelete
    Replies
    1. I think there's always been some sort of limit; at least (and this will surprise no one who regularly reads the comment streams here) I have periodically run up against it, and had to divide my comment.

      Delete
  6. This comment definitely comes under the blind-leading-the-blind category, Wombat, so I'm not sure I'll be much help; whatever I do, I seem to be perennially behind on grading (also, I hope maybe we could re-post or re-position this post sometime in the new year, when more people are reading, since I'm pretty sure it's a subject of perennial interest to many of us).

    I do both a couple of large, semester-long assignments, and a number of shorter, lightly-graded ones (mostly things that count as stages in the process of writing the larger papers). This is especially true when I teach at least partly online (when I teach entirely face to face, I try to cut down on the number of graded interim assignments, instead making them the subject of in-class workshops, etc., but that can be difficult, since many students seem to need the carrot/stick of a grade to do almost anything). I try very hard to stay on top of providing feedback for the truly crucial small-step assignments, but I inevitably fall behind on the less-crucial ones. However, when I frantically catch up at the end of the term, I find all kinds of shortcuts for grading such things quickly, which I try to remember to use earlier in the term the next time. I also find that I've missed a few spots where intervention would have been useful; this term, it was some of the plagiarism-checking reports, which revealed problems I could have caught earlier. So I'm going to make it clear(er) on the syllabus next semester that students are responsible for checking their own reports, and alerting me to any problems/questions. Ideally, I'd keep up with them better myself, but realistically, that's not going to happen, and it really is valuable for my students to do more work than I can realistically grade in any intense way (especially when you take into account the chaos quotient; it's one thing to grade a set of lightly-graded work once; it's quite another thing to be picking up late and revised posts for weeks afterward, which is inevitably what happens).

    The one thing that I do that works is that I now read drafts of the major long individual paper (and also the major group project) in a conference with the author(s), and offer feedback on the spot (with a 15-minute written sum-up just after the conference for the individual paper). The conferences are much longer than they used to be, but this is the only way I've found to keep myself on track reading that many drafts in that short a time I also think it helps me use time better, since I get some feedback from the student (as well as vice versa) along the way. Following a colleague's suggestion, I also have them write at the top of the paper before the conference three things they think are going well, and three things they know need work. That's definitely a help (there's no point in telling them things they already know).

    I need better, and more, rubrics (I have sort of a rubric-lite, but I think they're a good idea, as long as there's some room to add individualized comments). I'd be interested to hear what sort of software, if any, people who grade electronically use to generate, and use, rubrics (I know Bb has a built-in system, but I don't trust Bb quite that far; though I grade electronically, I prefer to do it on my own computer, then upload files to Bb, which may be time-consuming, but builds in a redundancy/a backup).

    I need to figure out how to use multiple-choice better this term. I'll be teaching a class large enough to require multiple-choice tests for the second time; the first time I tried it (last year), I ended up curving like mad just to get to the point where the majority of the class was passing the tests (which I wasn't even trying to make hard).

    ReplyDelete
  7. P.S. Great graphic; that's exactly how I feel (complete with the threat of imminent avalanche) much of the time during the term (but I'm afraid I don't look quite so chic perched on a ladder).

    ReplyDelete
  8. Sorry Wombat, forgot to mention these details in my previous rant and rantlet ...

    In my 9^n circles of online hell, I have no say in how/what assignments are doled out.

    Both the under- and graduate program have weekly discussion (essay exam-esque) questions. The undergrad program had focused on 2 - 3 medium-range papers/course but has been shifting to auto-administered M/C exams with 1 medium-range final paper.

    The graduate program adds regular short papers and then a jumbo final paper.

    ReplyDelete
  9. Like others who have already responded, I teach writing. My courses are capped at 24 each and I teach 3 sections (plus one section of lit capped at 38, but rarely get more than 20--except this semester, when I started at 35).

    Here's what I do. For all classes, I have a discussion board/forum set up. For a particular reading, I give a series of possible topics, or they can make up their own. Each post needs to be minimum 150 words and they're responsible for 2 responses to classmates' posts. There are 5 of these, frontloaded in the semester for the comp classes, spread out for the lit. I grade the first two or three assigned postings (responses are always p/f did them/didn't do them). The rest are p/f. I'm always surprised at how many of my students skip these, despite the fact that combined, they equal a major essay grade. Skip one, the best you get is 80/100. Skip two, 60/100. Stupid.

    For comp classes my department used to demand 5 essays/semester. I always only did 4 because 5 in 15 weeks doesn't leave enough time for drafting/revision, which my department also decided it wanted. As Hiram points out above, as my students have gotten weaker, I've dropped to three--because they really need more guidance and time. The first two are worth 100 points each, and the last is worth 200 because they are supposed to take the accumulated knowledge and apply it. I have some smaller assignments but those are p/f.

    I use rubrics for all of the major essays, broken into 5 categories (math works out better that way). I grade electronically, as our LMS has an app on the iPad. It doesn't take me as long to grade, and I can offer highly targeted feedback. I grade in the order they turn the work in (so the superkeeners get their feedback first), and my turnaround time this semester was 6 -12 days (and I never collect a major assignment unless I have given the feedback for the previous one). I can also see who has picked up feedback, and if enough of them haven't, I remind them in class that they need to do it before they turn in the next assignment because I come down hard on repeated (stupid) errors.

    In the big lit section: discussion posts (5), 2 essays, a midterm, and a take-home essay final that is faster to grade than you'd think with a holistic rubric and no comments.

    I have been at this a long time, and I despaired of ever figuring out how to cut down on the amount of time I spend grading. The one thing I tried this past semester was grading 5 (major essays) per day on "work" days, which left me fewer to face on my non-teaching day (Friday).

    I feel like I should also point out that I have a host of nasty little health problems that often make it hard for me to do the feedback as thoroughly as I'd like. They don't seem to notice, and on those occasions where I've had to cut corners, I cut and paste a "See me if you have questions about your grade" message into the feedback box. So far, nobody has bothered, so I might just go ahead and take Aware and Scared's advice and cut back on the commenting.

    ReplyDelete
  10. Being in a quantitative discipline there are some bread and butter classes where I can use a online homework system for routine assignments with only a modest amount of guilt. Doing that gives me enough time to hand grade the assignments for upper-division classes as well as all exams.

    Big goal: not above five actual days and two class meetings between receiving and retuning papers with well defined feedback.

    To achieve that without going insane I have a "comment code" in which common problems are assigned a letter, and I can markup most problems with useful feedback in ten or twenty seconds. More over, I give the students the code sheet at the beginning of the term, so they can see that "units should be used throughout" and "principles at work in the problem should be named in words" before they start the first assignment.

    I was getting things worked out. I was on top of the job and starting to feel like I knew my business.

    This was before I started teaching our "for poets" general education class. That thing is a time-sink like you wouldn't believe, and now I'm struggling to keep up with the grading again.

    ReplyDelete
    Replies
    1. I like your moniker. Thanks for the warning about the poets' course. I follow the same methods for my upper level classes. Spending time grading their exams and homework is more rewarding for me.

      Delete
  11. Some Educational Douchebags (Ed. D's) are welcome to disagree with me, but it has nothing to do with "appearance", "gimpiness", or any of the other niceties. Some people just don't respond well to teachers, period. And, no matter what you do, you will always be "unclear", "confusing", "disorganized", and "untimely" in grading.

    Back when I worked in the tutoring center at Wolf359, we had lots of people there who had developed many theorems about teaching. In response to why tutoring was often uneffective, one individual managed to derive what I call the "Stupid Theorem": "If ya' seem too much like a teacher, then people aren't gon' getcha." In other words, the presence of a teacher suddenly made people stupid. Like electrons in Quantum Mechanics, the presence of a third party disturbed the system.

    To which there is an obvious corollary: "We could have won the war in Iraq by dropping chalkboards, via parachute, to land behind the Iraqi generals when they were briefing the troops. Cue instant confusion..."

    Or would that only have an effect on the people from the states... I hope North Korea doesn't find out about this one.

    ReplyDelete
  12. Oof. I teach writing-intensive classes, for the most part, usually at least two a semester.

    First, I grade electronically, and come down with far fewer colds now. There's an app for that!

    But what I have now is a system that pretty much works, although I still spend more time grading than I want to. What I do is this:

    1. A simple rubric, just five or six items, yes or no. It's more a checklist than anything. Students still don't check it. One advantage of using electronic grading is knowing what students read and what they don't. They don't really give a shit why you gave them a C.
    2. A whole bank of canned responses I can cut and paste. I actually have so many now that it looks like I'm writing very original stuff even if students compare papers. But yeah, I can get about two hundred words of comments just on "you don't have a thesis, so why the fuck am I reading this?" and "paragraphs aren't decorative, please, throw me a goddamned line here, I'm drowning in your bullshit." My data show that they read this more often than the rubric, so I spend more time here. I'll usually plop the first part down: "Every paper must have an arguable thesis yadda" and then follow it with something specific to the paper "So when you say 'many things are similar and different,' you need to be more specific if you wish to convince a reader."

    The particular system I use also allows me to create rubber stamps, which are wonderful time savers. I can just drag "commas don't work that way; maybe you shoulda paid attention in eighth grade" onto the margins.

    The comment sandwich -- say something positive, then criticisms, then something positive -- is pure pernicious bullshit. I say what's true. That's it. If that's all bad, it's all fucking bad, and live with it. I was yelled at for that in grad school by my mentor (a gumdrop unicorn if ever there was one) and she was wrong then, and is now. If a student spent ten minutes on a three page paper I'm going to make it clear that I know that. I'm not a chump, and this shit matters to me.

    I probably average ten minutes a paper, but that's spending two minutes on purest crap that just needs a "see me ASAP" on it, and spending twenty or thirty minutes on something that might not be good, but shows some hope and promise. A lot of stuff gets a "See me," even though it's more work in the long run, because some students clearly don't respond to comments at all and need a face-to-face (guess how many ever, actually, see me?).

    ReplyDelete
  13. Hey! The comment box is showing up on my Android device today! (It's hit or miss.)

    Great thirsty; great ideas I will be using, particularly from Aware and Scared, Pissed Pumpkin, and (as always) Frod. Is the plural of "genius" "genii"?

    I'm in the trenches teaching all lower-division evolution classes, 5 per semester, 4 for poets, 40 students per class. We have no T.A.s or readers.

    I stopped our program offering online classes because the Little Dears didn't have the skills: reading, writing, understanding that they were supposed to demonstrate actually learning something. My huge time investment in setting up that class and providing unused feedback soured me about online grading. Plus I have kids to haul around to sundry practices and competitions, and bringing a pile of papers to grade works best for me.

    It helps me to color-code the classes and assignments, using small bulldog clips of various colors and washi tape labels. Each prep gets its own color paper for the syllabus and major assignments. Students have said this helps them too. My other big organization tool is an alphabetical sorter (e.g., Amberg P31) for use before recording grades.

    Then I can quickly sort a pile of grading by which class meets next. I rough-sort that bundle by obvious quality, focusing on one central but challenging question. Then I can fly through the incomplete and slipshod papers, give a few comments on the middling ones, and take more time on the few carefully done ones at the end.

    My majors classes have more writing assignments, drafts and rubrics. My poets classes have more worksheets and one big writing assignment, with a rubric. All my classes have numerous quizzes and three exams, with MC and "show your reasoning" items.

    My biggest problem is grading the exams promptly, a task both boring and important to get right. I daydream, fall asleep, and even clean my fridge rather than grade another damn test. What I need is a boot to the head.


    ReplyDelete

Note: Only a member of this blog may post a comment.