Spring 2011's course evaluation results for both sections of Hamster Fur Weaving 301 just arrived, and the results are...peculiar. It appears that in the one section, I'm nigh-on incompetent, but in the second section, I'm quite the professional educator. Check out the most egregious disparities:
"The instructor has an effective style of presentation."
Section 001: 2.444
Section 002: 3.600
"The instructor provides useful feedback on student work."
Section 001: 2.667
Section 002: 3.636
"The instructor provides timely feedback on student work."
Section 001: 2.667
Section 002: 3.727
"The instructor clearly explains the grading system."
Section 001: 2.444
Section 002: 3.545
Both sections met on the same day, so it's not surprising that I had my rap down better for the second section...but a full point spread on these scores? Something's not right in evaluation-land.
Interesting.
ReplyDeleteI also had two groups of the same course on the same days this semester. One met just before lunchtime, and the other at 4 in the afternoon.
I haven't seen my evaluations yet, but if there's a discrepancy, I'm actually expecting it to be the other way around, with the good evals from the morning class, and the not-so-good ones from the afternoon class.
The morning class was just better prepared and overall more engaged than the afternoon class. Sometimes that's just how classroom dynamics work out. While you might expect that every class will contain its share of keen students and its share of slackers, sometimes a class has some indefinable quality that makes it work or not work.
The hour of day also makes a difference, I think. The late-morning class was late enough that it didn't require students to be out of bed early, but was also early enough that they had not yet slogged through a full day of classes. The students were generally awake and ready to go. The students in the afternoon class, on the other hand, often seemed to be suffering from a combination of post-lunch coma and general couldn't-give-a-damn. There were some good students in that class, but the overall feel was much more lethargic.
Also, to be honest, I sometimes felt like I wasn't as "on my game" for the afternoon class as for the morning one. Sure, whenever you teach a class for the second time, you can always correct the mistakes that you made the first time, and get in stuff that you missed, but there were times when I felt a certain weariness at having to give the same lecture or have the same discussion again about three hours after the the first time. Teaching isn't just about knowledge and understanding, it's about performance, and there were times when I felt that my second performance of the day might have been a bit flat. Gives me more respect for stage actors, who give the same performance over periods of weeks or months.
Further evidence that student evals are pretty useless. If you've been on the job for any length of time, I highly doubt there's that much difference in terms of what you do in earlier and later sections on the same day.
ReplyDeleteI've had this happen more than once, and I, too, suspect that time of day, classroom (though sometimes the disparate sections have been back-to-back in the same classroom), and class makeup play a role. At my institution, time of day can also correlate with preparation and/or general organizational skills; students in sections that meet in the more desirable time slots also tend to be more advanced and/or organized, since earlier registration slots and knowledge of what one needs to take, and when, both play a role in which student lands in which section.
ReplyDeleteBut, at least on our old paper forms, which made such patterns easier to see (we're transitioning to online, which makes it harder to associate comments and scores), the most common reason for such a discrepancy between sections -- or for any noticeably low cumulative score -- has been 1 or more students who gave me straight 0s or 1s, and expressed extreme unhappiness with grading, most often citing the supposed fact that I "don't give As" (I do, in fact, but fully satisfactory work earns a B; it says so on the syllabus). With the number of surveys returned frequently falling in the low to mid teens (out of a section that theoretically numbers in the low 20s; especially toward the end of the semester, students tend to abandon the gen ed class I teach in favor of "more important" classes in their majors), one or two very low scores can make a significant difference.
I'm also interested in the differing ranges of evaluations at different institutions (and perhaps in different departments). Assuming you, like we, have evaluations that use a 5-point scale (I realize you might be working with 4 points instead), then the numbers that are interpreted as okay vs. alarming definitely differ from my own institution. I've never been able to pin anyone down on numbers, but people seem to start getting concerned any time a cumulative score falls below 4.0, and 4.0-4.25 is considered just barely satisfactory. Congratulatory letters start going out somewhere around 4.7-4.8. The whole scale seems to be skewed unreasonably high to me, but that's how it works, though no one can explain why. Maybe we're located, psychologically at least, in Lake Woebegone.
I'm curious to see if the different sections also had differeing grades (i.e. did one class do better than the other)? I have two sections (we're still going since we're on the quarter system) and the 9 a.m. one is a dud. The 1 p.m. one is amazing!
ReplyDeleteBut what others have said: this goes to show how useful student evals really are!
To add confusion to vagueness, my livelier, more engaged class was the one with the lower numbers and more hostile comments.
ReplyDelete@ Cassandra -- The scores are out of 4. I should've specified that in the original post. *facepalm*
ReplyDelete@Jonathan -- I've had that happen, too. I suspect that more able students may have higher grade expectations, and/or that the gang mentality that can take over at evaluation time may be more pronounced in classes whose members interact well. Or maybe my perspective is just a bit skewed after a semester with two sections of very grade-oriented scientists whom I had working mostly in groups. Yes, I'm worried about those evals.
ReplyDelete@Mindbender -- that makes sense. Taking into account that the percentages wouldn't be the same (but who really thinks about math and what's statistically significant and all that silly stuff when evaluating evaluations, anyway? If they did, we wouldn't have them), it sounds like the senses of what's good and what's worrying are actually pretty similar at our two institutions.
I'm sorry ... I posted a similar observation earlier this year and the CM Chorus responded with a resounding:
ReplyDeleteWHY THE FOCK ARE YOU EVEN READING YOUR EVALS?
Was there a change in policy? Did I miss the memo?
I am with Aware and Scared - though I admit that my chair wishes I would occasionally look at the numbers, even if I refuse to read the comments. But I figure my chair is there to tell me if there's a problem. let HIM read the numbers.
ReplyDeleteSo: that's interesting, and why are you reading these things?
If you don't have tenure, though, forget I said that.
@A&S: maybe the tenured, or at least tenure-track folks, at research institutions started that chorus? There really is a difference between institutions, and kinds of jobs at the same institution, in whether one can afford to ignore them. If I could ignore them, I would (or would read them quickly, mentally store away any nuggets that might actually help me improve the course, and pay them no further mind). As it is, I have to think about them -- even though that doesn't lead to any conclusion other than, perhaps, that the powers that be would probably be happier with me if I took note of which students are very fixated on getting As, and gave those students As (and -- a key point -- didn't tell the powers that be that that was what I was doing, since that might force them to contemplate, even for a moment, the possibility that Arum and Roksa are right: overuse of student evaluations to evaluate contingent faculty increases grade inflation and helps undermine the value of a college degree).
ReplyDeleteThe fact that there's a long thread over at ProfHacker on what to do if students write something inaccurate in evals, with many comments suggesting detailed memos to chairs, deans, etc., plus a follow-up post , suggests that this is a matter of interest to many people (or that the folks who frequent ProfHacker are a bit neurotic about audience/"customer" reactions; probably a bit of both).
The above was cross-posted with Merely (I'm trying to teach myself how to use basic html tags; it's going slowly).
ReplyDeleteWhat students see.
ReplyDelete