Thursday, December 22, 2011

BREAKING NEWS: Teaching quality guaranteed by snowflake crowdsourcing: Study

Rate My Professors is a valid instrument after all. Didntjaknow? Happy holidays!

35 comments:

  1. I've been at my college for 2 years, and this "RMP is REAL" vibe is in my department. I've had serious conversations with a couple of colleagues about how valid it is.

    So, I did what any tenure track proffie should do, I rated myself.

    Once wasn't enough. I got into it. I liked it. I liked there being a public record that I was a "dynamite" teacher.

    There are 11 rankings of me on RMP. I wrote 7 of them. I gave myself the fucking chili pepper 6 of those 7 times.

    ReplyDelete
  2. Just checked my ratings and they're in the tank. I didn't get any for an entire year, and now I have 6 - four bad, one neutral, one good. Blargh. Just gave myself a nice rating to boost my spirits!

    ReplyDelete
  3. Well, I have very crappy ratings on RMP, and very good ratings from student evals.

    I never thought about rating myself on RMP, but I don't really care that much. The last RMP responses I got were so egregious--one stated I didn't keep office hours (uh, I do--I haven't missed any this semester), that all I could do was just roll my eyes. I think it would be far more suspicious if I got great ratings on RMP and low ones on student evals, so I can live with it.

    But I'll tell you, I don't see how anyone could cite RMP as a legitimate source because as Tony has stated, everyone can contribute their own.
    If it were me and I was starting out somewhere, I would carefully and regularly contribute to RMP on my own behalf. I would start early, in my first semester, and contribute on a 1-1 basis with students.

    If they're going to start taking bullshit seriously, at least I'm going to be the one shoveling it.

    ReplyDelete
  4. My friends and I contribute glowing reviews to one another's RMP profiles.

    ReplyDelete
  5. Dropping by again to add: Someone's PARENT left me a rating on RMP. Oh my Jesus fucking Christ.

    ReplyDelete
  6. What happened to the days when kids were embarrassed to have their parents at their recitals, let alone writing reviews of their classes?!

    ReplyDelete
  7. Don't give yourselves good reviews on RMP. That just makes the slacker students think that you'll b easy. Give those ratings to your colleagues. Let them deal with that crap. You give yourself ratings that say you are too hard. Maybe sprinkle in a little of the classic "tough but fair" for good measure. It scares some students away and those who do take your class might be pleasantly surprised that you're not an ogre.

    ReplyDelete
  8. I have been teaching for over 2 years and haven't made it onto rmp

    ReplyDelete
  9. For years, the place where I used to teach used 2 types of student evaluations.

    The basic one was used by the teaching staff and it was prepared anonymously. Theoretically, the results were sent to the instructor in question and were supposed to be private but I long had the sneaking suspicion, though, that someone in charge managed to see them before I did.

    The other one had to be initiated by the department head, the students had to sign them, and only that administrator got to see the results. Usually, something had to be seriously wrong if the second one was ever used; after all, someone's reputation was at stake. That evaluation was, therefore, designed to give an instructor an opportunity to tell his or her side of the story.

    Ah, but that place had the mentality that if something worked, it had to be fixed, so the administrator-initiated evaluation was eventually abolished and only the basic one was used. Anonymous student comments were encouraged because, according to my last department head, they were more accurate as those comments could be made without fear of retribution. Unfortunately, those same anonymous comments were often used in the annual performance appraisals, making things much more difficult for an instructor as he or she was rarely allowed to know the identity of whoever complained. There was no obligation on the part of the department head to investigate any further and, in effect, an allegation was a conviction.

    I quit teaching shortly after RMP started. I'd hate to think the comments on that site might be used to as part of an official performance appraisal.

    ReplyDelete
  10. What does it mean if I don't show up on RYP??? I feel so left out. How will I know if I'm doing a good job or not if random people don't rate me anonymously online?

    ReplyDelete
  11. Even more ridiculous than my RYP entries are the student evals I read today. It's the first time I have ever seriousl contemplating just quitting.

    ReplyDelete
  12. My evals say I wear black too much. maybe I am just mourning the lack of intelligent discussion in the classroom.

    ReplyDelete
  13. Dear Reverend,

    I remember having precisely that reaction one year. I realised that my fatal error was reading the student evaluations. I have never done so again.

    My institution makes this easier, however, by giving me the option of leaving the comments out of my file, which I do.

    ReplyDelete
  14. Harumph. I thought RMP was dead - except for one prof uber-flake in the dept (of about ~30 profs), several have had a couple of new entries, and everyone else hasn't had any new entries in 2, 3, sometimes 4 years. Have the students moved on to another prof rating site?

    ReplyDelete
  15. I read the paper.

    {{{ yawn }}}

    It states, "The primary objective of the current study was to determine the reliability of students’ ratings."
    Reliability, yes. Validity, no.
    "Validity" (or "valid") was mentioned only a few times in the paper.
    This psychology proffie from Eau Claire seemed to make some presumptions that aren't necessarily supported by logic--but could be!

    On a lighter note, when I read the name of her town, I always think of a passionate, orgasmic teenage boy shouting, "Oh, Claire! Oh, Claire!!!!"

    ReplyDelete
  16. Huh. What will they be using next, what's written on the walls in the bathroom?

    ReplyDelete
  17. Kids, it's easy to deal with the-site-that-will-not-be-named (as we used to call it on a site that was created to parody it, RYS). If you ever get a review you don't like, click on the button to the lower right, "Report this rating." In the box this will bring up, type: "This is libelous," and send it off. I've done this several times now, and every time, the review was removed within a day or two.

    That said, the number of posts per time clearly show that student interest in using the-site-that-will-not-be-named is waning. But then, Gen Y is noted for their minuscule attention spans.

    ReplyDelete
  18. "There is the possibility that people may feel legitimized to use the information in potentially dangerous ways," says April Bleske-Rechek, ... who is a co-author of the new study.

    Ya Think?!?!?!?!?!

    Meanwhile, back at Tuk U, any criticism of my precious little snowflakes (like say, "Normie Numnutz not only copied his assignment, but did so with such breathtaking incompetence that he left Dan Dingleberrie's name on it") requires hours of formal hearings, complete with the involvement of student advocates.

    ReplyDelete
  19. Merely Academic:

    What fucking pisses me off is the idea that knowing your shit, and being engaging in student discussion and so forth, is considered arrogant. These students must love my colleagues who are real pussies, such as the multiple-time teaching award winner tells his class every day how much he loves them. And suggests passing beach balls around class and other "insightful teaching tips."

    I'm never looking at these agian except in the presence of a supervisor. I'm not paid enough for this and it's not my job to deal with this shit. I had one comment that is the sort of thing that would lead to a lawsuit if it were stated openly. I can handle most of the bullshit, but a student crossed the line with me this time.

    ReplyDelete
  20. @R and/or G:

    Tuk U sounds like where I used to teach.

    @Reverend:

    Why do you think that my former employer encouraged anonymous student comments?

    ReplyDelete
  21. I've found that all of the rants about how awful the professor is are always borderline illiterate...both on written evals and RMP. Do other people find this as well?

    ReplyDelete
  22. Sorry to be a stick in the mud, but check out some other folks from your school on RMP. See what you think.

    I've found that no single comment means much of anything, but taking all the comments together gives you a pretty good idea of what someone's class is like. It works pretty accurately for my school.

    On the other hand, one of the things I've learned from reading CM is that the pricier and more prestigious the school, the more privileged the student, the more likely it is that snowflakes will blame teachers for their own failings.

    I'm at a community college with mostly Latino/Chicano working-class and working-poor students. When they don't come to class or don't do their homework, they (usually) don't blame me.

    ReplyDelete
  23. Full disclosure:

    My RMP ratings are good, so there's no doubt I'm biased.

    My last rating was over a year ago, so students at my school don't spend much time on RMP any more.

    I wrote two (out of about 30) of my own ratings.

    I gave myself chili peppers.

    ReplyDelete
  24. Reverend FU -

    I taught a class several years ago in my specialty. It was the first time I'd had the chance to teach on my actual field of research, and I knocked myself out to do a good job. My teaching evaluations (from the university) were extremely good, the class went well, and I was pleased.

    Until the next time I taught it, when less than half as many students signed up for it. I couldn't figure out why. I looked for scheduling issues and there weren't any. Finally it occurred to me to check RMP.

    There was a single review on it, from an obviously disgruntled student who probably got a C on the class and was unhappy about it. Nothing about the class was good, apparently. I was the world's worst teacher, the class was dull as ditchwater, etc etc.

    But that was the only review of the class. So anyone checking RMP would see that the class had an average grade of "lousy".

    I was really, really depressed about this. It was such a great subject, and the students who had done in-class evaluations had got a lot out of the class and they'd done good work, and one dull-witted, mean little shit could make such a huge difference. I spent an entire year feeling really lousy about my job, my ability as a teacher, everything.

    Eventually I realised that my mistake was in reading what one mean-spirited little bastard had to say, and decided I would never do that again.

    I also read a couple of books on 'how to teach", took a couple of the seminars the university gives on teaching, and made sure I was personally certain I was doing a good job teaching. I know I am. I give it my best. So I don't read comments anymore.

    ReplyDelete
  25. @Philip: Aren't we silverbacks supposed to be teaching that evidence trumps anecdotes? Can you back up your assertion with a statistical analysis, which is better than the one presented in the paper? What you have here is subject to confirmation bias (one tends to remember the hits, and forget the misses), just like the Lunar Effect (the common misperception that accidents and crime are worse during Full Moon: they're not).

    @Merely: E-mail "this is libelous" to the-site-that-will-not-be-named. Chances are good they'll take the comment down.

    ReplyDelete
  26. Froderick - I would certainly do that, if I ever visited the site, which I don't.

    ReplyDelete
  27. I won't visit it. I have colleagues who check me out from time to time and tell me my reviews are "mostly good" and say that I'm "tough but fair." I don't want to give them the page hits. I think it's interesting that my best students have told me they don't use it at all. They say that only the students looking for easy proffies go there.

    Those of you who followed my chair saga earlier will not be surprised to learn that Outgoing Crazy Chair actually spent a good 10 minutes of my year-end evaluation time asking me if I'd been to the Voldemort Site and raving about what a great tool it could be to improve teaching skills. The only questions I can think of that are worse than the ones on our current student evaluations are the ones I've read RMP uses. Hotness? Easiness? "Clarity?" The lies I see on my official student evals are bad enough. (Wasn't available to help even as I sat in my office for every required hour, didn't keep students informed of their grades even though they never waited longer than 9 days.) I don't need to see ones posted on the Internet.

    ReplyDelete
  28. Frankenstien: My comments--pretty obviously--are based entirely on anecdotal evidence. My RMP ratings are good, so I'm biased. The sense I get from looking at colleagues' RMP ratings tends to confirm what I "know" about their teaching.

    We all have opinions about our colleagues, and most of them are not based on empirical evidence (and we'd spend decades arguing about what kind of empirical evidence could measure effective teaching). Our--nah, let me say MY, opinions are based on hearsay, overheard office-hour conversations, glimpses into a classroom as I walk by, watercooler gossip, who knows what all? And even if I've done a classroom observation and filled out an evaluation form, I know that that's just a 50-minute snapshot of one single class.

    "Osmosis" is the metaphor I use to describe the way I form these opinions. I'll bet almost all of us do the same thing. I think my judgements/opinions are usually pretty accurate, and I'll bet most of you all think your opinions/judgements are accurate, too.

    So why not RMP? As I said earlier, no single RMP rating means a damn thing, but taken together, they're--in my experience, anyway--somewhere close to reality.

    ReplyDelete
  29. "I think my judgements/opinions are usually pretty accurate."

    Funny, I hear my students say the same thing.

    ReplyDelete
  30. @Philip: I am not convinced. Two wrongs don't make a right. I say we get rid of this abjectly useless practice of anonymous student evaluation of teaching altogether. Two of the best RYS posts on this were:


    What Should Really Be Given As Instructions On the Day You Pass Out Student Evaluations In Class (Tuesday, April 18, 2006)

    http://rateyourstudents.blogspot.com/2006/04/what-should-really-be-given-as_6940.html

    and:

    On Student Evaluations (Tuesday, November 28, 2006)

    http://rateyourstudents.blogspot.com/2006/11/on-student-evaluations_918.html

    Neil Postman also had an interesting discussion on using numbers where they don't belong in his book, "Technopoly."

    My physics department now relies almost entirely on peer reviews for evaluating instructor quality. It's not perfect, and it's more work for us, but it's worth it. We are very thankful to the instructor a few years back who got great evaluations by coddling his students: once we managed to use a bad budget situation to get rid of him, we haven't been fooled by what students say ever since.

    ReplyDelete
  31. I've gotten to the point where I try predict how posters will respond. I even have a rather large stack of nickels hidden in a mostly empty desk drawer here, and I'll scoot a small stack of these nickels to one corner of the drawer and make a mental note that I believe so-and-so will write thus-and-such. If I win, I get to use those nickels to do something frivolous like buy the most sugar-filled soda pop from the drink machine. If I lose, I have to add the coins to the next tip I give a waiter or random other service person.

    I lost the bet on Frod here. Could have sworn he was going to reprimand Phil for his ridiculously mediocre so-called inductivism and then implore him to read some Karl Popper. I imagined it was going to be a concise paragraph. But I was going to give myself a lot of leeway with regard to the length and the content. Well... I lost big. Frod's comment wasn't even within the vast penumbra I was going to allow. This is massively discouraging. Fourteen nickels. And an even bigger ding to my self-esteem.

    Thanks, Frod.

    ReplyDelete
  32. Why am I unsurprised when I see one of those RMP unsmiley faces next to the name of the guy who takes the first part of every meeting to point out--incorrectly, as often as not--where the commas should go in the minutes from the last stupid damn meeting?

    He's usually the same guy who has so little sense of the room and the vibe in the room that when everyone else is looking forward to ending something a miraculous ten minutes early, he'll raise his hand to point out another objection to whatever. And he can't even hear the audible groans from his "colleagues."

    If this is "ridiculously mediocre so-called inductivism," then I'll live with it. But I'll also read Karl Popper to see if I need to change my mind. Promise.

    ReplyDelete
  33. The prof's who rate themselves are just wasting their time. I cross check course evals, grade distributions, and the RMP ratings all together before cherry picking classes. I don't care if you're the meanest and hardest prof in the world...you give out A's and B's according to the grade history, then I'll tolerate you to get the curve.

    ReplyDelete

Note: Only a member of this blog may post a comment.