A ton of people have sent us this little Slate article today. Here's some flava (a portion of the whole), and a link to the rest:
[+]
Do Professors Read Their Reviews on Rate My Professors?
by William B. Harrison II
 |
| Just wait till I get on RMP! |
I gave up most of my professorial indignation over Rate My Professors several years ago. It doesn't really matter whether faculty like it. Just like the massive open online courses (MOOCs), we could not will it away if we wanted to. Now, maybe once a year but not on any schedule I look at my reviews to see what students have said. I smile when students say things like, "He's nice, but don't let him fool you. He grades hard!" It does not surprise me when they say, "He's not the most organized professor." One student said I shared too much of my personal life. I thought about it and made adjustments.
I suspect that sharing the fact that a professor is an easy A may backfire. When a slack reputation gets out, gets too wide and too deep, instructors tend to attract too many lazy, handout-seeking students. Often, the next thing that happens is that these instructors begin to feel hassled and bored with unresponsive classes. At some point, they tighten up in the classroom and in their grading. One of my favorite comments was posted to a graduate school professor of mine: "Easy B, tough A." I could live with that.
MORE.
If I were to ever look myself up, I might have a moment where I thought, "Maybe I am not the be all."
ReplyDeleteNah, who the fuck am I kidding?
Showing your characteristic modesty as always, Walter!
DeleteI understand the problems inherent in anonymous student evaluations, the bias that affects ratings of female faculty, type of course, time of day, etc. Given all that and more, I find the RYS evaluations for my colleagues to be pretty accurate, at least for those with more than a few posts. That's based on my experience working with them, seeing their presentations and talking to them about teaching. A bad teacher isn't just bad for 50 minutes, three days a week. That incompetence is around 24/7 and shows up in lots of ways outside the classroom.
ReplyDeleteWe use RYS rankings, with an appropriate amount of salt, when we are judging candidates for a job. You might flip out over this but consider our other sources of data: a CV, research plans and teaching statements all written by the candidate and letters of recommendation written by his friends. Those sources are also biased, probably even more so since the writer has a significant incentive to say positive things about him/herself or a friend.
Ben: just keep in mind that some of us, with exactly this possibility in mind, write some of our own RYS reviews. At this point, c. 1/3 of my recent reviews are posted by me (the content is usually a paraphrase of mostly-positive comments I got on official reviews, and/or simple, good advice for succeeding in the class: do all the steps more or less on time; email her, she emails back; etc.). I wouldn't bother if I were tenured, but, since I'm not (and not eligible for tenure in my current position), I figure it's wise to "curate" my online persona, including RYS ratings, just in case I need to go job searching at some point. Students get good advice; potential employers, well, caveat emptor (and watch your blood pressure; consuming a sufficient amount of salt to compensate for all the variables that might apply is probably not healthy).
DeleteOn the brighter side: if you were to read my reviews, you'd get a mix of actual student reviews and advice that reflects the kind of teacher I want to be (and sometimes am, and might be more frequently/completely with a more reasonable course load). I suspect even reviews written by the subject tell you something about him/her.
Another data point: I've also been told, by a reliable authority (a lecturer in a very large class at a fairly prestigious very expensive private university with very demanding/entitled students, who monitors and curates the reviews her TAs receive), that one can get pretty much any review taken down by sending an email protesting that it's libelous, or simply untrue. They don't investigate; they just take it down. I've never done this myself, but have heard others (including some who disapprove of the idea of writing one's own reviews) say that they regularly get negative reviews removed, just by asking.
DeleteI've written them to take down particularly nasty reviews of colleagues I've had over the years that included inappropriate references to their bodies (there was more than one of these). They were always gone the next time I checked.
DeleteJust to be clear, you all mean RMP, right, Rate My Professor, the anonymous professor rating site, and not RYS, Rate Your Students, Cal's old blog.
DeleteOH! Stupid abbreviations! Yes, I meant Rate My Professor. Sorry! Talk about a libelous comment...
DeleteI was wondering if anybody wrote their won comments. That's the surest way to bring down a site like that. Using RMP for evaluating job candidates is one use but there's lots of ways students use them that piss me off. I'm alll in favor of posting your own comments. I've wanted to post complimentary comments about my colleagues so that students stop trying to overload my classes.
Ben, I've read your comment a couple of times to make sure I'm understanding it. Your letters are written by people you consider friends? Well, in that case of course they don't have much value in a job search / grant situation.
DeleteI have 6 letters I use for different purposes and only one of those writers would I consider a friend - and one about 10 years and a thousand miles removed.
Do you not have letters from bosses, Deans, and such?
I'd never include a letter from a current friend for tons of reasons, the most obvious being that it'd have so little value.
If any others are following this thread, could you weigh in. Am I off base here? Are academic rec letters collected from people you consider friends?
Ben, maybe we're using the word friend in different ways.
I know or knew the people who have written my letters, but they were always my boss in some way, from a department chair to a VP of instruction. They knew my teaching because they observed me.
And of course I had tons of colleagues who could have written letters, but it would never occur to me to use them. Some were idiots, so why them! And the rest were my pals, so obviously NOT them.
And most of the rec letters I've written in the past are for people who worked for me, not alongside me.
If I've cocked up the understanding, help me.
No, the misunderstaning is my fault - I'm actually on friendly terms with people who might write me letters of recommendation. Most have been to my Superbowl parties. Instead of friends, I should have said, "people who view the andidate favorably." When writing their letters, they probably consider how much of their reputation is at stake since they don't want to recomemnd an idiot. However, a reasonably wise candidate would select writers who view him or her favorably. While they aren't friends, the writers are favorably disposed to the candidate.
DeleteOops, yes; I, too, meant RMP (blame Ben!). And just to clarify: my ratings weren't terrible without my contributions, but they tend to hover just in the range between the yellow neutral face and the green smiley one. I figure it's safer, from a job-hunting perspective, to be in the green range.
DeleteBoosting your ranking is acceptable. Giving yourself chili peppers is not.
DeleteTHAT would be laughable. Claiming that I'm tough but fair isn't, at least on my good days (even on my bad days, I tend to be mostly tough and always fair; I can just be way, way too slow at getting work back to students, and that does impede their learning -- or at least the learning of those who're going to actually read the comments, but I'm not too good at telling which ones those are).
DeleteOh, for crying out loud, folks.
ReplyDeleteIt is a bad day when "the-site-that-shall-not-be-named" is, at long last, named. FOLKS: RYS was founded specifically as a backlash to "the-site-that-shall-not-be-named." Until now, it was rarely or never named.
In the name of mercy, please stop naming it!
@Ursula: That's correct. If the-site-that-shall-not-be-named ever says anything about anyone that is libelous, all you have to do is to e-mail them a message that says "This is libelous," and it will usually be removed within 24 hours.
Good God, Ben, really?
ReplyDeleteI throw my hands up as an indication of my innocentce. What did I do wrong?
DeleteI want to reiterate that I value RMP comments based on my own interaactions with faculty. Stdents at my school have some of the same criticisms of facutly that I have. Perhaps students can provide some valid criticisms of faculty or, upon reflection, maybe my judgement is that of a 19 year old. It wouldn't be the first time...
Yes, like others I'm amazed that a self-respecting department would use RMP in candidate evaluation.
ReplyDeleteAnyone can post on RMP; there is zero control that the people posting comments actually took the class in question. As Cass points out, professors are well-known to seed their own pages, as well as their enemies'.
Here is something else: RMP has local "moderators", drawn from either the student body or the faculty. In my case, whoever is moderating my page is either a faculty enemy or one of his student minions. How do I know? Positive comments (from legitimate students; I can recognize an accent) are flagged and quickly removed. A friend of mine has taken an interest in curbing the excesses on my RMP page, and had some of the more outrageous comments removed, on grounds that they made verifiably false concrete claims. Then the "moderator" got aggressive, and "checked" some of the more recent comments (so they can't be removed.) As soon as finals are over, I'm going to visit some local libraries and flood my page with positive comments, in an attempt to overwhelm the "moderator".
So the problem with RMP is that a small number of anonymous posters--who may or may not have gone to any lectures--post garbage that has an outsize impact on a person's reputation, and causes measurable professional/financial damage. I'm always amazed that this doesn't meet the definition of "libel", that it can be done with no legal consequence.
As to Ben's other points: no more than 1/3 of my TT colleagues have 10 or more RMP comments, and the comments bear no correlation with my impression of them as lecturers or teachers. RMP ratings have nothing to do with "bad teaching", they correlate with pandering and low demands on students. As for selecting candidates, I have a broad enough understanding of my area and related areas to just go and read their fucking papers, which is what I do. I can judge for myself how good they are, and that--not candidate statements or LORs--determines my ranking of candidates when I'm on a hiring committee.
This must vary by school, since I'm pretty sure no one is monitoring ours (but TSTSNBN doesn't seem to be all that popular among students at my school. A few professors get a lot of ratings, but I suspect they may ask students to post. Most of us get a few a year -- even if we teach c. 200 students that year -- and those seem to be a mix of a few students who regularly use, and post to, the site, and those who are Really, Really Unhappy, usually because they're "A students" who got a B+. Hence the need, from my perspective, to inject a bit of balance -- because no, I'm not going to encourage students to post).
DeleteI'm not going to go searching for it right now, since I really should be in bed (so I can get up and grade even more papers tomorrow), but I seem to remember that there's a fake profile and set of reviews for a professor with an amazing range of academic specialties, supposedly at a real school, up on That Site, which hadn't been removed as of a year or so ago, despite repeated mentions here, and, I believe, at RYS (and yes this time I do mean RYS). That would suggest less-than-careful monitoring for that school, too.
Fair enough. If we hire for an instructor (no research) and we find the following:
ReplyDelete1. She is converational, funny and confident, and
2. Her RMP comments say that she's entertaining, friendly and delivers a good lecture,
then I'm inclined to say that the RMP comments provide additional evidence that the professor's performance during the phone interview was not a one-time thing. If the interview and RMP comments differ, then maybe it's a wash.
Now that I've thought about it during this discussion, I might be inclined to say that RMP comments can be clearly unreliable but maybe the interview is as well. Think about it: you're judging a candidate's acceptability to work in your deparmtent for several years based on one day of interviews? That seems like an unreliable indicator of success.
You're right about that; I don't think interviews should be decisive in the ranking of candidates, either. We use them to check if the candidate's accent is acceptable, that they can interact socially as civilized human beings (not a given) and to introduce them to the faculty. In the end the faculty vote invariably follows the search committee's ranking (unless the top candidate completely blows his/her talk, which happens); and the committee is made up of area experts, who have contextual information about the lines of research and the candidates' scientific background and postdoctoral training environments. The last time I was involved in a search I had met three of the five finalists at conferences over the years.
DeleteWe don't have this problem for lecturers, since all our long-term lecturers fall into one of two categories: former graduate students who didn't finish a doctorate, but are competent teachers, or "weak solutions to two-body problems" (faculty spouses, male and female ) with PhDs and active in research.
Also, the person who has difficulty making quick connections with people (s)he's never met may turn out to be a reliable, if never scintillating, colleague, while the person who does better in an interview situation may turn out to be flaky or just too much on longer acquaintance. There's definitely a crap shoot element to the whole thing.
DeleteI've never been on an academic hiring committee (just church ones), but it seems to me that you're putting together the best picture you can from a whole bunch of pieces, none of them entirely reliable on its own. I suppose reviews on TSTSNBN might play a part in that picture, but I suspect that trying to read between the lines in recommendations, or the employment history on the c.v., or any of a number of other places, probably makes more sense. And of course what really makes sense, if you can manage it (and if HR lets you) is trying to find a few people who know the person at least a bit in real life, and getting their impressions.
I do have considerable sympathy for hiring committees, since the stakes are pretty high, and the available data is pretty fuzzy. That may be the only upside of the academic hiring process being as unpleasant for all concerned as it is; usually something goes wrong enough-- and/or the process is simply exhausting enough -- that you get a sense of how the candidate will behave under pressure.
Just for kicks, I'm taking a gander at my TSTMNBN ratings. Let's see...
ReplyDeleteOne of the more scathing comments is from the immediately recognizable maroon who thought that because he had an internship at a hamster farm that he knew eeeeeeverything about weaving hamster fur. Yeah, bud...you withdrew from the class once and then got a C the second time through 'cuz you're such an expert.
Oh, look...a snowflake who felt "degraded" because I deducted points for an incorrect filename.
Oooh...this one's good. This particular snowflake dropped my course because s/he was going to get a C but then took the online version, got an A+ on every assignment, and wrote, "Apparently he is doing something wrong..." bwaaaaahahahahahaaaaaa No, fuckbrain...you got an A in that version because the prof who teaches it doesn't actually have a degree in Hamster Fur Weaving, and the class is a joke.
And one of my favorites: "One of the hardest graders I've ever had. Answers questions with no, yes, & maybe. Terrible sense of humor."
Damn, I'm good.
I've flagged comments on Dr. SO's TSTSNBN page as libelous--because they are--and nothing has been done. Actually, something was done once--one of his positive reviews that supposedly couldn't be flagged was removed as I sat there looking at the screen. Sort of a little slap in the face as though saying, "Ha, ha...see? Complain and we'll really fuck you up." Administrators and hostile faculty members use the site as a political tool. High scores aren't anything to crow about since they are indicative of a lack of rigor. It saddens me deeply to think it would be used in any form or fashion to decide someone's fate.
ReplyDeleteIt's all fine to ignore it because you have tenure and/or are popular, pretty or whatever, but it's being used to erode academic freedom. It feeds directly into the remaking of academe into a soulless corporate entity and the de-professionalization of the professoriate.
I've considered forming a deep web site to launch disinformation attacks. Just pick out random pages and send out positive or nonsensical reviews.
EquallyOutragedSO
The effect on academic freedom and on enabling the corporatization of higer-ed are good points. But the broader point is that these concerns, like fairness or credibility, are completely immaterial to the people running RMP; all they care about are click counts leading to advertising revenue. The overhead to run the site must be minimal: it could be a handful of frat boys sitting around a beer keg and typing one-handed (and probably is).
DeleteIt's an instance of the more general problem with anonymous internet rating systems: while some percentage of the ratings is legitimate, they're very easy to game. And they are (or can be made) out of reach of any country's law. But I agree with you: it is high time for a deep web credibility-destroying attack. Maybe I'll start by giving my academic friends a Christmas present.
Actually, something was done once--one of his positive reviews that supposedly couldn't be flagged was removed as I sat there looking at the screen.
DeleteSame thing happened to me. I flagged a negative review for removal. When I checked back a few hours later, the negative review had a green flag next to it (signaling that they will never, ever remove it) and a positive review had been taken down instead.
I truly feel that TSTSNBN is a reflection on two things: personality and grading. Certain personalities can pull off tough grading, but others cannot.
ReplyDeleteI do not feel it is a reflection of teaching ability.
I base these opinions on my own experience and the experience of others , people whose teaching ability I am aware of based on concrete things like teaching observations, student assessment activities, and seeing how successful students are in the next class after having a given professor for the first class.
What I notice (and I do read TSTSNBN because my Dean does and I need to be aware of what she is thinking about my folks) is that there is no certain corollary between high or low ratings on that site and actual teaching ability. But there is a definite corollary between the ratings there and the grade distribution EXCEPT when the professor has a certain personality that is so engaging and non aggressively authoritative that the students just accept the grade they earned a bit easier.
So someone like Ben might want only that latter type of professor, but he can't know, from the site, whether all he is really getting is a high grading slacker.
And really----do we all have to be that nice to be decent professors? I have a smiley face on that stie----but just barely. I tend to get a lot of ratings, and they are evenly split between love and hate. I think I know why----I have a sarcastic humor. I can't help myself sometimes, and it comes out. When that happens, I outrage the students at whom it is directed-----and please very much the ones who have been trying to work around the target snowflake. Over the years, I've worked hard to reign in that sarcasm. I meditate, I do the square breathing thing, I count to ten. But sometimes, ....I slip.
I remember Dana from Decatur----anyone else remember her? I loved this post "I can work with stupid." (here it is: http://rateyourstudents.blogspot.com/2010/03/welcome-blast-from-beloved-dana-from_5203.html ) Anyway, stupid is safe from me. I have endless patience with someone who just does not understand. But the others....oooooh. Well, so they get me on TSTSNBN. Oh well.
Ah, Dana. I remember and miss her presence. Someone needs to find her and dragoon her to CM.
DeleteI love that post. Love it. I wish I could say that to some of my students.
DeleteBrilliant!
DeleteAlso, whatever happened to the tradition of linking to semi-related stuff from Amazon in posts?
The Amazon joke-link it is called in the mod handbook. I usually don't do it.
DeleteOne of the mods corresponded with Dana in the past year but we were not able to get her back. She is terrific.
DeleteThat's an interesting insight about student evaluations in general: it's not just grade distributions, but the combination of grades and perceptions of personality: it could be (based on some of my RMP comments) that I'm perceived as "out-of-sight authoritative", and I make no conscious attempt to tone that down. (And, thinking of CC's "gentle giant" comment, I'm not that: I'm 5'11'' but slim, with a runner/swimmer body type. Seriously! Doesn't seem to help.)
DeleteThere's another component that's rarely talked about: instinctive student reactions to the instructor's national origin. Students are a sample of the general population, so a nontrivial proportion of them are (possibly subconscious) xenophobes. You can be a hard grader, but not if you're "not from here". This is specially a problem in math where, for instance, in my dept 2/3 of the TT faculty is foreign-born. If you look at the RMP ratings for the 12 of my colleagues with at least ten comments, those with higher ratings are all US-born. And yet the comments rarely include complaints about accents. Accents are not the real issue, that's just a proxy to avoid non-PC comments about national origin (or skin color: among the foreign-born, Germans and Brits get higher ratings.)
Hi Peter! I really do feel like it's a reflection on personality and grading, and also that a person's cultural background influences their personality quite a bit. Of course, prejudice plays a role too, and profs with a strong accent tend to get poor ratings (even in the "legit" evals). I am wondering, though, if cultural norms might play some role, too. What I mean is, a prof from a country with certain expectations for behavior between prof and student would fare worse, ratings wise, than one who truly understood the students' cultural preferences. I really feel that your.method of showing that you are in authority plays a huge role in getting students to gracefully accept your policies, for example. So if the prof is showing authority in a way that is culturally acceptable in their country but comes off as arrogant or aloof here....that ends up getting lost in translation.
DeleteI still feel, though, that while personality can help a ton, students are responsible for learning from a variety of personalities, not all of whom they are going to love. Suck it, snowflakes, is what I say. Neither I nor the profs in my department are here to make you comfortable. If we do, that is a bonus, but we are here to teach you something.
"EXCEPT when the professor has a certain personality that is so engaging and non aggressively authoritative that the students just accept the grade they earned a bit easier."
ReplyDeleteIndeed. I know a few professors who meet that description, and they are, indeed, very good teachers. They're also all male, and fairly large -- sort of a "gentle giant" phenotype/personality combination. I'd love to meet that description myself, but even if my personality were more engaging (something I could work on, at least to an extent), I'm not sure that a short, round female can pull off the "non-aggressively authoritative" bit. I'm hoping increasingly gray hair will help, but students' preconceptions about what legitimate authority looks like play a big role. Maybe as we get more female legislators, CEOs, Supreme Court justices, presidents (?!?!), etc., preconceptions will change? One thing we've got going for us as college teachers is that most of our charges are fairly young, so the last 10-15 years looks like longstanding precedent to them.
Oops. This was supposed to be a reply to Bella.
DeleteHi there Cassandra! You know, I am not sure what part being a "gentle giant" plays----probably some part, indeed. Physical presence does mean a lot. However, when I was writing this, the person I was thinking of first and foremost is a female in her late 30s. Attractive. And commanding as hell. A second person who comes to mind who sort of fits this description (although he tends to grade somewhat higher than I think he should) is 5'8" and around 150 pounds. And scruffy looking! Damn, is that dude scruffy looking. But when he starts talking, all you can think of is this guy knows his shit. He's interesting as hell, commanding as hell, and he makes them write a ton. A ton. And they don't complain. I make them write and they whine and whine and cry and complain. They don't do it to him. I want what he's drinking!
DeleteWhat I always want to know is, as a grad student, why I'm still subjected to RMP. I feel that as a student I should be protected under FERPA and that I shouldn't have my name up on such a site. Anyone care to comment on this? Anyone ever had their RMP page taken down on the grounds that they are protected by FERPA? The TA who emailed homemade porn to her students was apparently protected by FERPA, and that's why the university couldn't release her name.
ReplyDeleteAs for everything else: I agree with Cassandra. I doctor my own reviews; I think I've written maybe half. What always surprises me is how open to suggestion my students are. All I had to do was say a few nice things about myself, stress my tough-but-fair grading policies, rattle off a few references to how "cool" I am, and the other reviewers followed suit. Now the same language ends up recycled in my university evaluations.
My TSTSNBN page stayed on the site for several years after I quit my teaching position. It gave certain people an opportunity to trash my reputation, some of which cited incidents which happened soon after I started at the institution. Oddly, one comment was for a course that I never taught, making me wonder who actually wrote it and why.
DeleteThis reminds me of a couple of mysteries surrounding TSTSNBN. Once in a while I'll look for a colleague there (just out of morbid curiosity), only to find out that an entire department is missing. Uncovered, no presence. And I'm not talking obscure places: major universities, often with other departments covered. My only theory is that a dept chair or dean got a lawyer to write them a convincing "cease and desist" letter, and they complied.
DeleteHere is the other mystery: recently we made a young man an offer, and after he had accepted the position I decided to look him up on TSTSNBN. After all, he's been teaching at three different places around the country for the past six years; his vita lists a whole range of the typical undergraduate classes. Guess what: no track at all on the site, at any of the three schools. Again, sounds as if somehow he managed to have all his reviews removed. My only theory: he hired a professional to take care of it (whether the legal kind or the Strelnikov kind, who knows.)
By the way, QWV, welcome to the page. Hope you have fun.
DeleteI have written all of the comments/ratings (except one) on "my" page on TSTSNBN--both good and bad. I don't pull any punches and I don't suffer fools gladly, even if they are I. Preemption is my philosophy. And I like to contemplate how I could be better as a proffie. Also, occasionally a colleague will say, "Hey, you really got burned on TSTSNBN the other day." To which I reply something like, "Oh? Well, I'll have to take a look at that. They can be tough little bastards, but I love them."
ReplyDeleteOnce, I even contacted TSTSNBN and angrily told them one of the comments was libelous, so they removed it.
DeleteYou're killing me, Bubba!
DeleteRMP is, and has been for a while, a joke. I know a few truly terrible ones - one who failed the first year class in a row, two years running - who have better than a 4/5 on the site. In my own course, I just got hit with two bad reviews to the effect that the course was confusing, notes were poor - in other words, complete nonsense. Some people didn't seem to know that we had a textbook - you know, the one all the notes were coming from - and complained that they cried because things were too hard. In a class with a 78% average. Okay.
ReplyDeleteThe fix for RMP is in and has been for ages: a few anonymous ragers mixed with professors writing their own reviews (and, for those doing it: I can tell when it's you writing it, BTW). I'm not sure why it's an issue up for discussion: the situation suggests a class-action lawsuit, frankly, and I'm equally surprised this hasn't been done already.
A lawsuit means you have to prove your claim that the posts are false, which would mean an audit of your lessons, grades, lectures, remarks and feedback to students, student interviews, etc. Are you sure you can survive it? Chances are the results would lead to many teachers getting suspended, reprimanded, or sued privately.
DeleteCome at me, bro.
Delete