Student course evaluations, given out at the end of every semester to gauge a teacher’s performance, are a key factor in determining whether a faculty member is worthy of promotion to a higher level of tenure. Roughly 49 percent of SF State’s faculty members that are in tenure track, or have full tenure, stand to benefit from larger salaries, health care and retirement if they receive positive student course evaluations.“The purely quantitative comments are more troublesome because it’s not always clear to students the effect and the use of the data that’s being made,” Green said. “Often students have a tendency to take these evaluations lightly as one more thing that’s wasting their time.”
Natalie, a senior psychology major who declined to give her last name, has been through four years of student course evaluations and feels apathetic about them.
“I don’t really care about student evaluations,” Natalie said. “It’s just something I have to do. I fill in the bubbles and get it over with.”
MORE.
My last school counted student evals at 50% for tenure and promotion. That's insanity. My students, like one in the article, didn't give one second of thought to these documents. Some, I know, bubbled all great numbers. Some bubbled all bad numbers.
ReplyDeleteAt my new college, student evals only count 20%. I think that's okay. I don't mind some accountability to the 18-19 year olds who are in my classes. If a big number of them think I'm doing a lousy job, maybe there IS something to learn from that. But it can't be more important than anything else I do for the college.
I tell my students up front, as I hand out the evaluations: Your ratings of this class are very important, and have a direct influence on my employment here. Be honest, but be fair.
ReplyDeleteOr in other words: Look kids, it's not a fucking game for me.
For many of my students, evaluations were simply used as a means of revenge for not getting the marks they believed they deserved. If those evaluations had ever been influential in my getting permanent status over 20 years ago, some would have gleefully done a hatchet job on me.
DeleteIt's just another way of administrators avoiding having to make a decision and take responsibility for it.
I NEVER give students any inkling of how evaluations are utilized. I don't want them to feel they have any kind of power over me. Thankfully, at this point in my career, they don't.
DeleteEons ago, evals were retained by my dept chair. They were discussed and used constructively to help me teach effectively. They were not used for promotion/reappointment assessments.
Now, they are a string factor. But I know that faculty known for giving A's get high evals. I tend to be tougher and I am not as popular. However, students fare well in internships and in having the skills they need to seek employment after graduation. So I'm doing something right. But student being able to rate me on my sense of humor, handwriting, or similar considerations is useless to me. Students do tend to rate me high on knowing the material. THAT matters to me. Students generally resist being pushed beyond their comfort levels, and evals are a way they take revenge of a sort.
I pay very little attention to evals any more. I don't find them useful or instructive, and the use of them in promotion and reappointment should be tempered by the larger scheme of things.
40% of T&P where I am. I taught someplace where 5% was given for research and scholarship, and they always told us to write and research and teach and be collegial, etc. Of course at that same place 60% was given for student evals.
ReplyDeleteOne of the things that drives me batty about our system is that there is no N/A option, and neutral is counted as bad. So if a well meaning student has never attempted to come to OH or e-mail the professor they might not have an opinion on whether the professor was available outside of class, so they bubble neutral. What the university thinks they've done is give the prof a 3/5 rating on their availability.
ReplyDeleteDoes anyone know what kind of statistical analysis is done with quantitative evaluations, like tossing outliers? I'm guessing that at least certain departments wouldn't have a great grasp of statistics and would misuse the data.
ReplyDeleteOne possible trick that I used, and I'm not sure if this really worked, was to prime the students with my own "supplementary" evaluation composed of questions that I made up. The last two, which they answered before starting the department's official evaluation, were "What could I as your teacher have done over the course of the semester to help you become a better [writer/student/whatever made sense for that course]?" and, more importantly, "What could you have done over the course of the semester to have become a better [writer/student/whatever made sense for that course]?" I don't know if forcing a little introspection before the official evaluation made any difference, but it likely couldn't have hurt.
Shortly after I started teaching, the president at the time decided to meet with the instructors, one section at a time. When our turn came, there were some who unloaded on him about evaluations. (Back then, which was over 20 years ago, they were still used as a means of finding out what worked and what didn't and had little influence, if any, in whether an instructor stayed on after the 2-year probationary period.) Some of those instructors were rightly upset that students had the idea that they could have someone sacked and used those evaluations for that purpose.
ReplyDeleteThe backlash about the evaluations caught the president off guard and he never held such meetings again.
Then the student-as-customer doctrine was inflicted upon us a few years later and things went downhill after that. Unfortunately, those evaluations were given greater weight in such things as the annual performance appraisals and whether or not someone should be kept on.
The fact is that they weren't terribly useful and they were rarely used for their intended purpose. Students with an axe to grind against a particular instructor used them to get even. Administrators who wanted to boot someone out gladly used them to get rid of that purpose.
The whole evaluation system in that institution was, to put it mildly, a joke.
When I was a graduate student, our department required the students to sign the evaluations in order for them to go into the instructor's permanent file.
ReplyDeleteWhen I started teaching, there were 2 types of evaluations form. The more common sort were handed out by the instructors but they were anonymous. The other variety was a different colour and was used at the discretion of the department head. Those had to be signed and were rarely used, usually if the head had some concerns about a particular instructor.
DeleteOne advantage with the second type of form was that the comments were almost always honest and factual because, by signing them, the student could be held accountable.
Many years later, I noticed that the latter type weren't being used any more. I never figured out whether they were phased out because they weren't used all that often or just simply abolished.
The result was that we were stuck with one type of evaluation form. That, of course, meant the students could say whatever they felt like because they knew they weren't going to be held accountable--and they often did. My department head loved them because, in his warped logic, the opinions expressed in them would be more "honest" (ha!). That meant that if he had a beef with an instructor, the students could do his dirty work for him, even though their comments likely couldn't be verified.
My biggest problem with them is that they are counted (or not counted) capriciously. My experience has been good overall (higher than my department average every single semester while I was on the TT, including in the "Learned a lot" metric, which students can use to f*ck up your rating by answering "1"). I actively encourage my students to fill out the comment section, and many of them do. Some of their complaints have led to changes in the way I do things in my course; some just made me laugh (like "This course meets too early"--as if I have control over that).
ReplyDeleteThe problem came when it was time to look at them for retention and promotion. Some people on the committee gave them weight appropriately, while others argued that they shouldn't weigh as much because "they're just popularity contests." It led me to wonder whether the folks who didn't like evals were the ones with lower-than-the-department-average numbers. And it also leads me to question how many of my colleagues intentionally "go easy" on late-term grades to avoid low numbers on their evals.
It's far from a perfect system, but since we're here to serve the customers, we're going to be stuck with it.
Feinstein understands capricious knowledge.
ReplyDeleteWe're currently at c. 20% student evals. for retention/promotion (such as it is)/raise percentage (if -- hah! -- applicable) for non-tenure-track, teaching-intensive faculty. The percentage is a bit lower in years when someone observes our class, and there's an observation letter to throw into the mix. The rest of the evaluation rests on a review of syllabi, assignments, and sample graded work. None of the above are perfect measures, but, in aggregate, I think it's a workable system. Of course it's easier (though not easy) to evaluate research, but we need to find a good way to evaluate teaching, and it can't be mostly student evals (though, like Darla, I have no problem with student evals forming one part of the picture).
ReplyDeleteHowever, my anecdotal evidence suggests a pattern along the same lines as that described in the article: at a time when everything from buying a burrito to buying something on ebay or Amazon seems to involve a rating system (and retailers are offering prizes for filling out surveys, and online sellers are emailing to ask for the highest rating or nothing), students just don't take the process very seriously. My whiniest student several semesters ago, to my delight, asked "do we have to fill these out? What happens if we don't?" She didn't even seem to realize that this was her chance to make her complaints felt, at least a bit (but perhaps complaining was just her hobby). As we transition to an online evaluation system, we certainly aren't having much success getting students to fill out evaluation forms on their own time, which suggests how much they value the chance to provide input. Unfortunately, the response percentages my school is willing to accept as valid are way, way low. I'd suggest throwing out anything under 50-75%, in a single class, or, for those of us who teach many sections of the same thing, for the semester as a whole. If that means some of us have no evals, so be it.
And then there's this:
ReplyDeleteNumbers compiled by Education News say assistant professors stand to make $71,681 annually, whereas associate professors make $81,445 and fully tenured professors make $98,510.
Lecturers, who cannot advance to a tenured position, are said to make $62,605.
It's not clear whether they're trying to provide figures for San Francisco, or nationwide, but, either way, these numbers strike me as way high compared to those in my department, and I live in a high cost-of-living area -- probably pretty comparable to SF if one takes into account the likelihood that lower-paid workers are commuting from somewhat cheaper communities in the same general area. Of course, they also seem to be a bit unclear on the concept of how tenure and the various ranks interact (or don't).
Where do they get these numbers, anyway? I really hope nobody is heading to grad school on the basis of such figures, and I have to wonder whether the ones for other professions -- e.g. nursing -- are equally deceptive.
I'm with No Longer an Academic and Academaniac--I don't like to think about empowering students any more than they're already empowered. I figure that students will do with evaluations what they please, and I don't want to give any additional motivation to lie or trump up charges on an evaluation. There are already a lot of students who feel that they have the power to get an instructor fired--I've often overheard them talking. "I'm going to get her fired come evaluation time!" or "He didn't come back after last semester--it must be because of the terrible review I gave him!"
ReplyDeleteI've seen students flat-out lie on my own evaluations. Luckily, most of my evaluations are fine, and I'm a TA, so they won't get me terminated from my job (though they do have to go into the teaching portfolio, ugh). But I'm always somewhat sickened by the few students each year who completely fabricate charges--"never held office hours," "never answered emails," "provided no feedback on my paper--just gave it back with a grade and no comments," "didn't address the material on the syllabus," "led the class on wild tangents that had nothing to do with the material," "was always late to class" (I *always* arrive five minutes early to class), etc. These students have the impression that they can fuck up someone's career--probably because they reaped the benefits of making wild complaints somewhere else in life.
Luckily, most students are either benevolent or lazy, so they don't put too much effort into evaluation sabotage. I prefer the type of student mentioned above--the one who simply scribbles "good class" because she figures that no one will read the evaluation--to a student who feels all kinds of empowered to determine the career trajectory of the prof who gave her her first B.
I recently ran across a non-traditional student, someone my age, who told me with a little gleam in her eye how she'd gotten an adjunct fired. She didn't know that I'm a TA at the same school where she's attending. "Oh, I got her fired. I went to the department and made a complaint. Then I gave her a terrible evaluation."
"What did she do that was so bad?" I asked.
Her: "She was boring. She couldn't keep the class interested. And she didn't know what she was talking about. I worked in the field [of social work] before, and she didn't know anything."
Me: "Oh. You're a social worker?"
Her: "No. But I know more about social work than she did!"
Me: "Oh. Well what else did she do?"
Her: "She got mad that no one paid attention in class, so she made us put away our laptops and sit in a circle. Like we were in fourth grade."
Me: "Hmm. All my professors in undergrad made us sit in circles, and none of us had laptops, so ...."
Her: "I'm paying SO MUCH MONEY." [A few thousand a year.] "I deserve better instruction than that."
Me: ....
Her: "I got her fired. She's not there anymore, so it must be because of my complaints."
While I was an instructor at the tech school I used to be at, it was students like the one you described who were highly prized by certain department heads. Comments by such students, ill-informed and ill-intentioned they may have been, were used as irrefutable proof that someone might be a poor instructor.
DeleteNo further investigation was necessary.
While I've generally had good experience with non-traditional students, sometimes there is a reason beyond life circumstances and laudable determination why somebody did not finish college at the traditional age, but remains convinced that they should, and will, do so. This woman's statements sound to me like they might reflect a personality disorder (though it's admittedly a bit hard to tell, what with programs to bolster artificial self-esteem and encourage people to see themselves as successful consumers; still, some people are more susceptible to embracing the worst extremes of such nuttiness than others). This kind of attitude is, of course, a good reason not to pay too much attention to any one student, or evaluation. In theory, the numerical nature of evaluations should make it easy to ignore the opinions of obvious outliers, and get a sense of the average student reaction to a professor -- in a particular class, a particular semester, or over large blocks of time. Even then, there's the question of what the evals are actually measuring (there's plenty of evidence that factors like gender, race, age, and physical appearance play a role). They probably could work to identify the few proffies who are having real problems with the majority of students, but that's about it (and somehow I suspect that department chairs know about those proffies anyway, and not because a single student complained).
DeleteMost of my non-trads have been great, but a few have somewhat unrealistic expectations about the obligations that instructors have to students. Some are coming back to school after working in customer service for several years, so they don't understand why you aren't available day and night to "troubleshoot" some problem.
DeleteI also had this one who wrote very specifically on my evaluation that I needed to smile more. "Needed to have a smile on her face. Needed to be warmer and more attentive to student concerns." She wasn't old (late 20s), but she had worked for a few years as a receptionist at a plastic surgeon's office, and I got the feeling that she was spitting back at me the kind of corporate ethos she'd been subjected to--always make patients feel good about themselves, always offer them coffee and tea and water. (Her evaluation was difficult to ignore because it was so detailed. It might have been an "outlier"--I'd never before and have never since had a student complain so much about my interpersonal skills--but it was convincingly and cogently written and therefore somewhat scary.
And I did have an administrator who used to bring "outliers" to my attention. She would sort of dismiss them in the same sentence, but I still found the experience rather disconcerting. "One student said that you provided absolutely no feedback on assignments. Based on the rest of the evaluations, it seems that this person had unrealistic expectations regarding grades, but I was just letting you know in case there's anything you can do to improve your feedback ..."
This comment has been removed by the author.
DeleteGone Grad:
DeleteI once had an older student who, like the ones you describe in your first paragraph, wasn't happy that I didn't give him the time and attention he felt he deserved.
I was teaching a CAD course and barely knew enough of the software to be able to teach it. That student was in one of the sections I taught and I had about 20 people in each of them. I was busy running from one station to the other helping whoever was there. (I could almost guarantee that after I showed the group what to do that about a quarter of my students would quickly put their hands up.) There was bound to be someone who I either wasn't going get to or would spend more time with. I guess, one day, he was one of those people and decided to complain to the department head. I wouldn't be surprised if he was one of those who trashed me in his evaluation.
I only conducted evaluations for only 2 courses each term, as that was what I understood I was required to do. Unfortunately, I had to waste some of my course time on them by doing one about mid-term and a follow-up near the end. Was that all I had to do? Nope. I had to "share" the feedback for the first one with the students, thereby wasting even more time. I'm sure that some of my colleagues turned that "sharing" into a warm-and-huggy session.
Doing all that malarkey made me wonder just what my job was: instructor or feedback improvement artist.
During my Ph. D. residency, I was a TA in a course for a term but after the course was over, unknown to me, I was bumped to the bottom of the list. I brought it to the attention of the department's grad student advisor, who didn't seem upset about it. He informed me that there had been a number of negative comments about me and that the department head decided that I didn't deserve another assistanceship. That decision, I assumed, was based on student evaluations, the results of which I never nor was I given an opportunity to tell my side of the story.
ReplyDeleteFortunately, the following academic year, I was back on the list, largely because that wishy-washy advisor was succeeded in that position by my thesis supervisor. He assured me that the way my situation had been handled earlier wouldn't happen while he was in charge.
I hated the student evaluation system at the place I used to teach at because it was a sham.
ReplyDeleteIt didn't accurately indicate an instructor's effectiveness, only his or her popularity. Those who gave out high grades like candy or didn't make any great demands of the students scored the highest and were, in the minds of the administration, excellent at their jobs.
The process wasn't entirely anonymous, either. I heard a story about a colleague who, while he was head of a different department, would often go through the envelopes into which the evaluations were placed prior to submission for processing. He, apparently, would look at the evaluations before they were sent off and remove the ones that were unreasonably negative. I also long suspected that my last department head, who long held a grudge against me, snooped through mine. For all I knew, he might have added some of his own.
The fact that the evaluations were, for the most part, anonymous was often used to initiate disciplinary action against an instructor without any investigation. The students could write anything about someone, no matter how nasty or frivolous, and the person in question was never allowed to know who was responsible. The justification for that anonymity was to prevent an instructor from getting even with them. Never mind that in my country, one has a constitutional right to not only know who their accusers are but to face them as well.
The evaluation system, frankly, was used for anything others than course improvement.
The justification for that anonymity was to prevent an instructor from getting even with them.
DeleteI've never understood this reasoning. It pretty much assumes the worst of professors--that we're childish and vindictive, and that we'll savage someone's grade out of revenge. Students are looked upon as selfless whistleblowers, while professors, who stand to lose the most in the process, are told they can't even be within a fifty-foot radius of the classroom after evaluations are distributed.
But even beyond that, I don't understand why anyone who would complain so vociferously against a prof would take another class with the same professor--unless we're talking about an extremely small school. But even so, I went to a small school, and I could easily avoid the professors I wanted to avoid.
My last department head didn't like that I didn't conduct evaluations for all of my courses, largely because he wanted to snoop on me. Just about every academic year, at around the middle of the first term, he would approach me quietly and say, "There's been a complaint about you." When I asked for more details, he would become evasive and refused to give me a straight answer and that's pretty much where it stayed. He didn't bother investigating the matter any further. After a few years, I ignored the issue as it obviously wasn't serious.
Delete