 |
| To this day, one of the finest and most subtle graphics Cal ever made. He must have been "not drunk." |
Using “impact factor” to “evaluate” publications is hurting younger faculty, especially working in emerging fields who are publishing in the IDEAL journals for their research. Important, groundbreaking, peer-reviewed essays that introduce the world to new ideas, approaches, subject matter, etc. are being questioned and at times “disallowed” for “low impact factor” of the journals
Does Dr. Dumbass even know what an impact factor is? And isn’t that about the journal, not the individual essays? What the fork has he produced? In a single year, my junior colleague has matched his entire production for the last ten years. But to be fair, one finds it difficult to have much time for anything other than capriciously judging and dismissing the work of underlings.
One journal in particular will not send out essays to its reviewers until AFTER the editorial board decides whether the essay is worthwhile, appropriate for its readers, interesting. So its acceptance rate is high. Yet the best venue for my colleague’s work is “not very tough to get into.”
We already reduce student evaluations down to numbers. (Freaking SIRs!) But now publications?
Does Dr. Dumbass not see the name of the school on the sign that cost several thousand dollars near the reserved parking spot he enjoys? To have such a promising scholar here - and he wants to know this?
So why don't you tell this to Dr. Dumbass? On Monday, I flat-out told a Program Review committee to their faces that assessment was pseudo-scientific nonsense the only apparent purpose of which was to give a bloated administration something to do. I thought I was going to be punished for it, and maybe I still will be, but yesterday they told the upper admin that our faculty do need release time for assessment, the way they do at other schools in our system. More surprisingly, they also tried to convince the admin to support a new faculty hire we want---not that I think it's likely we'll get it, with the admin crying poverty, as usual. But then, muggers won't mug you if they think you're crazy.
ReplyDeleteAnd of course, I have tenure.
Dear provost,
ReplyDeleteThe impact factor and acceptance rate of these journals are more than appropriate for the emerging type of research that Junior Colleague is doing. You have my word on that as a professional and expert in that (or closely related) field.
Sincerely,
Don't Fuck With Me.
Bravo.
DeleteAll this impact factor stuff is a sneaky way to give admins more control in hiring and tenure decisions. We won't be fooled.
Bravo, indeed.
DeleteJust tell the administrator who is tormenting you that at any university where this matters, doing research is much like that little dog that rides a bicycle in the circus. It's not that he does it well: what's remarkable is that he does it at all.
DeleteHow about, for demonstrating an impact factor that is truly meaningful to the provost, "donate" his car for Beyonce's next video?
DeleteAdministration by spreadsheet, because who the @#$! has time to understand what faculty actually do? Just give us something, anything, that can be reduced to a number, even if the processes that produced that number are complete bullsh!t. Because numbers don't lie, and can be compared (even if they resulted from counting apples, ozone particles, and wombats -- and the wombats ate most of the apples while the count was underway).
ReplyDeleteAll of this, mind you, comes from the same administration (broadly speaking) that is busy soliciting as many applications as possible (and otherwise skewing the data) so that it will rate as "selective" as possible in US News & other rankings. And then complaining about how much harder it is to predict enrollments now that students are applying to so many colleges.
Aargh.
Cassandra, this is one of the few times I can recall that you've appeared mad as hell and not going to take it anymore.
DeleteI dug up some links, which I'll (hopefully) curate and post in the morning. Long story short, journal impact factor is subject to gaming and skewing just as is admissions selectivity, and it was never intended to measure an individual paper or author.
Bullshit indeed.
I'm with her, meaning Cassandra.
DeleteI've tried commenting twice with links as promised, but even though I see it published, when I return to the page, it's gone. I'll try again.
Delete(Dear possible spam filtering algorithm, the following is not spam.)
DeleteSick of Impact Factors
Scientists join journal editors to fight impact-factor abuse
Impact Factor Distortions
Impact Factors: A Broken System
How science can beat the flawed metric that rules it
Do prestigious science journals attract bad science?
Hate journal impact factors? New study gives you one more reason
Major Science Journals Eliminate Impact Factor
As I understand it, Impact Factor was intended as an aid to librarians. It answers the question, "given two journals in a particular field, but a budget to subscribe to only one, which would serve my readers better?" The journal that is cited more is more likely to be sought out by scholars looking for primary sources, so that's the one to get.
It says fuck all about quality, at least not directly, in the same sense that McDonalds selling billions of burgers doesn't make it a better restaurant than my local pub. And even Wikipedia points out the problem of comparing across fields.
Yeah, Provost Doctor Dumbass probably thinks people will be impressed by his use of Impact Factor, because he's being all rigorous and "sciency" and stuff. The result is quite the opposite. And we should be mad as hell and not take it anymore.
OK, third time was not a charm. It was there for a minute or so, now it's gone again. I suspect an anti-spam Blogger bot.
DeleteI'll try once more, with fewer links.
DeleteHate journal impact factors? New study gives you one more reason
Major Science Journals Eliminate Impact Factor
As I understand it, Impact Factor was intended as an aid to librarians. It answers the question, "given two journals in a particular field, but a budget to subscribe to only one, which would serve my readers better?" The journal that is cited more is more likely to be sought out by scholars looking for primary sources, so that's the one to get.
It says fuck all about quality, at least not directly, in the same sense that McDonalds selling billions of burgers doesn't make it a better restaurant than my local pub. And even Wikipedia points out the problem of comparing across fields.
Yeah, Provost Doctor Dumbass probably thinks people will be impressed by his use of Impact Factor, because he's being all rigorous and "sciency" and stuff. The result is quite the opposite. And we should be mad as hell and not take it anymore.
Try with a few more links:
DeleteSick of Impact Factors
Scientists join journal editors to fight impact-factor abuse
Impact Factor Distortions
Impact Factors: A Broken System
How science can beat the flawed metric that rules it
Thanks! Interestingly, this doesn't affect me directly (mostly because research isn't part of my job, but also because I'm in a humanities discipline, and so made it through grad school and quite a few years of teaching without ever hearing about impact factors; I do have the sense, however, that this, along with the tendency to measure research success by grant dollars won, is affecting my TT colleagues). I do, however, try to keep up with the lingo of science/social science publishing, the better to teach writing in the disciplines.
DeleteFrom that perspective, the only *possibly* good thing I can see about impact factors is that they might help weed out some of the junk journals whose contents my students are more and more often lighting upon (this is particularly a problem when the journal's editing is bad to egregious to nonexistent, and both the article author and English is not the first language for either the article author or the student. Even if the content is basically okay, the whole business of learning to write by reading good models goes entirely out the window.)
But there are other ways to deal with that issue -- e.g. lists of predatory journals, or just sticking to well-curated databases and/or journals recognized to be high-quality (whatever their impact factors). Librarians -- actual human ones, who talk to other humans -- are, of course, crucial to making judgments of this kind.
And I am, just on general principles, annoyed by the trend toward trying to quantify everything (even as I increasingly suspect that I should go back and take statistics one of these days). I don't know enough about impact factor to critique it intelligently (presumably I'll be better at that after I actually read those links), but I do know that data-gathering methods (especially those involving surveys/questionnaires, which employ -- gasp! -- language,with all its nuances and pitfalls and so on) tend to affect responses, and, ultimately, the numbers that come out the other end (with, yes, student evaluations being a major example).
Sick of Impact Factors
ReplyDeleteScientists join journal editors to fight impact-factor abuse
Impact Factor Distortions
Impact Factors: A Broken System
How science can beat the flawed metric that rules it
Do prestigious science journals attract bad science?
Hate journal impact factors? New study gives you one more reason
Major Science Journals Eliminate Impact Factor
As I understand it, Impact Factor was intended as an aid to librarians. It answers the question, "given two journals in a particular field, but a budget to subscribe to only one, which would serve my readers better?" The journal that is cited more is more likely to be sought out by scholars looking for primary sources, so that's the one to get.
It says fuck all about quality, at least not directly, in the same sense that McDonalds selling billions of burgers doesn't make it a better restaurant than my local pub. And even Wikipedia points out the problem of comparing across fields.
Yeah, Provost Doctor Dumbass probably thinks people will be impressed by his use of Impact Factor, because he's being all rigorous and "sciency" and stuff. The result is quite the opposite. And we should be mad as hell and not take it anymore.
Sick of Impact Factors
ReplyDeleteScientists join journal editors to fight impact-factor abuse
Impact Factor Distortions
Impact Factors: A Broken System
How science can beat the flawed metric that rules it
Do prestigious science journals attract bad science?
Hate journal impact factors? New study gives you one more reason
Major Science Journals Eliminate Impact Factor
As I understand it, Impact Factor was intended as an aid to librarians. It answers the question, "given two journals in a particular field, but a budget to subscribe to only one, which would serve my readers better?" The journal that is cited more is more likely to be sought out by scholars looking for primary sources, so that's the one to get.
It says fuck all about quality, at least not directly, in the same sense that McDonalds selling billions of burgers doesn't make it a better restaurant than my local pub. And even Wikipedia points out the problem of comparing across fields.
Yeah, Provost Doctor Dumbass probably thinks people will be impressed by his use of Impact Factor, because he's being all rigorous and "sciency" and stuff. The result is quite the opposite. And we should be mad as hell and not take it anymore.
I co-wrote an article with a colleague in Europe. He's really sensitive about the Impact Factors of whatever journal we pick for submission. I guess his school really pays attention to it, and he was surprised my school doesn't seem to care.
ReplyDeleteAh, but there is a cure for this, at least in my field. Point out to the university administrator who is tormenting you that the journals with the highest impact factors have PAGE CHARGES. That will be the end of that.
ReplyDeleteNot if the page charges come from your grant. He won't care unless they come out of *his* budget.
DeleteThis comment has been removed by the author.
DeleteBut if you have grants, they tend to forget all about impact factors.
DeleteThanks for the comments.
ReplyDelete