Saturday, February 23, 2013

Well, Duh (but good to have it confirmed, I suppose)

Courtesy of Inside Higher Ed:

Some Groups May Not Benefit From Online Education
Some of the students most often targeted in the push to use online learning to increase college access are less likely than their peers to benefit from -- and may in fact be hurt by -- digital as opposed to face-to-face instruction, new data from a long-term study by the Community College Research Center at Columbia University's Teachers College suggest. . . .Students of all types completed fewer courses and achieved lower grades online than they did in face-to-face classes, men, African-Americans, and academically underprepared students had the biggest gaps between the two mediums.
Full report from Columbia's Community College Research Center here.  

This is not a surprise, of course, to those of us who have taught online.  It's also worth noting that these were probably relatively small, labor-intensive online classes, not MOOCs, which, as far as I can tell, would work even less well for these populations.


  1. Who would have thought this could happen?

  2. The give-and-take of a normal classroom interaction, with instructor and students asking and thinking about unplanned, unpredictable questions, is essential to the learning process and does not "scale up". It doesn't even scale up from a group of 20 to a group of 100 students in old-style classes, let alone to online instruction (which, thankfully, I have no direct experience with.)

    I still don;t understand the difference between asking a beginner to watch a canned lecture and telling the same person to read a chapter in a book, then test them on the content. If they can't ask for clarification in real time, isn't it the same thing? In math, a motivated student can learn independently fairly elementary material, but as soon as it gets a little more advanced, you lose a lot (contextual, peripheral information) by not having real-time contact with an expert.

    But, as somebody said in the NYT two days ago, online delivery is "crack for administrators": the financial incentives are transparent, and have nothing to do with learning.

    1. I am so off the charts tired of elevation of classroom "give-and-take" into being some sort of sacred, universally validated, exclusively successful mode of education.

      Jeebus H on a cracker ... isn't this blog overflowing with examples of just how DISengaged students can be in a classroom?

      No, I can't disagree with the "crack for administrators" comment, but again, isn't this blog replete with examples of how administrators are clueless as to how authentic learning is accomplished in classrooms? (I was actually told by an administrator that a "good" teacher should be able to teach any subject!)

      I have successfully learned and taught both online and on campus. Hell, I even have learned (and earned college credit) via videotaped lectures. I have even -- gasp -- learned via a truly old school correspondence course.

      (Anyone remember the New York Institute of Photography? They still exist, but no longer rely on mailing of completed assignments and returning of instructor commentary on cassette tape as they did when I took the course.)

      You "thankfully have no direct experience with" online learning?
      Puts you rather in the perfect position to have a totally uninformed opinion, now doesn't it?

      The Columbia study (upon which the NYT article was based) focused on community college students which -- again as we have learned here -- generally need much higher levels of remediation and hand holding. Under those conditions, this veteran online learner and educator would agree, those students are not likely to succeed with online courses.

      On the other hand, for average to above average students with reasonable self-regulation and time management skills, online programs can offer education which might not otherwise be accessible.

      Ultimately though ... online or on campus ... there is an unpredictable flake factor which can undermine even the most interesting and engaged course.

    2. "I still don;t understand the difference between asking a beginner to watch a canned lecture and telling the same person to read a chapter in a book, then test them on the content."

      There isn't one -- and, as Frod points out below, alternative delivery methods have been tried before, with only slightly different technology (there really isn't that much difference between a filmstrip, various forms of TV/video delivery, and video streamed via the internet).

      The internet does, however, allow for forms of non-synchronous interaction that can provide some of the same benefits as in-person interaction. No, responses aren't instantaneous, but it is possible to let students discover things for themselves, make mistakes, ask questions, etc., and to patiently steer them toward greater understanding, much as one would in a classroom. However, it's tremendously labor-intensive (why am I on the internet on Saturday afternoon? Because that's when many of them have time to do the work, and are asking questions, and/or making mistakes that I want to catch as soon as possible). Theoretically, that doesn't rule out it's being scalable, but in practical terms, it does (especially since for administrators, scalable=teach more students at a significantly lower cost per student).

      There's a tendency to confuse the various kinds of online teaching, and especially, I think, to confuse online delivery of content with more interactive learning activities: discussions, group work, etc., especially when those activities are actively monitored by a teacher (even though (s)he may not always immediately intervene when (s)he sees a problem, hoping the students will work it out themselves -- in that way, it's very much like f2f teaching, with all the same tricky judgment calls, and, as a result, all the same benefits that come with an experienced instructor who is able to tweak the class on a semesterly basis). One interesting point that came up at a "Dark Side of the Digital Humanities" panel at the MLA this January is that faculty may have fostered this confusion (and a related confusion between Digital Humanities and online pedagogy), precisely because anything with "digital" or "online" in the title *is* "crack for administrators" -- i.e., they'll pay for it, often without asking too many questions.

  3. Doesn't anyone remember programmed instruction, from the late '60s? No, I suppose not: it was infamous for that. It was cheaper than online instruction, since it didn't require everyone to have a computer.

    Doesn't anyone remember teaching machines? No, I suppose not: they were infamous for that too. They had more of the disadvantages of online instruction, including the high costs of the machines themselves.

    1. I wonder if anybody has tested books? Much as I love the things, I fear they wouldn't score all that high, either. Learning is hard work (and traditional-age college students are awash in hormones and subject to other distractions, as are less-traditional-age students), and one needs a really good incentive to master a body of knowledge/skills, and ongoing use/practice, to really stay current with a subject. Otherwise, the value of a course may well not be in subject mastery achieved, but in the experience of having thought carefully about it, and learned (or at least learned about) new ways of thinking about the world in the process. Except in the case of a few rare students, that probably requires some interaction with, and guidance from, other students and a professor.

  4. @Aware, I don't think you would question the benefits of classroom give-and-take in real time, when it happens. You're right, in a typical class most students are completely disengaged. But there are always a few who aren't, and just from observing them, gradually others are moved to participate. I wonder if there is an online equivalent of this process.

    For self-motivated students and instruction that amounts to learning technique (like, say, photography), or with the main purpose of certification in a technical area, I suppose online learning works fine, and improves accessibility (I learned how to repack bearings on a boat trailer from watching YouTube videos). There's another kind of learning that requires full-time immersion into a specialized culture (be it mathematics, anthropology or woodworking), including socialization into it. For that kind, online doesn't cut it, and I doubt it ever will. I realize it's an elitist concern, but so be it.

    @CC, I interpret your comment as saying that, for the kind of "non-synchronous interaction" required, online teaching is far less efficient than in-person. I imagine in the same amount of time your students would get a lot more out of their interaction with you and with their peers (with less effort needed on your part) if instead of being online you were able to physically meet with them in a classroom, on a Saturday afternoon. (It just may not be physically possible).

    1. Actually, Peter, I would question the benefit of classroom give-and-take in real time, even when it happens.

      Look, I earned my BA conventionally back when computers were those things you had to BE at a college to use and they were reserved for those engaged in actual research, not simply preparing a term paper. (The fortunate of us had electric typewriters!) So, I do remember some of the mythical exchanges that you hold in such high regard.

      But I remember them because I was "average to above average students with reasonable self-regulation and time management skills."

      I enjoyed college life, but I -- like many of us here -- kept my focus on learning and exploring seeking out opportunities, engaging the faculty, beginning some of those gives-and-takes.

      At the same time, I saw back then (and still see today) students who seemed enthusiastic and engaged because they were talking about anything BUT the topic of the class. Many took pride in getting the instructor off topic so that a tangential current event because the spirited discussion instead of the actual subject of the day.

      I yearn for those authentic exchanges where interested people energetically engage each other in a mutual exploration. I was just saying to my SO how I got an end-of-quarter EMail from one of my stellar online students and I so wish the student and I could be meeting at the coffee shop or the quad and continue the conversation in real time.

      But, those opportunities -- and events -- are rare.

      That is why I chafe at the automatic presumption that all on campus classes are filled with enthusiastic engagement. Remember, you can nap in the back of a classroom and still be marked present. In an online class, your "discussion" is likely a written submission and those cannot be completed while unconscious.

      (Yeah, of course, the quality can be an issue, but that is a separate conversation.)

  5. @Peter: I think that's true, at least for engaged students (given the same student population, well-done online learning is generally somewhat less efficient than face to face learning). However, I really wouldn't underestimate the extent of a phenomenon to which A&S alludes: the ability of students registered for traditional classes to mistake attendance (even occasional attendance) at class meetings for actually taking the class. There's an analogue online (students who I can see checking into the LMS fairly regularly, but never seem to post anything), but, for the most part, the difference between being registered for the class and actually participating in the class is a bit more clear: either you're completing the posts (most assignments in my class are discussion board posts of one kind or another, some aimed at discussion, many aimed at completing stages of group and individual projects, and/or critiquing others' work on those stages -- which is something I do not as a substitute for my own critique/feedback, which comes later, but to give students more practice with the concepts involved).

    I also wouldn't underestimate the value of having online classes available to the population A&S describes: "average to above average students with reasonable self-regulation and time management skills," especially those with complicated and/or variable schedules, due to work and/or family life. It's also worth noting that, in my area, commuting time (i.e. time spent sitting in traffic) is a non-trivial consideration at most times of day.

    Overall, I'd guess that my institution may already be offering a few too many online sections of the required course I teach (a writing-in-the-disciplines class required of pretty much all juniors, including both transfers and those who started with us as freshmen -- so, a very big required course, taught in many 20-25 student sections). It's not a good way for a traditional student to try to cram one more annoying required course into hir already-overfilled schedule (and/or in between working too many hours to pay for college -- and that, incidentally, includes the athletes. Oy, the athletes! Or, I should say, some of the athletes. Others are doing just fine, and some I hadn't even known were athletes until the person in charge of doing such things sent me one of those email check-up messages.) It can be a very good way for a well-organized returning student to fit in the class around other commitments, or for a well-organized traditional-age student to fit it into a semester shaped mostly around an internship or field experience or similar. And, although to my knowledge I haven't had many students who fit this description, it could work well for deployed military personnel (at least those who have internet access and somewhat predictable schedules) or physically disabled students for whom transportation is a major barrier to access.

    1. (continued; I managed to hit blogger's limit again). And I suspect it can work in many fields, including those where socialization/enculturation is part of the process (after all, we've built something of a culture here at CM, and, not so long ago, scholars built community almost entirely through letters and published journal articles). I think we're in a shakedown phase, and will need both to encourage fellow faculty members who insist that nothing worthwhile in our fields can be done online to be a bit more open-minded, and to discourage administrators who think everything can be accomplished online (and cheaply, to boot). At the moment, I think the latter is the greater concern, because it's the direction in which those with power and money (but, often, little if any actual teaching experience) are pressing. But if we're going to have a say in what combination of face to face, online, or hybrid approaches, in particular classes or in a program as a whole, will work best with our fields, I think we need to be experimenting, sharing the results of those experiments, and insisting that our voices carry weight in the conversations about the direction of higher ed. It's easy to dismiss someone who says "no part of my field can ever be taught effectively online" as a luddite; it's not so easy to dismiss someone who says "this works well online, but this works much better face to face, and here are several ways to structure a program in a way that builds on that knowledge." Or at least I hope so.

      And, by the way, I'm really, really looking forward to spring break, which comes in a few weeks, and when I will actually be able to afford to log off for a couple of days.

  6. My department chair wants to talk about offering our courses in the teaching-masters program online. Currently they meet in the evening, once a week (it's a medium-size town, no commuting problem.) I wonder what the motivation is, and I also wonder if the (non-traditional, mostly teachers) students would miss the opportunity to socialize with their peers.

    And I'm not a complete luddite (hey, I own a prepaid cell-phone, I just can never find it.) I do math software demos in my lower-div classes, and I tell my students about relevant math links all the time. Once I tried to start a discussion board for a course on Blackboard, but there were no takers. I'm all for the kind of technology use that increases interest in the course; but students worry that this might mean more work for them, so I make it optional.

    1. Well, one obvious first step in that situation would be to survey the current students (and, if you can manage it, any identifiable potential ones) to see if they'd be interested in an online version of the course (or perhaps a hybrid one -- every other week face to face, or maybe one Saturday a month, with online activities in between).

      The other thing you might consider, if you're not already doing it, is to put some sort of assessment instrument that helps you measure how well you're meeting your goals for the course in place now, so that you have a baseline against which to measure any online or hybrid approaches.

      It would also be worthwhile to sound out your chair to see if (s)he has visions of attracting a large number of new students in this way. As far as I can tell, pretty much every administrator has those visions these days, which suggests that they're all hoping to attract new students from a pool that is considerably smaller than they think (and is even smaller if you're talking about students who will persist past the first semester/course, especially if you attract them via advertising that overemphasizes ease and convenience).

      Of course, there is a possibility that you're losing (or about to lose) actual or potential students to someone else's online effort. That might be an argument for offering some sort of "best of both worlds" approach -- which brings me back to the hybrid possibility, and to surveying present/potential students to see what they think. That's a much better strategy, I think, than laying plans to attract some mostly-mythical set of well-prepared, self-disciplined students who will pay full tuition but demand very little in the way of services/support (the average administrator's ideal online student).

    2. Oh CC! If only they (administrators) would just listen to your reasoned arguments.

      Instead, all they see is the potential of $$$ and not of losing students to a "better brand" who offers the same education... maybe even at a much higher cost, but with a better brand name for their diploma.

    3. Thanks, Dr. D. I suspect that if I were an administrator faced with dwindling state support and rising costs (and legislators clamoring for me to deliver more education more cheaply, or they'll cut support even more), I'd probably be clutching at straws, too. It's always easier to kibbitz from the margins/monkey gallery.

      The "better brand" question is a fascinating one. It seems to me that schools with a nationally-known/respected name (at least in a particular field), and the same need as many other schools for new revenue streams, are going to have the trickiest decisions to make, as they weigh the possibilities for a much broader student pool against the dangers of cheapening their brand names. It's probably not a coincidence that a crisis fueled partly by disagreement over how and how aggressively to move into the online market erupted at UVA (a school which pretty closely fits that description). My very vague sense is that "executive MBAs" may be serving as a testing-ground for these questions (perhaps in part because B-schools tend to be more entrepreneurial than the rest of a school, and also in part because they can always be dismissed as somewhat separate from the university's traditional/core mission if things go pear-shaped). My even vaguer sense is that most of those executive MBA programs include, or at least offer, at least some face to face time (on campus or elsewhere), so that students can claim to have been in the same room as the star faculty who give the programs cachet. Frankly, I've been surprised by how many pretty prestigious schools are willing to lend/risk their names to/on such programs. Maybe they know something that I don't know, or maybe they're just unrealistically optimistic about this brave new world (see: nature of B-schools, above).

      I'd guess that the *really* big names with equally big endownments (Harvard, MIT, Stanford) are going to stick to creating free offerings for the greater good of the global public (or perhaps marketing something MOOC-ish to be used as the equivalent of a multimedia textbook). They don't have the same desperate need for revenue from any source possible, and do have something very valuable (though admittedly probably not easily seriously harmed) in their brand names.