Update 10/25: Bumped comment from Darryl Yong, a member of the research team, into its own post here.
Update 10/26: We now have Rachel Levy and Nancy Lape (who was the researcher interviewed by USA Today) both agreeing with Darryl’s comments. That’s three of the four members of the research team. While I do not claim to understand how the reporter developed her story line (I have asked for comment), it is quite clear that the article does not represent the work of the Harvey Mudd research team and has a misleading headline and lede.
USA Today published an article today titled “‘Flipped classrooms’ may not have any impact on learning“, based on research from four Harvey Mudd professors. This research is backed by a “$199,544 grant from the National Science Foundation to study the effects of the flipped classroom on students’ learning”. This is newsworthy, right? A real research report with NSF funding finding no statistical difference in learning outcomes from flipped classroom seems to contradict much of the recent promise of ed tech.
Upon closer reading, however, there are some major problems with the story. Exaggerated claims by ed tech enthusiasts are not helpful, but neither are exaggerated claims by ed tech skeptics. We at e-Literate have been critical of both flavors (witness our analysis of San Jose State claims, Desire2Learn claims, and edX claims for examples of the former).
Let’s review today’s story as an example of the latter.
In a flipped classroom, students watch their professors’ lectures online before class, while spending class time working on hands-on, “real world” problems.
The potential of the model has many educators thrilled — it could be the end of vast lecture halls, students falling asleep and boring, monotone professors.
This is a decent summary, although I would argue there should not be a one-size-fits-all mentality. Flipped classrooms, even where successful, should not replace all lecture-based classes. But for national media, this is not a bad description.
But four professors at Harvey Mudd College in Claremont, Calif. who are studying the effectiveness of a flipped classroom have bad news for advocates of the trend: it might not make any difference.
This is the lede, and the claim that we should examine. What is the basis of their study?
Though their official research is just beginning, the professors flipped their STEM classrooms as a pilot during the 2012-2013 academic year and gathered some first impressions on the matter.
While [Professor Nancy] Lape stresses that their preliminary research is just that — preliminary — she says the benefits of flipping a classroom are dubious.
During this pilot, each professor taught two sections of the same course — one “flipped” and one traditional, using the same material as much as possible.
So there is no report yet, and the study is not complete. But they are ready to make conclusions? I’d be willing to bet that the vast majority of readers will remember the headline and lede and not the qualifier of “preliminary research”. There is no ability for people to study the team’s results given the nature of this article, so we are supposed to just trust the research team and the reporter.
The potential of flipped classrooms, or any redesign, is not based on just changing delivery. It often comes from changing or enhancing the learning content to fit the course redesign. This study seems to force-fit traditional material into a flipped classroom. While that might be appropriate for their classes, it is not the only option.
And making conclusions on eight classes – four traditional, four flipped – taught by the the same four professors? While that is great for self-reflection and experimentation, I’m glad they aren’t extrapolating their preliminary findings.
“I would say that the fact is that there is no statistical difference,” Lape says. “People are really gung ho about the (flipped) classroom, but there’s no real results.”
Wow. They have enough data to claim the fact of no statistical difference and no results on flipped classrooms.
Does the research team know about the work of the National Center of Academic Transformation or the Open Learning Initiative and their documented results on course redesign? There are other studies showing promising results from the flipped classroom concept, including the recent redesign at San Jose State University. None are fully conclusive for all cases, but to claim that there are “no real results” seems disingenuous.
Did I mention (thanks to reminder from @GlobalHigherEd) that Harvey Mudd has a 9:1 student to teacher ratio? That’s not exactly a good basis for extrapolating results to the broad usage of flipped classrooms. If all colleges had 9:1 ratios, it would be difficult to find any course redesign with improved results. Not impossible, but the bar would be set much higher.
Maybe the research team will be cautious about advising other faculty what to do, given their very unique experience at Harvey Mudd.
Professors, too, had to spend considerably more time making and editing the videos and crafting engaging, hands-on sessions for their classes, she says.
Given these drawbacks, the fact that the actual learning outcomes seemed unaffected by the switch suggested that it might not be worth the hassle, Lape says.
“(The professors’) lives might be easier and their students might be happier if they just do a traditional class,” she says.
Yikes. Those are some pretty audacious claims in national media for preliminary results from eight classes taught by the research team themselves at a school with 9:1 student to faculty ration. At the end the reporter adds some needed context.
Andrew Miller, an education consultant who teaches online classes for a variety of universities, agrees that benefits such as students’ ability to review material are promising, but says nothing will change if professors don’t handle the “flip” correctly.
For example, the newly freed-up class time can be daunting for professors, especially those who are particularly gifted at lecturing, he says.
Sometimes these professors aren’t able to come up with good hands-on activities and resort to filling the time with even more lecturing.
“If you’re not a good instructor, flipping the classroom won’t really ensure better learning,” he says. “If you aren’t doing something to fill that space, it won’t do you any good.”
And to be fair, the quoted professor at the end softens her tone.
Lape says she hopes those within academia take a more critical look at flipped classrooms.
“It’s a hot topic, and there are reasons why I think people believe it will be a good method,” Lape says. “But I would really put the call out to more people to really look at this.” [emphasis added]
That is a call that I support.
There could be an argument that this article is a case of a reporter trying to find a sensational topic from a nuanced report. But the real problems in this article seem to be direct quotes from one of the research professors, despite the qualifier of “preliminary”.
We should expect better from both ed tech enthusiasts and from ed tech skeptics, especially when lending the credibility of official research. I look forward to the full report from the Harvey Mudd researchers.
my own nsf proposal incorprating moocs was rated excellent by all the faculty reviewers, but denied by the program officer, a small liberal arts college rotator, who was disdained the mooc concept and thought it unproven. easy for him to say in his ivy covered northeastern remote campuswith 10:1 student:teacher ratio. would love for him to live in my public community college world
“Flipped” has become a shorthand very like MOOC itself, that gets tossed about like a beachball in a cricket crowd (bear with me, it may only be Australians that do this). It doesn’t inspire careful handling, so much as reflex reactions that keep it moving. So I’m currently seeing “flipped” all over planning documents and in KPIs as if everyone knows what it is and everyone means the same thing, and whatever it is we want to do it.
I think you’re right to focus on the use of technology in good design conceived much more broadly than this. “Flipped” is really a limited and backwards-looking design because it’s forever fused to the model it’s supposedly inverting, which means that when we talk about flipped we go on about large lectures as the dominant educational standard, in all places and in all disciplines. This is so behind the times, and not what we need.
Kate is on target. When we become adults we say to ourselves that we will never do what our parents did (usually about raising children). But in a tight situation we react with the only experience that we have. The same with education. MOOC’s, flipped and similar models are variances on the current classes whether brick space or bricks to clicks. These have appeared over many years well before click space. The real and, perhaps, disquieting change in the education system (but not in the world outside) is recognition or “credit” based on competencies. Can one take theory, from philosophy to engineering and medicine into practice.
Do we need knowledge deconstructed into bite-sized units called classes and lectures delivered in variances in institutions around the world by “n” different scholars and have students march, lock-step,often in age-defined cohorts? As we see, today, the post-secondary student body is a changing/changed population, globally.
Yet the faculty, with the above variances such as flip and MOOC, like adding tail fins to cars, persists in trying to maintain their hegemony, while faced with a far more radical competitor in competency based alternatives. Similarly, the “Institution” faced with the equivalent of a “stripped-down”, no frills, low cost competency-based alternative are finding themselves competing like Detroit with their over accessorized product line.
Thanks for your excellent reply to the USA Today article. I would add that the article author ignored the 30+ years of science education research on how people learn in STEM classes, including research PUBLISHED research on flipped classrooms (including side-by-side comparisons) showing improved learning of basic concepts. Of course, that research focused on the use of research-validated pedagogies during the “in-class” phase of the flipped classroom. I am amazed how journalists can treat such as story as if the Harvey Mudd profs are the first people to ever study this phenomenon, and their highly preliminary results are treated as if they settle the issue. TERRIBLE journalism, but I blame the professors as well for “publishing” in the USA Today like this.
I learned from the originators of the flipped classroom concept (Jon Bergmann and Aaron Sams) back in the early days and have had my classroom flipped for years. What I have seen is what Sams and Bergmann realized early on–the real key to success in flipping has little to do with the videos. It has everything to do with what you replace the lecture with. After spending thousands of hours making high-quality videos, they have become nothing more than a supplement for my own students, because I’m finding that if I do the classroom exploration right, very little direct instruction is needed.
I don’t see anything wrong with the USA Today article. The null hypothesis in any experiment is that there is no effect. That is what the researchers found.
You are writing as if there was a mountain of evidence showing the effectiveness of flipped classrooms. There is not. Perhaps there will be one day. But again, to show that a hypothesis is true, the burden of proof is on the one showing it is true, not the other way around.
Matt, I wrote some very specific critiques of the article, and it is difficult to respond to a comment with no specifics. I would also point out that the actual research team (3 of 4 members) have argued that “the USA Today article paints an inaccurate picture of our work”. You don’t have to agree, but it would help to be more specific in comments.
It seems you are trying to position my post as arguing that flipped classrooms have statistical proof of their effectiveness and that I have a hypothesis to this effect. That is not my point at all – read the post I actually wrote.