Update 10/26: We now have Rachel Levy and Nancy Lape (who was the researcher interviewed by USA Today) both agreeing with Darryl’s comments. That’s three of the four members of the research team. While I do not claim to understand how the reporter developed her story line (I have asked for comment), it is quite clear that the article does not represent the work of the Harvey Mudd research team and has a misleading headline and lede.
As a follow up to my response to the USA Today article on flipped classroom research, there was a very informative comment one Google+ from Darryl Yong (one of the members of the Harvey Mudd research team). I thought this comment deserved to be seen by a wider audience, so I’m reproducing in full.
Thanks, Phil, for this post. The USA Today article paints an inaccurate picture of our work and I wanted to try to clarify some things and continue this conversation with you and your readers. (I am only writing on my own behalf and not for my collaborators on this study.)
My biggest regret is that the article greatly oversimplifies things by portraying our study as an attempt to answer whether flipped classrooms work or not. That kind of research question is too blunt to be useful. Our goal is to better understand the conditions under which flipped classrooms lead to better student outcomes. As Phil and others here point out, there are many different manifestations of what we mean by the term “flipped classroom” and that we should be wary about talking about it as if there is one canonical implementation. How are the benefits of flipped classrooms affected by school and student contexts, and are there other benefits that we haven’t yet characterized? We don’t have any preconceived answers to these research questions. While folks might disagree with us whether these research questions are interesting or not, they should at least know that we’re aware of the good work that others have already done and we’re trying to build on it rather than debunk it.
Phil rightly points out that Harvey Mudd College’s learning environment differs from other institutions, and that we should be cautious about extrapolating our experience here to other contexts. There are three important things folks should know about Mudd with regards to this study. First, there is a healthy culture of cooperation at Mudd, which is mostly residential and has about 800 undergraduates, and students already spend a lot of time working together in groups in and out of class. Second, our students generally have high self-efficacy and positive attitudes about learning science and mathematics. Third, a lot of us faculty at Mudd currently use formative assessment, active learning and group work in our classes (minute papers, iClickers, think-pair-share, etc.). Some of us participating on this study have been flipping our classrooms for a while and the four of us got together because we are interested in better understanding the conditions under which classroom inversion could lead to improved student outcomes at Mudd.
There are lots of studies that show that active learning, formative assessment, group work, project-based learning, just-in-time learning are all measurably good things for students. If someone, through classroom inversion, adds these demonstrably good things to a learning environment where those things are not the norm, then we would expect positive changes in student attitudes and learning. This leads us to wonder: are flipped classrooms good for students because they create more time for these demonstrably good things, or are there other benefits that we haven’t yet characterized? If it is the case that flipped classrooms have positive effects for students mainly because they free up more time for active learning and those other good things, what happens when you already do those things in your “lecture” class? Is there some sort of saturation effect? And if it turns out that there are positive aspects about the delivery of instruction via videos, we should try to understand them too.
The idea of our study is to try to control for as many variables as we reasonably can so as to understand the effect of classroom inversion on real students’ performance and attitudes. The instruction, whether via video or in person, is given using the same set of instructional materials (handouts, slides, etc.); all students complete exactly the same tasks, assignments, quizzes, exams. We do our best to randomly assign students to treatment or control groups. We are trying to measure both student learning gains (using vetted instruments when possible) and attitudes in three courses from three different disciplines and to track students’ performance in “downstream” courses.
Many studies of flipped classrooms involve comparing students taking different versions of a course (often taught in different semesters/quarters). Students in the flipped version of the course get to practice more and/or learn more material—if students practice more or get exposed to more material, you would hope for improved student outcomes. So if you’re still reading this far in, you might be asking why one would expect improved student outcomes from classroom inversion if students aren’t asked to practice more or learn more material in the flipped classroom. Because we’re trying to avoid giving students in our flipped classrooms tasks different tasks, we are mainly using class time in our flipped sections for students to work together on homework problems. What if those problems are designed so as to elicit potential misconceptions? Could another potential benefit of flipping be to allow instructors to more quickly identify and resolve students’ misconceptions, and might that lead to increased student learning? That’s one potential idea, and we’d love to hear your thoughts about what other things one might be able to do as a result of having more time for instructor-mediated learning.
I was not present when Nancy was being interviewed by the USA Today reporter, but I strongly believe the reporter took Nancy’s words out of context. Nancy was not saying that flipped classrooms are dubious in general. She was saying that given our study design and Mudd context, we have not yet seen any difference in student outcomes. Of course, this was only the first year of the study and we are admittedly working out all of the kinks in our flipped classes.
As Chuck Severance points out, the danger is that flipped classrooms become the next flavor-of-the-month and that people adopt it without really understanding why it might lead to better student learning. This is our small way of contributing to what is known about flipped classrooms, and we’d appreciate your ideas and constructive feedback.
Per Twitter, Rachel Levy stated that Darryl’s comment also reflect her views (she is another member of research team). I appreciate both of their replies and this important clarification to the article.
Phil, thank you for this follow-up. The report as reported was a topic of conversation for faculty, especially those on the fence about flipped classrooms. There’s lot’s of good takeaways from the Darryl Young’s response, but this one stood out:
“And if it turns out that there are positive aspects about the delivery of instruction via videos, we should try to understand them too.”
It stood out because critics of flipped classrooms like to point out that video-based or video-enhanced instruction failed once (a la waned DVD courses) and will fail this time. However, the first wave occurred before the ubiquitous influence of YouTube, video casts and recorded webinars. Taken together they present not only technological and pedagogical shifts, but may evolve psychological and physiological changes that lead to video-viewing becoming a best-practice or the best practice for certain situations or people.
As with a lot of the discourse in higher education, the conversation around flipped classrooms is decidedly detached from and construed outside a discussion about the real life phenomena and influence of “youtubing,” which is huge among “digital natives.” This is the case because a large part of the conversation is lead by non-natives and academics who see video-led or enhanced instruction as a couch potato past-time and not worthy of serious consideration.
Corey, good points about video and “youtubing”. I think another aspect that is overlooked is the simple ability to play, replay as much as needed, all with no embarrassment and at the student’s control.
One needs to understand that the “flip” is still within the context of a time-based, space-based educational experience. If, on the other hand, the students were in a competency-based program then the idea and even form of the “flip” might be significantly different.
It is also noted, above, that the pgm at Mudd involves a number of different practices. Such practices cannot be ignored in this study without committing the same type of flaw that economists create when they lump intractables into externalities and reduce them to a constant in order to have a soluble model.
Similarly, there are many other emergent practices from variances of MOOC’s to competencies which directly or indirectly impact on the practice of “flipping”. Again, as the discussion above notes, to attempt to ignore these ideas in the analysis is to commit to the reductionist models of the natural sciences transferred to the social arenas.
Practitioners and those or track these practices need to take a more systemic approach to efforts in this arena. And, as the comments on the USA Today article shows, one needs to carefully weigh from where such information originates.
Tom, for the reasons you mention, I think the research team (and NSF) have quite a challenge to produce meaningful results. Can they really isolate variables in the reductionist model and present quantitative, or “statistical”, outcomes (I add the scare quotes due to the treatment in USA Today article)? I suspect that the better path is to discover more qualitative outcomes that have fuller descriptions embedded.
At the least, I’m much more confident in the ability of the HM team to produce meaningful results given Darryl’s extended comment. We don’t know, however, whether the full research team is on the same page (as Darryl and Rachel point out, they are offering individual comments and not speaking for the team officially).
Phil, thanks for posting Darryl’s response. I wanted to let you know that I also whole-heartedly agree with Darryl’s comments.
Thanks Nancy
good day Phil,
There is a recent Oxford University Press book, Aid on the Edge of Chaos. While the focus is on international aid, the author and many others who are familiar with complexity science have repeated, many times, in the open literature, particularly with work in the social research arena, that reductionist models are extremely dangerous to use in making assumptions, drawing conclusions, or making decisions/recommendations. Economics is a particularly troublesome arena. But the type of research which is being done on “flipped” and MOOC’s and options in educational change maybe even more troublesome, particularly when they are essentially “works in progress”.
To “legitimate” by producing a mathematical model or carrying out “statistical” analysis, again, as economists and others will tell you, does not necessarily hold, even, at times, for the natural sciences. It is an area that needs addressment beyond the exchange on this list.
But it should raise serious concerns when reviewing and reporting these changes in academia and citing the “data” regardless of which side of the issue one sits. This makes it very difficult for a lay audience and generalists seeking “answers”; and, it is just, if not more, problematic for those who need to make decisions in this arena.
I haven’t been following this story, but am saddened – though hardly surprised – about a nuanced study being decontextualized in its coverage by a news organization.
I’m reminded – as I so often am – of PhdComics satirical portrayal of The Science News Cycle.
While I don’t want to defend the article, I believe USATodays mission of fostering “better understanding and unity” tends to promote simplification – and oversimplification – as a byproduct.
Thanks for your perseverance in recontextualizing the study, and providing a forum in which the voices of the researchers who conducted the study can be more broadly heard.
Joe – love that comic!