I’m delighted to tell you that our featured guest for this Blursday Social is Rahim Rajan, Deputy Director of Post-Secondary Success at the Bill & Melinda Gates Foundation (BMGF). Rahim has been at the foundation for 10 years, so he quite the experience base from that perch. He’s also a fun and thoughtful guy to talk to.
Rahim Rajan, Deputy Director of Post-Secondary Success at the Bill & Melinda Gates Foundation
I’m interested in asking Rahim his perspective on how higher education has changed over the past decade, how the foundation’s approach to it has evolved, what he sees at this current moment of inflection that we’re in, and also just generally how BMGF goes about its process of sense-making and developing a theory of change in this wonderfully messy and complicated sector.
But of course, as usual, we’ll talk about whatever y’all want to talk about. It’s a party, not a webinar.
Please do join us this Blursday, 3/18, from 4 PM to 5:30 PM ET.
My latest Chronicle column is up. It analyzes the results of the SRI Education study of the Gates Foundation adaptive learning grantees, some of which we’ve covered in our e-Literate TV case studies. If you’re looking for evidence that adaptive learning is going to deliver on the promise of a robot tutor in the sky, you won’t find it there. But it’s easy to flatten that result into “adaptive learning doesn’t work.” I don’t believe that the SRI study shows any such thing.
First of all, what is our standard of proof? A good half of my column is devoted to the methodological challenges of doing big meta-studies like this one. It’s really hard to (ethically) control the variables across multiple classrooms well enough to get a clean result. SRI had to throw out most of the data they had for some measures.
But equally importantly, there’s just a lot that meta-studies can’t get at precisely because the goals of each implementation are different. For example, one of the goals for implementing OLI at UC Davis was getting students more prepared to engage in higher-level critical thinking in class discussion. Here are some Davis faculty talking about their course design goals:
I’m not sure how one would empirically measure such a result; nor could I see how to incorporate it into a meta-study that also includes, for example, Essex County College’s developmental math course.
While the Gates Foundation should get credit for bringing in a credible third-party evaluator to review the results of the grants, ((Full disclosure: Our company received grant money from the Gates Foundation to cover personalized learning)) the design parameters for this particular study do not appear to be as useful as they might have been. That said, the larger point is that it’s really hard to do educational research well. Rather than using these studies as Rorchach tests, we should be taking the time to improve our educational research literacy and better understand what each study can and cannot prove.
This is almost old news now, but we just haven’t been able to dig into it yet. As part of its Adaptive Learning Market Acceleration Program (ALMAP) program, the Gates Foundation funded SRI to do a study of the results of the grants after two years. I hope to finally clear some time to parse through it this week, but two high-level points jump out at me at first glance (neither of which is a big surprise):
There are no conclusive wins here. This is not a robot-tutor-in-the-sky moment. A few programs did well here and there. A handful produced promising incremental gains. But this is not a report that screams, “Wow, adaptive courseware works!” The most you can say is that adaptive learning looks like it could be another arrow in the quiver that helps out in some situations—like developmental math courses at two-year colleges, for example.
Large-scale educational research is incredibly hard and may actually be impossible to do rigorously for certain kinds of questions. I’ll probably get into this more when I’m ready to really write about the report, but one reason the conclusions are murky is because there so many variables in each class—not just each course subject, not just each course at one university, but even with each section of each class taught by one teacher—that really matter, some of which are impossible to control and others of which are unethical to control. It would be a mistake to overinterpret the study as showing that adaptive learning doesn’t help much. I’ve seen studies with narrower focus that have gotten clearer gains. This brings us back to the arrow-in-the-quiver theory. The wider the scope of the research focus, the more that pockets of real benefit will be obscured by noise and uncontrolled variables.
Today we are thrilled to release the fifth and final case study in our new e-Literate TV series on “personalized learning”. In this series, we examine how that term, which is heavily marketed but poorly defined, is implemented on the ground at a variety of colleges and universities. We plan to cap off this series with two analysis episodes looking at themes across the case studies.
We are adding three episodes from the University of California at Davis (UC Davis), a large research university that has a strong emphasis in science, technology, engineering, and math or STEM fields. The school has determined that the biggest opportunity to improve STEM education is to improve the success rates in introductory sciences classes – the ones typically taught in large lecture format at universities of their size. Can you personalize this most impersonal of academic experiences? What opportunities and barriers do institutions face when they try to extend personalized learning approaches?
You can see all the case studies (either 2 or 3 per case study) at the series link, and you can access individual episodes below. (more…)
In thesetwo episodes of e-Literate TV, we shared how Arizona State University (ASU) started using Khan Academy as the software platform for a redesigned developmental math course ((The terms remedial math and developmental math are interchangeable in this context.)) (MAT 110). The program was designed in Summer 2014 and ran through Fall 2014 and Spring 2015 terms. Recognizing the public information shared through e-Literate TV, ASU officials recently informed us that they had made a programmatic change and will replace their use of Khan Academy software with McGraw-Hill’s LearnSmart software that is used in other sections of developmental math.
To put this news in context, here is the first episode’s mention of Khan Academy usage. (more…)
Today we are thrilled to release the fourth case study in our new e-Literate TV series on “personalized learning”. In this series, we examine how that term, which is heavily marketed but poorly defined, is implemented on the ground at a variety of colleges and universities.
We are adding two episodes from Empire State College (ESC), a school that was founded in 1971 as part of the State University of New York. Through a lot of one-on-one, student-faculty interactions, the school was designed to serve the needs of students who don’t do well at traditional colleges. What problems are they trying to solve? How do students view some of the changes? What role does the practice of granting prior-learning assessments (PLA) play in non-traditional students’ education?
You can see all the case studies (either 2 or 3 per case study) at the series link, and you can access individual episodes below. (more…)
Phil, I’m interested to know if you found anything out about the pay rates for coaches v TAs. I’m also interested in what coaches were actually paid to do — how the parameters of their employable hours fit what they ended up doing. Academics are rarely encouraged to think of their work in terms of billable increments, because this would sink the ship. But still I’m curious. Did ASU really just hike up their staffing costs in moving to personalised learning, or was there some other cost efficiency here? If the overall increase in students paid off, how did this happen? I’m grappling with how this worked for ASU in budgetary terms, as the pedagogical gain is so clear.
This comment happened to coincide with my participation in WCET’s Leadership Summit on Adaptive Learning, where similar subjects were being discussed. For the purposes of this blog post, we’ll use the “personalized learning” language, which includes use of adaptive software as a subset. Let’s first address the ASU-specific questions. (more…)
