I’ve been thinking quite a bit recently about efficacy in education in light of the Pearson realignment. Is efficacy, a medical concept, a good lens through which to view our current educational technology challenges? Michael Feldstein laid out some reasons it might be in his treatment of the shift, but I’d like to expand on one of them — the “last mile”.
One of the crucial elements of efficacy is that it is *not* solely a lab concept. Certainly you can talk about a drug’s efficacy in research trials, but in general what medical researchers are interested in is how the drug performs in real-world scenarios. And in real-world scenarios, regimen compliance (more recently referred to as “adherence”) is a huge issue. Here’s a good explanation of the problem:
“We’ve got so many great medications out there, but how do you get the patient to take the drops correctly?” said Dr. Tsai. To answer [this] question, he turned to diabetes researchers, who have been at the forefront of the field. “The researchers don’t see the patients as being bad,” Dr. Tsai explained. “They just see barriers to adherence.” On their advice he created a taxonomy, or systematic classification, of all the barriers to adherence.
Through a series of open-ended and specific questions, Dr. Tsai and colleagues described 71 unique reasons for nonadherence, culled from interviews with 48 patients with glaucoma. The number of barriers was higher than expected.
Patients reported the typical reasons, including problems with drop administration, following directions, bottle design, cost and forgetfulness. But nearly 50 percent of the barriers were “situational/environmental,” instances in which some other factors come into play. For example, the patient travels to Europe, forgets his drops and has no way to refill them. Or a loved one is in the hospital and the glaucoma patient feels guilty focusing on her own health.
“Our major finding,” said Dr. Tsai, “was that solutions are going to have to become more innovative and individualized. It’s likely to be different for each patient.”
Medical regimens are simple compared to educational ones. And yet, as the above shows, they are still profoundly complex. The key, however, is in that first paragraph:
“The researchers don’t see the patients as being bad,” Dr. Tsai explained. “They just see barriers to adherence.”
If there is a good lesson to take away from efficacy research, it is this one — no one cares how your drug performs in the lab. While educational technology has run towards the air-drop model of intervention, modern medicine has actually been running away from that model. The “last mile“, the name some epidemiologists give the gap between having effective treatments and staging effective interventions, turns out to be the majority of the journey; and if you are focussed on health instead of pills what you need to look at is the whole ecosystem. You need to see the barriers to use of your medication as being as much a part of your medication as its chemical structure. You need to do crazy things, like actually talk to people on the ground about what their issues are, observe product use, provide support, and customize implementation to address local realities.
As I mentioned above, educational technology implementation is far more complex than taking glaucoma drops; yet, amazingly we treat it with less nuance than the medical community treats its own products. We engage in technology air-drops, listen approvingly to Sugara Mitra’s hole-in-the-wall pitches. We release breathless press releases about projects like ConnectED, the multi-billion dollar project to wire up America’s schools which, as far as I can tell, has not a lick of professional development funds tied to it to help educators figure out how to use that connectivity. We made the pills, problem solved, right?
I remain somewhat ambivalent about Pearson’s corporate re-engineering. And there are many issues with applying the concept of efficacy to education. But I’m willing to deal with them all if we can finally recognize that the last mile is the longest and most important, and ultimately has to be a shared responsibility.
Very interesting post. We (the EDUCAUSE Learning Initiative) have been struck also by what we have called the last mile problem in our evidence of impact initiative. The problem is that you can do even rigorous evaluation of the impact of a curricular innovation and those results will still not change practice.
We became aware of this problem due to a remark made by Peter Ewell in his chapter in the book “Building a Scholarship of Assessment” (2002). He wrote “Most campus assessment activities… continue to be implemented as additions to the curriculum… rather than being integral to teaching and learning. [It] centers on “doing assessment” rather than on improving practice… Although firmly established in the mainstream by the year 2000, assessment as a movement is still striving for the cultural shift its original proponents had hoped for.”
This seems to me to be exactly what Mike is addressing in this post. The lesson we draw from this is that the assessment and research is a necessary but not sufficient condition for changing practice.
Malcolm, thanks — I’ll take a look at that Peter Ewell chapter. I’ll also dig up some articles I’ve been reading on the challenges of doing medical interventions in the third world. I know that seems like a horribly dismal comparison, but I’ve found that a lot of their issues are our issues and that their insights are quite good.
The other area of literature which I find really pertinent is the research on emergency preparedness. Local agencies had to make changes to how they operated after 9/11, but in many cases those changes were ineffective because of the lack of deep integration with the local context. A friend of mine, Kerry Fosher, has done some great research in this area — and you can’t read her work without thinking about our problems in higher ed: http://aaanet.org/press/an/1005/Kosher_Lathrop.htm (again, I think there are some great lessons for us here).