The “Big Picture” category covers larger trends and topics that influence both the problems that technology can help address in education as well as the barriers to implementing high-quality technology-supported education. This includes research-based topics such as learning science and program effectiveness studies, philosophical discussions such as outcomes definitions, and macro-forces such as government policy, markets, and business models.
The current pandemic is forcing us to rethink much that we used to take for granted. This is true in EdTech as much as anywhere else. For example, while privacy advocates have been shouting into the void about how we should all pay more attention to seemingly boring and definitely complicated legal and technical privacy protections in the EdTech products we use, the Zoom debacle has finally brought more attention to the issue. To reduce the chances of many equally bad privacy problems going unnoticed, we must improve both the tech privacy literacy of people purchasing or selecting the products and the craft of transparency on the part of the product providers. University folk need to become better at reading privacy assurances critically while the service providers need to become better at writing those assurances clearly.
The same goes for the teaching value and effectiveness of products. While this term has been a scramble just getting students online and doing the best that we can for them, next term there will be less excuse for slapdash use of random EdTech products. We must get better at evaluating the strength of vendor claims regarding the improvements in student impact that educators can achieve using their products and, likewise, the vendors much get better at communicating their evidence of impact. This has always been true, but now it is also more obviously urgent. We need to get better at talking about what works, under which circumstances, and how we know what we think we know.
EEP sponsor Macmillan Learning, my most recent guest on the e-Literate Standard of Proof webinar series, talked about how they both test what they know about the effectiveness of their products and communicate what they think they know to customers in the crucial early stages of product development, when there aren’t enough data to make strong claims based on mountains of evidence. This work must be a collaboration with educators in at least two senses. To begin with, EdTech products generally rarely have a significant impact on their own. They usually have the most effective when they are paired with appropriate and effective teaching practices. So early EdTech product development work must involve working with faculty to understand both what they do in the classroom and what they might be willing to do try doing differently if they had a product with affordances that supported a teaching practice that is novel to them. Second, as product developers making increasingly confident claims about their product, those claims—and the process by which they were derived—must be reviewed by academic experts who can provide peer review to both the experimental design before it is used with students and the interpretation of results as the data come in.
As a (paid) member of Macmillan’s Impact Research Advisory Council (IRAC), I have had the pleasure of both observing and participating in their process. Their model of early research and transparency in reporting is a contribution to the field.
Here’s the interview:
Macmillan Learning on e-Literate’s Standard of Proof webinar
If your main intellectual response to the current health crisis is to double down on your previously held beliefs, then you are probably not thinking deeply enough. For example, suppose that we develop both treatments and a vaccine for COVID-19, but the immunity conveyed by that vaccine only lasts for a season (much like other flu vaccines). Suppose that flu strains become permanently more virulent and more deadly. Some of the changes in the healthcare system that we are seeing now—in point-of-care, supply chain, and research efforts—become permanent. The risk of getting together physically in groups is permanently altered. Not to the degree that it is now, but significantly higher than it was. Think about how that permanent change would impact all the many businesses that are being impacted now. The growth sectors of our economy would change. The geopolitical landscape would change. Everything would change. Much of what we thought we knew about the future of work would have to be thrown out and rethought. And much of what we thought we knew about the future of education—what jobs we need to prepare students for, how we should be teaching, what sustainability looks like for an educational institution, and so on—would also have to be thrown out and rethought.
It is hard to think such thoughts. I find my mind desperately casting about for reasons to believe that this scenario is not plausible. That life will someday, somehow, return to normal. That the world has not changed.
But it has. And we must all learn how to think the unthinkable. We have to find ways to let go of the comforting assumptions that enable us to glide through the day on autopilot, somehow not swerving off the road while we fiddle with the car radio or talk on the phone or think about what will happen when we arrive at where we are going. We have to look at the road with fresh eyes. See what is in front of us. Prepare our minds for unexpected obstacles. The adaptive cruise control and lane-assist and other comforting tools that we have built to protect us from the need to be in the moment will not serve us well on this road we have found ourselves on. In fact, they provide a dangerous illusion of comfort. We must seize the wheel and learn to be navigators again.
Right now, as we respond to an unprecedented crisis, a lot of the safety mechanisms in the system are being thrown out even as we face massive uncertainty. Multi-trillion-dollar legislation is being passed through Congress incredibly quickly. The medical system is finding various shortcuts to get around the system of checks that are designed to make sure treatments that tinker with your immune system are safe. These safeguards were created for good reasons, one of which is to mitigate the impact of sloppy or reflexive thinking on consequential actions. If we are going to remove them, then we must think harder.
Even under normal times, I would have been disturbed by the Christensen Institute’s argument that “California should double down on Calbright College—by leaving it alone.” The post author Richard Price asserts that California’s new, experimental online community college, which received a $100 million initial allocation and a $20 million annual budget from the California State Legislature, is already fabulously successful. And it will be even more successful if only it is freed from the shackles that the other 114 California community colleges manage to live by and State Senators would back off of their call for Calbright to be audited for its finances, progress, and its compliance with the conditions under which it was authorized. He makes these assertions without evidence or even much in the way of argument to back them up. While reading the piece, I can’t escape the feeling that I am in the back seat of a car with a habitually distracted driver who is reflexively trusting his GPS to guide him through territory that has not been mapped.
I have been patient with the more careful proponents of disruptive innovation in education, despite my skepticism, because it is not their fault that the term has been appropriated by testosterone-addled Silicon Valley teenagers, and because they have made cogent (if unconvincing) arguments. But the opening line of Price’s Calbright post, while sounding completely anodyne, alarms me:
As the COVID-19 outbreak increasingly prompts universities to move instruction online, California lawmakers may want to reconsider their deep skepticism of the state’s first fully online community college—the young institution is proving itself more vital than ever.
There are just so many things wrong with this sentence. For starters, the author links to an article about lawmakers’ “deep skepticism” without mentioning anywhere in his own piece that the article describes the reasons why lawmakers are skeptical. He certainly doesn’t attempt to refute those reasons. To the contrary, he asserts that the “young institution is proving itself more vital than ever” without ever really explaining what that means. Second, the sentence conflates skepticism about Calbright with skepticism about online education. Reading it, one would never guess that the California Legislature has poured something like a quarter of a billion dollars into online programs for the California Community College System alone, plus hundreds of millions more into the Cal State and UC systems. Lawmakers are not demonstrating skepticism toward online education. Rather, they are demonstrating skepticism of Calbright’s implementation as the best application of the substantial investment they are making in online education.
Worst of all, the introduction sets the tone for the article, which is to accuse others of dogmatism while itself making its case on pure dogma. In doing so, it links to another post on the Christensen Institute’s site by another author about another system. This article, unfortunately, also gave me the feeling of being an uncomfortable passenger sitting beside a driver who is barreling down the road toward a destination that he is determined to reach, heedless of what may lie between here and there.
The end result is a specious chain of arguments made from a position that indirectly has Harvard’s imprimatur—because Christensen was a noted Harvard professor—arguing that the state of California should commit to throwing tens of millions of dollars at an unproven program. And that it should do so while declining to hold that program accountable or even expecting it to follow the rules that all the other community colleges manage to live under. The main justification for this position is a theory that was designed to explain what happened to steam shovels and old disk drive technologies and is now being applied to a dramatically different domain without attention to context or evidence.
If you are going to argue for removing safeties at a time of historic emergency, then you must think harder.
It is time for the more serious proponents of disruptive innovation in education to take a good look in the mirror and ask themselves if they have been thinking hard enough. As their predictions about mass closures of universities and other dramatic changes have failed to materialize repeatedly over many years, their assertions about the future have become increasingly strident and sloppily constructed. It is time for them to seriously consider the possibility that they are just plain wrong, and that their attachment to cherished ideas is doing more harm than good.
My own prior assumptions
If I’m going to be fair, I should lay out my own prior thinking about both disruptive innovation and Calbright so that you can decide whether I am the one who is stuck in old thinking.
I am a fan of Clay Christensen’s work. I have three of his books on my shelves, all of which I have read at least once, and two of which I find myself occasionally dipping back into for reference or inspiration. I find his basic thesis to be clever, well-argued, reasonably grounded in evidence, and provocative. That said, his theory is interesting because of the inference chain he constructs rather than because any of his individual insights are particularly novel in and of themselves. For example, one central concept in disruptive innovation starts by “competing against non-consumption,” which is a fancy way of saying you should go to where the competition isn’t by figuring out who isn’t being served well. While this assertion is worth noting, it is the opposite of revelatory. It borders on the banal. Another stepping stone in his argument is that successful organizations will tend to resist ideas that are different than their historically successful ways of doing things. We nod at this without pausing. Christensen’s argument pulls us along not because the steps along the way stop us in our tracks but, to the contrary, we glide down an easy path and suddenly end up somewhere unexpected. As Price puts it in his article, “In the Innovator’s Dilemma, Clayton Christensen made an unsettling observation—CEOs of established companies that fell to disruptive entrants often did everything right, and for that very reason failed to save their companies.”
Disruptive innovation theory boils down to the idea that companies that are fat and happy tend to result in products that are fat and customers that are less happy. At some point, Microsoft stopped building features into Office that were useful to everybody and started focusing on features that were useful to their most lucrative niche customers. Office customers who were not in those niches became less happy as the software became more expensive and harder to use. But the company was focused on maximizing revenues from its best customers.
So they didn’t react much when Google came out with Docs, Sheets, etc. This was a perfectly reasonable business decision for them to make, since Google was not an immediate threat to their customer base and some very profitable corporate customers wanted specialized features. Google’s products were obviously inferior for mainstream word processing tasks. I tried and abandoned Google Docs a few times in the early years because it just didn’t meet my needs. But it got better over time, first satisfying needs for which Word wasn’t well suited, then nibbling at the simplest applications of Word, and eventually becoming sophisticated enough to challenge the dominant word processor for a substantial percentage of common tasks. Today there are a billion users of Google Drive. I am one of them. I have not abandoned Microsoft Office, but I have thought about it at times.
Disruptive innovation is both a particular kind of disruption and a particular kind of innovation. It is important to understand this. When we learn to trust a theory, we begin to take it for granted like we take our cruise control for granted. We stop thinking about it. But we need to understand how our tools work so that we can know when we are safe to trust them and when we are not. As I put it in 2014,
These days, “disruptive” and “innovation” seem to always come together in the same sentence. It’s a bit like “big galoot.” Theoretically, “big” is a modifier for “galoot.” But you never hear people talking about small galoots, average sized galoots, or galoots of indeterminate size. In modern common usage, galoots are always big. “Big galoot” has pretty much become an open form compound word, like “post office” or “real estate.” But “disruptive innovation” is not a compound word. Disruptive innovation is a particular kind of innovation, and a fairly narrow kind at that. Specifically, disruptive innovation is a phenomenon in which a new market entrant can overtake a leader in an established market by offering cheaper and simpler solutions. It’s important to remember that some of Clayton Christensen’s seminal examples of disruptive innovations were steam shovels and disk drives. This is not the same kind of innovation that produced the iPhone. It’s essentially about identifying the slow fat rich kid and taking his lunch money. To be fair, it’s not that inherently mean-spirited, because presumably one takes the rich kid’s lunch money (or market share) by providing solutions that consumers prefer. But the point is that disruptive innovation is generally not about solving new problems with brilliant out-of-the-box ideas. It’s primarily about solving old problems better because the old solutions have gotten overbuilt.
The reason that I have been skeptical about disruptive innovation’s applicability to education has to do with differences in both demand and supply from the situations that Christensen analyzed in his books. On the demand side, I argued in 2013 that education doesn’t fit the disruptive innovation framework because it is not a product:
Silicon Valley can’t disrupt education because, for the most part, education is not a product category. “Education” is the term we apply to a loosely defined and poorly differentiated set of public and private goods (where “goods” is meant in the broadest sense, and not just something you can put into your Amazon shopping cart). Consider the fact that John Adams included the right to an education in the constitution for the Commonwealth of Massachusetts. The shallow lesson to be learned from this is that education is something so integral to the idea of democracy that it never will and never should be treated exclusively as a product to be sold on the private markets. The deeper lesson is that the idea of education—its value, even its very definition—is inextricably tangled up in deeper cultural notions and values that will be impossible to tease out with A/B testing and other engineering tools. This is why education systems in different countries are so different from each other. “Oh yes,” you may reply, “Of course I’m aware that education in India and China are very different from how it is here.” But I’m not talking about India and China. I’m talking about Germany. I’m talking about Italy. I’m talking about the UK. All these countries have educational systems that are very substantially different from the U.S., and different from each other as well. These are often not differences that a product team can get around through “localization.” They are fundamental differences that require substantially different solutions. There is no “education.” There are only educations.
Students’ reasons for the college choices they make are much more complex and varied than their reasons for choosing a word processing program. So I’ve been skeptical about whether the nebulous cloud of ideas and ideals that we call “education” is a thing that can be disrupted in the Christensonian sense.
There are also mismatches to the theory on the supply side. Christensen’s theory of organizational behavior fits best with organizations that conform to economist Ronald Coase’s theory of the firm. Command-and-control organizations that are optimized for profitability and growth will be particularly vulnerable to the pathologies that he identifies. Most universities don’t function that way, and no university systems do. While universities do have to worry about optimizing for sustainability, that goal is complicated by their strong mission focus and their shared governance structures.
And university systems are even further from Christensen’s model because they are governed by political systems, which operate under entirely different decision-making processes than companies do. For example, with 114 physical community colleges grouped into 72 districts across California’s 58 counties, every state legislator has a local interest in protecting one or more of those colleges. There were mission-related reasons for this design decision, having to do with making education accessible in an era before modern online learning. Regardless of the degree to which those reasons may or may not still apply, decisions for the California Community College System are still driven by political representation of those county interests in the state’s legislature. Christensen wrote an entire book—The Innovator’s Solution—on how entrenched corporations can stay innovative. As far as I can recall, none of the example organizations in that book looked remotely like the California State Legislature. So at the very least, somebody would need to make the affirmative case that Christensen’s prescriptions for corporations can be applied in this very different context.
And remember, it’s the chain of inferences that makes disruption theory interesting. If any of the links break, then we have to question whether the theory holds. Cherrypicking evidence for one or a couple of the individual common-sense assertions in the inference chain puts us on a fast road to Stupidville, however our GPS display may label the destination.
I have historically been skeptical of Calbright for reasons that have less to do with disruptive innovation and more to do with math. COVID-19 has been giving us all a hard lesson on the mathematics of growth. If every person carrying the disease spreads to between two and three other people, then the number of people with the disease will double every few days. On the other hand, if the average infected person spreads the disease to at most one other person, then it will eventually die off.
The challenge of Calbright has been that the math never worked for the kind of growth that the legislature expected it to achieve. If the main goal was to create an incubator that would eventually teach us better ways to serve underserved students, that would be one thing. But the goal was to actually serve millions of underserved students in a relatively short span of years. Calbright’s growth rate would have to be unprecedentedly high in order to achieve that aim. Phil Hill broke it down in a post in 2017:
The maximum growth rate of [some] cherry-picked successful schools ranges from ~1,200 / year for Excelsior to ~7,700 / year for SNHU (note that Rio Salado at ~1,400 / year is the only public institution). Add to this the fact that all of these schools have been around for decades. No accreditation issues, no time-consuming establishment of core leadership team, etc.
There is a big difference in dealing with institutional issues and statewide issues, particularly in California. One in five US community college students in the US do so in California, and the statewide issues tend to come in large numbers. Statewide issues tend to come in hundreds of thousands while institutional issues tend to come in tens of thousands.
What this points to is that for a new fully-online institution to get to some meaningful level of enrollment (let’s say 20,000) in the same ballpark as these comparison schools, I estimate it would take a full decade at the least. This is the reason, by the way, that Mitch Daniels and Purdue University made the Kaplan University deal even though Kaplan’s enrollments are dropping. Daniels did not want to wait a decade to get to meaningful enrollment numbers for an online college serving working adults – if everything works out, within a year Purdue will have a fully-online institution serving 30,000+ working adults. That is a big if, by the way.
I have no problem with the idea of Calbright as an experiment in supporting underserved students. But the math doesn’t support the idea that it is going to have an impact at scale any time in the next decade, even under the most optimistic of scaling scenarios. So my concern with Calbright is that both California politicians and proponents of disruption theory have seized on it as a means for solving different problems than the ones it seems suited to solve.
I have one more predilection that you should know about, which is that I believe theories should be disprovable if they are to be credible. You can’t say that disruptive innovation is going to result in 50% of schools going bankrupt—no, wait, maybe it’s 25%—and that it’s MOOCs that are the disruption—or maybe just online learning in general, or possibly CBE, or maybe adaptive learning—and continue to be taken seriously. There has to be a way to arrive at the statement, “data X would disprove my hypothesis.” It can’t be the case that any old data can support your hypothesis while no data can disprove it. That’s not a theory. It’s dogma. My patience with proponents of disruptive innovation in education has worn thin over time because I haven’t seen them demonstrate the willingness to ask themselves the question, “But what if we’re wrong?”
Now that I have laid my own comforting assumptions on the table, it’s time to examine the assumptions that are evident in the arguments from the Christensen Institute.
Is Calbright a disruptive innovator?
Let’s return to the hyperlink in the very first sentence of Price’s post, which is attached to the words “their deep skepticism.” What is the skepticism described in the Education Dive article behind that link? First, the article emphasizes the question of whether Calbright would be best run in the existing structures rather than as an independent campus. That’s one surface resemblance to disruption theory. Again, you don’t have to believe in disruption theory to believe that sometimes a little autonomy from a large, entrenched bureaucracy might lead to fresh thinking. But at least it’s consistent.
On the other hand, Price neglects to mention other salient facts. Like the fact that the legislative analyst’s report in 2018 said there is no evidence that Calbright will solve the problems that prevent Californians from enrolling in college (or, in Christensonian parlance, that it can compete against non-consumption). Here’s the relevant paragraph from that report:
Unclear If Providing Online Offerings Will Solve Key Barriers for Target Student Group. One of the proposal’s goals is to increase educational attainment for adults who currently have no postsecondary credentials. Although this is a laudable goal, the administration has not provided any evidence that an online community college will address the key barriers for this potential student group. Although an online program can increase convenience, working adults may not be pursuing additional education for a number of reasons. The administration also has not provided evidence that those working adults who are interested in more education cannot access it through existing online or in‑person community college programs.
California LAO report
Is there reason to believe that the innovations of Calbright’s design will cause large numbers of students who were previously not enrolling in existing online offerings to enroll in Calbright’s? I don’t know. The California LAO’s office didn’t see such evidence, and Price fails to even mention it as a problem.
The Education Dive article also mentions that Calbright has duplicated existing academic offerings of other community colleges in violation of state rules. Here Price does have a rejoinder, though he does not dispute that Calbright has violated state rules:
[C]oncerns that Calbright’s courses may duplicate existing OEI offerings fail to acknowledge that competency-based, self-paced courses differ fundamentally from seat-time based courses covering the same material, and better serve time-constrained adult learners.
OK, that’s a claim. Where is the supporting evidence? Such programs exist in the world. Western Governor’s University, SNHU’s College for America, and others implement the instructional approach that Calbright has chosen. I’m not arguing that evidence doesn’t exist. But I do think that if somebody is advocating for a $100 million initial allocation plus a $20 million allocation in support of a new and unproven institution while simultaneously advocating that rules should be relaxed and oversight waved off, then one should present any evidence so that we can examine it together. The action Price is calling for requires less reliance on cruise control and more on the blind-spot camera.
Rather than focusing on making an evidence-based case for Calbright, Price chooses to make an argument against California Online Education Initiative (OEI):
Consider CCCS’s Online Education Initiative (OEI), a course exchange program, which Calbright critics point to as obviating the need for a separate, online college. In theory, students can enroll in online courses offered by other state community colleges. In reality, CCCS’s enrollment-based funding model discourages school participation, since only the college that offers the course earns enrollment revenue. This has limited course offerings and engagement overall.
Here is an example of how Calbright, operating under a different business model, could be an asset to CCCS as an autonomous institution. Free from the constraints of the funding model that has proven incongruent with expanding online course options, Calbright could work towards revenue and course enrollment models that best serve large numbers of job-seeking adult learners.
This is factually wrong on several levels. First, the course exchange is just one component of OEI. Luckily, MindWires—the consulting practice that I used to be a partner in and that Phil Hill still runs—produced this convenient explainer on behalf of OEI (which is less than four minutes and is easily findable on YouTube):
Intended Consequences
(Video link: https://youtu.be/1DdlaIZYiDI)
As Phil recently wrote, Calbright is currently serving about 450 students. In contrast, in the 2020-2021 academic year, OEI projects 99,000 enrollments in new academic pathways. (And word is that the OEI number is likely to be revised upward significantly based on more recent data.) Apples-to-apples, this translates into probably a couple of tens of thousands of students.
The course exchange part of OEI is one component. It may or may not succeed at scale. But the broader OEI program is indisputably reaching many more students than Calbright. Worse than getting the incentive mechanism wrong for the course exchange, Price apparently failed to conduct even cursory research regarding the nature and impact of the program he was critiquing. Even if the course exchange portion of OEI fails utterly, the overall program is still tracking to increase enrollments by two orders of magnitude more than Calbright will in the next year.
That doesn’t even count the ways in which Calbright’s progress toward (unspecified) milestones was accomplished, in part, by using resources built with OEI funding. Here is a screenshot of Calbright’s essentials course:
The link at the bottom of that screen leads to this page:
Of the Calbright students enrolled in October through December of last year, 449 enrolled in essentials courses, while 20 enrolled in a program pathway. In other words, the vast majority of students taking Calbright courses last term were taking courses that were using OEI content resources.
Also, while Calbright’s mission was specifically to reach underserved students that aren’t being reached via the 114 existing community colleges, the results so far are not promising on that score. For example, despite specifically targeting Latinx students, they represent 16% of Calbright’s enrollments—in contrast to 45% Latinx enrollment system-wide. Calbright’s students are, in fact, disproportionately white:
In addition to these reasons for questioning Calbright’s progress toward and ability to accomplish the mission set for it by the State Legislature, there’s also the fact that Heather Hiles, former Calbright CEO, resigned abruptly without public explanation less than a year after being hired.
So given these facts, is it reasonable for California to ask whether Calbright makes sense and is working as intended at this time? I think it is. While I don’t prejudge the outcome of that introspection, the legislators have a responsibility to at least ask the hard questions.
But, worries Price, whether or not OEI may work in practice, does it work in theory? He seems to take a kind of libertarian position that any program that is regulated will inherently be inferior to one that is not. He is aghast at the LAO’s concern that Calbright will eventually have to comply with the system’s collective bargaining agreements and spend 50% of its budgets on instructional salaries.
In other words, Calbright being embedded in CCCS subjects it to strictures that will eventually hamstring the school in its efforts to innovate for adult learners. The LAO report ironically makes the argument that even as an independent school, Calbright isn’t autonomous enough.
Does it? How much does it cost to run a Calbright-style CBE program with quality? How much of that money goes into instructional costs? How does that compare to the requirements that it is currently required to comply with and that OEI online programs do comply with? Again, the data exist, but Price either doesn’t believe he needs to justify his position or doesn’t have the data in question.
I don’t have it either. What I do have is a quote from the President of Southern New Hampshire University (SNHU), which the Christensen Institute valorizes without understanding—more on that in a bit—about the amount of human time and attention that is required to deliver quality CBE education:
I think there is a general lack of awareness of how rich now the underlying data analytics are. We monitor our students 24/7. We know when someone hasn’t logged on. We know when someone has struggled with a project. We know when performance has dropped off. We actually have closer to a 360-degree view of our students than most traditional institutions do. Then, when those students are engaged in the work, they have ready access to qualified faculty if they’re really stuck. We’re never going to let somebody get stuck on a math concept, for example, and just say, well, just figure it out. We’re going to get you help.
SNHU is putting significant time and money into instructor support. Is it 50%? I don’t know, and Price apparently doesn’t either. Is there a case to be made for granting Calbright some flexibility based on evidence of results in successful programs? Possibly, but Price doesn’t make it. Instead, he simply declares,
Without an autonomous Calbright, CA will struggle to properly serve its adult learner population. Innovators like WGU and SNHU will take good care of these learners, but CA will have ceded, rather than seized, the future of learning.
With little in the way of argument and nothing to speak of in the way of evidence, he concludes that (a) only Calbright can “properly” serve California’s adult learner population, (b) Calbright can only do so if it is “autonomous,” (c) Calbright represents “the future of learning,” and (d) if the California Legislature doesn’t cede oversight Calbright while continuing to fund it, then out-of-state competitors may disrupt or otherwise somehow surpass the California Community College System. It’s not at all clear how Price got us here, or even what he means by “properly” serving California’s adult learner population, or making Calbright “autonomous,” “ceding” or “seizing” the “future of learning,” or what role he believes that “innovators” such as WGU and SNHU will play in California’s future should the State Legislature fail to follow his prescription. He does link to (without really explaining the relevance of) another Christensen Institute post by another author about how the PASSHE system partnered with SNHU. I wish I could say that I found that article more persuasive.
Alas.
Is SNHU disrupting PASSHE?
Michael Horn’s post from February of this year bears the breathless title, “Why disruptive innovation is stealing Pennsylvania’s students.” The trigger for the article was an articulation agreement signed between Pennsylvania’s State System for Higher Education (PASSHE) and SNHU which enables PASSHE students to transfer up to 90 credits to SNHU and complete their bachelor’s degree at SNHU at a 10% tuition discount.
Horn emphasizes his belief that state regulations and slow-moving bureaucracy have hobbled PASSHE’s ability to innovate in online education, leading it to the necessity of outsourcing. He does so mainly by analogy to SNHU. There are several problems with this approach.
First, as I pointed out earlier, the idea of increasing autonomy to spur innovation was not newly introduced by disruption theory. Bell Labs embodied this strategy from its formation in 1925. Clayton Christensen wasn’t even born until 1952. Second, as I also pointed out earlier, the success of this strategy does not validate all of disruption theory, never mind its applicability to education. As Horn acknowledges in his post, PASSHE is not failing because its leaders did everything right in pursuing its most lucrative customers. (Not that we would want public education systems to do that anyway, which is yet another problem with disruption theory’s applicability.) To the contrary, PASSHE arranged for the articulation agreement because many of its (tuition-paying) students are not finishing their degrees, according to PASSHE Chancellor Dan Greenstein. Presenting those students with another reasonably priced online option was intended to increase student graduation rates, even at the possible cost of the system’s most lucrative customers. We have to follow Horn down a chain of several inferential steps—each of which is given with either thin factual support or none at all—to arrive at the implication that “disruptive innovation is stealing Pennsylvania’s students.”
Horn relies on a loose comparison to SNHU to do most of the work from him. To give him the benefit of the doubt, let’s stipulate several points:
SNHU’s growth has been undeniably impressive at a time when PASSHE has been experiencing a slow-motion crisis.
SNHU President Paul LeBlanc, like me, is a fan of Clay Christensen’s work. He has thought about it a lot and credits it with influencing his thinking.
Even so, it’s not that simple. LeBlanc hasn’t applied the separation principle dogmatically. SNHU operates under one shared governance structure. Autonomy is not the same as freedom from rules or oversight. Second, he has not hesitated to revoke autonomy when it wasn’t working. SNHU’s competency-based education unit, the College for America, used to be autonomous but has since been pulled back into the larger online learning unit.
Overall, Horn’s article is a confusing muddle. He readily acknowledges that the causes of PASSHE’s troubles look nothing like the causes of disruptive innovation described in The Innovator’s Dilemma. He also admits that better funding of the chronically underfunded system would help. He then swerves off into an explanation of disruption theory, connected only by the thread that SNHU employed the same basic strategy that Bell Labs used successfully nearly 30 years before Clayton Christensen was born. He ends with a call for state systems to create small incubators where new ideas can be piloted. I agree with that idea, but I don’t think it bears any relationship to the completely unjustified clickbait title of his post. I don’t think one has to believe in disruption theory to come to the conclusion that university systems have become sclerotic and need to find ways to get out of their own way. This could have been a perfectly good article if it weren’t trying so desperately hard to validate disruption theory. Horn’s rather modest call to action is not in any way proportionate to the dire pronouncement of the headline; nor does it justify Price’s call for Calbright to be exempted from oversight or collective bargaining rules before it has even proven that it can serve 1,000 students.
This is all hauntingly familiar. I mentioned that I own three of Christensen’s books, two of which I occasionally dip into to this day. The other one is Disrupting Class, which is the book that Christensen co-wrote with Horn about the application of disruption theory to education. In stark contrast to The Innovator’s Dilemma and The Innovator’s Solution, it was a disappointing mess. Parts of it attempt to stretch disruption theory beyond recognition in order to fit educational examples. Other parts contain thoughtful, reasonable observations that bear no credible connection to disruption theory. It doesn’t hang together as a cohesive work. I found the book unpersuasive when it was published 12 years ago and do not believe it has aged well. And in subsequent publications by disruption theory proponents in education, the flaws that were apparent in that original work have been papered over rather than addressed. Christensen has passed away, but the people carrying his torch in this domain have not done his legacy a service, however diligently and earnestly they may have tried.
Twelve years of trying is enough. It is time to think harder.
Grieving our errances
Alexander Pope wrote that to err is human, but he might have added that to err repeatedly due to habits of mind is particularly human. The disruption theory proponents love their idea. As a man who has written 7,000-word blog posts, I cannot credibly claim immunity from that particular siren song.
It is not a propensity to be ridiculed. Our capacity to fall in love with a thing as aetherial as a concept has been a key to our survival as a species, and to creating lives for ourselves that are about more than just surviving.
Sadly, we have not evolved to feel equal love for the process by which we produce, examine, and sometimes destroy ideas. We became toolmakers because tools enable us to accomplish more work with less effort. Ideas are tools that enable us to accomplish more work with less effortful thought. To err is human; to think is divine. These days, I am less inclined toward mocking people for being passionately wrong than when I was young enough to believe that only other people ever commit that sin. But I will attack their arguments without hesitation or mercy when I think they are harmful.
It has become clear to me that all talk of disruptive innovation in the context of education is harmful. Not just when applied thoughtlessly by people who don’t understand the theory, but even when applied diligently by people who do. It is harmful to advocate for a failed idea as if the fate of education depends on believing it is true.
Disruption theory in education is dead. Be sad if you need to, but move on. Seize the wheel and learn to be a navigator again.
As for Calbright, I remain agnostic. I believe it is a good idea in principle for California to create a safe space to experiment with alternative means of reaching the millions of Californians who need education but aren’t getting it. I worry that this characterization is not fully aligned with the stated goals of the Calbright program. I do not know whether the experimental college is being run well, or whether now is the best time to prioritize it over other educational exigencies of the moment. I do believe it is not only fair but important to ask these questions and insist on arriving at fact-based, well-reasoned answers.
I have organized my last two posts into the beginning of a series, which allows me to create a running table of contents on the relevant blog posts. This post is the third in a series focused on the creation of an education resilience network that can respond more effectively in times of crisis, whether the crisis is a pandemic, a hurricane, a war, or some other disaster that disrupts the normal functioning of educational institutions. I will have at least one announcement to make regarding the formation of such a network very soon. In the meantime, I will be using this series to lay out a vision for what such a network could look like and how it could function. In the context of the current COVID-19 crisis, my goal is to organize a better response for next term while providing some tips to the people and organizations that are working on an immediate response regarding how to increase their impact (e.g., putting an explicit and permissive license for re-use on any resources that are intended as donations or contributions to the community).
As a community, we’re learning a lot about what a good (or bad) disaster response looks like by watching our governments and health systems. The logistical challenges are daunting. We have to produce good information resources for everyone, like what hygiene measures work and how to practice social distancing (or how to use a web conference system or develop a rhythm for working in an online class). We have to make sure the information about such effective practices is accurate and gets to everyone. We also need to get more specialized information to caregivers, like how to protect themselves and prevent disease transmission in hospitals (or how to create an effective online lesson). Then there’s information that needs to go to people who run care-giving institutions, like how to get test kits (or access to increased online teaching infrastructure). All of this needs to be accomplished consistently, accurately, and efficiently to reach everyone who needs accurate and useful knowledge. Some of that knowledge, like proper hand-washing technique (or how to create an effective instructional video) has a long shelf-life, while other knowledge, like how healthcare workers (or educators) should respond to local needs based on changes in available resources and supplies, is continually evolving.
Beyond creating and distributing information resources that are helpful—and combatting the spread of misinformation that is harmful—we also have to identify and distribute other resources. Help, either in the form of consultative expertise or supplemental front-line workers, is a crucial one. We need to match available resources with current needs quickly and effectively, all while providing some means for vetting the help to make sure that the people, however well-intentioned they may be, are qualified to offer help. And then there are non-informational resources, whether they are N95 masks or web conference accounts. Some of these will be new resources that people need for the first time, while others will be providing financial relief to prevent the loss of ongoing access to current resources.
How well we respond to the current crisis can make an enormous difference that differentially impacts more vulnerable populations. In an epidemic, the homeless, prisoners, and people without health insurance are among those who are least likely to be protected by our existing response mechanisms. Likewise, we know that first-generation students are more likely to struggle in online education programs even when those programs are carefully designed and not thrown together in haste. Our response to the current crisis can also put us in either a better or a worse position after the immediate crisis has passed. We can pass emergency economic legislation that protects workers and preserves jobs, or we can pass legislation that enables companies to use bailout money for stock buybacks and executive bonuses. Likewise, we can engage with educators on how to serve their students more effectively online, or we can just start spending money on a whole lot of Zoom accounts and robot tutors when the free offers run out.
I don’t intend to denigrate the efforts or good intentions of anyone’s current response to the COVID-19 situation when I write this, but if our crisis response capability does not evolve past Google Sheets, webinars, and limited-time free product offers, then we are going to fail to support the people with the greatest need during this extended crisis while also failing to learn how to respond better to the next one.
In this blog post, I’m going to write about the different kinds of resources for which we need to develop a supply chain in an education resilience network. In the next post, I will write about the challenges of creating, vetting, and distributing those resources.
Content
By content, I mean information that can be broadcast one-to-many rather than person-to-person expertise sharing. This encompasses a vast array of information resources targetted at distinct but overlapping audiences, including (but not limited to) the following:
Technical information, like how to post an announcement in your LMS
Learning design information, like how to make better use of formative assessments in an online context
Policy information, like guidelines for what instructors can and cannot expect their students to be able to do during the crisis
Logistical information, like where to seek help for particular kinds of problems
These resources can be either durable or not and either general or context-specific. Different characteristics call for different responses:
[wptb id=11021]
We should be mindful during this time of massive change that now is the time when new practices—either good or bad—take hold. For example, we know that instructors only tend to redesign their courses substantially when there is an extrinsic driver forcing them to rethink how they teach. We are currently experiencing the mother of all extrinsic drivers. We should be taking the opportunity to make sure we are teaching them about evidence-based effective practices. This is, in part, an equity issue. Disadvantages students are disproportionately vulnerable in the move to online. We lose them more easily. As a sector, we have an ethical responsibility to ensure that we are applying everything we have learned about educating well in order to help these students. That is even more important at a time when these very same students are particularly vulnerable in the rest of their lives, such as during a global pandemic that is shutting down the economy.
Resources
In an educational resilience response to a crisis, where the emphasis is on moving students to online, there will be a need for technology as well as information. People should have defined resource centers where they can go to find what they need and get advice about how to use it. I’ll offer one example. I am getting inundated with messages about COVID-19 resources and offers of help that various folks want me to help them promote. While I could easily turn e-Literate into a running announcement board for such offers, I have resisted that temptation. It’s not what e-Literate does best, and I see no point in duplicating work that others are already doing. For example, ISTE and EdSurge include a directory of vendor offers on their Learning Keeps Going web site. They have the resources to maintain such a directory and are a natural home for it. We should just use that rather than creating something else (or 100 something elses).
I would love to see them include guidance to both schools and vendors about topics like how to design and vet such offers to address student data privacy concerns. In other words, there is a whole set of practice knowledge that attaches to this kind of a list. To the degree that one organization, network, or community becomes a hub for a resource (whatever that resource may be), it should also become a community hub for sharing information about the effective use of that resource.
In this way, an education resilience network isn’t a hub-and-spoke kind of centralized response but rather a network consisting of multiple nodes of expertise concentration that are coordinating with each other as needed. I’ll have more to say about this in my next post.
People
This is an extension of my previous paragraph—including my comment about having more to say about this in my next post—but the most precious resource we need to coordinate is humans who can help. Many organizations are stepping up to play a role in matching helpers with need, but (a) a lot of companies and universities are just tossing information out into the ether without the benefit of coordination, and (b) the associations and other community nodes that are coordinating with their constituents are not coordinating with each other. This is the hardest part of the education resilience network to optimize, but it is also the most important. Ultimately, we will find that effective content and resource production and distribution in an emergency are second-order effects of the ways in which we organize ourselves—or don’t—for a whole-sector response to the crisis.
Resource needs are also dynamic. At this moment in the midst of a crisis for which the education sector has not been well prepared, I find that most people I talk to fall into one of two categories. Either they are about to drop from exhaustion because they are frantically working the front line or they are climbing the walls trying to find some way to be useful because the work that they normally do to help has ground to a halt. We are doing a very poor job of coordinating those imbalances at the moment. And they will change as we finish the current term and people start preparing for the next one. We need organizational structures that enable us to balance human resource utilization—matching people who need help with people who can offer help—more dynamically.
CC BY Aaron McCollough (adapted from a table by Anita Walz)
I’ve started sorting through the generous mess that is the COVID-19 educational resource sharing with an eye toward a lasting resiliency network that I wrote about yesterday. One good place to start (of many, I’m finding) is Lumen Learning’s list of recommended resources. ((Disclosure: Lumen is a sponsor of the Empirical Educator Project.)) While Lumen is sharing their curation free with everybody online, one of the strengths that the commercial providers offer during this time of crisis is that they also often have fairly direct lines of communication with their customers. Their survival depends on their ability to cut through the noise and reach particular people with well-targeted messages. In times like these, their communications and marketing teams can be turned into information distribution networks. We need to reach educators efficiently with high-quality information. We need these distribution networks.
Right now, a lot of EdTech companies are doing what Lumen is doing. They are curating their own lists of content and getting them out to their customers. Many of these resources are being produced by and for academics. The companies are curating and distributing. Ideally, we would be working as a sector consistently, over time, to create and organize educational resiliency content that organizations with effective distribution capabilities. These could be commercial providers, universities, or professional associations. The most important thing is to get high-quality information to people who need it at their moment of need.
In this kind of situation, we should reduce the friction of sharing as much as possible. One source of friction is licensing. People that want to share high-quality content created by others need to consider whether the creator has granted them the right to do so. That is the law, and it is also the right thing to do. But it is a source of friction to sharing. If your top priority is to create content that will reach and help as many people as possible, then you should reduce that friction as much as possible.
There is a simple way to do that. First, make sure that your content bears an explicit license that grants sharing rights. Don’t make people come to you and ask for permission. Second, choose the most permissive license that you can justify. The most permissive license possible would be a public domain mark. This simply says, “I am giving this to the world and don’t need credit or payment. Please use it.” The second-most permissive option would be the Creative Commons Attribution license (CC-BY). This license says, “I am giving this to the world, and all I ask in return is that you give me credit for my work.” Either of these licenses will enable the broadest range of distribution networks to pick up your contribution with as little friction as possible.
I do not recommend Creative Commons license conditions such as Non-Commercial or Share-Alike for this kind of resource. Non-Commercial (CC-BY-NC) is the worst. If a commercial entity shares your CC-BY-NC-licensed resource to their customers and they don’t charge for it, is that commercial use? Nobody knows. So commercial entities have both legal and ethical reasons to be cautious about sharing CC-BY-NC resources. Share-Alike (CC-BY-SA) is not quite as bad but also problematic. Again, if a commercial entity, or even a professional association, wanted to bundle your resources with some of their copyrighted material in order to help educators in their time of crisis, could they do it under this license? Do you want to prohibit that use?
The Non-Derivatives clause (CC-BY-ND) if you’re worried about a piece of information with your name on it being chopped up in ways that you don’t approve. But every restriction clause creates a trade-off in terms of broadest use. Please think carefully about potential trade-offs with the ones that you impose.
I am not making the blanket argument that all educational resources should be licensed CC-BY or public domain (although the argument could be made). I am urging my readers—both academic and commercial—to explicitly contribute as much of your crisis support content as possible under as permissive a sharing license as possible.
Friends, the first thing I want to say to you is that you are in my thoughts. As I watch so many of you make heroic efforts to support your students and colleagues even as you worry about yourselves and your loved ones, I am inspired. My sister, who is both a teacher and a rabbi, has banned the phrase “social distancing” from her vocabulary, preferring to use the term “physical distancing.” She is right. Now is the time to be closing the social distance among us, not increasing it. That is exactly what so many of you are doing. You have inspired me to think more seriously about what e-Literate should be doing to help.
That’s what this post is about. Feedback, as always, is welcomed.
Immediate term: postponing a webinar
Right at this moment, the most useful thing I can think of to do is to stay out of your way. After discussing the situation with Standard of Proof webinar guests Tyton Partners, we agreed that this week is not a good time for the conversation we had planned to have. It’s unfortunate because their research has a direct bearing on how to better support the students who are most likely to be hurt in a sudden, unplanned transition to online education. But this week, many of you are either preparing for that transition, in the middle of it, or trying to deal with the immediate aftermath of it. You have more time-sensitive priorities at the moment, including self-care.
We will set a new date for the webinar once the dust settles and it becomes clear to us that more of you are in a position to have a constructive conversation about something other than your immediate responsibilities. I don’t think we’ll have to wait terribly long for this, but it won’t be in March.
The role of the Empirical Educator Project (EEP)
I have come to the conclusion that EEP can play a supporting role in this crisis. Therefore, it should. I am embarassed to admit that this wasn’t my idea. I got a text from a friend who is also an academic participant in the EEP network. When I didn’t catch on to his suggestion immediately, he found time to call me, after a full day of work, while he was cooking dinner for his family, to explain his thinking and urge me to consider the possibility.
Among other things, our response to COVID-19 is a collective action problem. But in education, there is an unusual amount of goodwill and agreement about our high-level collective responsibilities. The real problem is that there are costs to collaboration, and the coin in which those costs most often need to be paid is the one which educators are sorely lacking: time. The sector has considerable expertise on how to deal with this crisis as competently and humanely as possible under the circumstances, but it is not evenly distributed and it is not being distributed effectively at the moment.
I’ll give you one heartbreaking example. Some of the good folks in the POD network have taken precious time away from their crisis management at their own institutions to organize a list of information resources about remote teaching continuity plans that their respective institutions have put together so that their colleagues at institutions that don’t have such plans can benefit from them. About 200 institutions have contributed so far. Two hundred people who are directly responsible for helping their institutions make the transition right now took a moment out of their long, long days to share with their colleagues at other institutions. They are doing what they can. It is not their fault that the value of this information resource is severely limited. First, a lot of the people who need it aren’t going to know about it. It’s in a Google Sheet. One of a bazillion. Every time I tell people about it, they say, “Oh, is that the one that Bryan Alexander is organizing?”
(No, that one is here. It has over 250 contributions. I don’t know how much overlap there is between Bryan’s sheet and POD’s.)
Second, while the list is better than nothing, using it is far from easy. Some of the links are to resources that are behind a login. So they’re useless to outsiders at the moment. Others go to sites with varying amounts of information of varying quality covering varying topics. Somebody trying to make use of the resource in its current form would have to keep clicking through to web sites and sifting through them until they found what they needed. Maybe.
And yet, this is hardly the first situation that has called for a more organized response to helping fellow educators. Where was this level of effort for the universities in Puerto Rico during the hurricane, for example? Nor will this be the last such crisis. Both natural and human-created disasters disrupt the ability to support students somewhere in the world all the time. The pace of these disruptions is not going to slow down any time soon. The sector needs to become more resilient. We need to find ways of lowering the cost of educators helping each other so that we can achieve more impact with less educator burnout.
The Empirical Educator Project was designed to address this kind of collective action problem. Its founding hypothesis is that, while higher education is in the early stages of making the transition from a philosophical commitment to student success toward operational excellence at enabling it, this transition is being both slowed and deformed by ineffective sharing of actionable knowledge. This opens the door wider for both dangerous shortcuts and harmful opportunism. The problem we face with COVID-19 is the same, only accelerated. There is both opportunity and danger here. The opportunity is that the stress on our system is making the flaws more obvious and gives us a chance to optimize the system for the challenges of the present and future. The danger is that the imperative to act quickly will lead us to act unwisely. We need to both lower the cost of effective knowledge sharing and put some ethical guardrails in place.
This is what EEP will try to help with.
Short-term: Improving the vendor response
Let’s be honest: While educators are going to try to do their best they can for their students, the likelihood that most students will lose educational quality this term is pretty much 100%. No EdTech magic will change that. There will be no miracle “vaccine” this month or next. Everyone will do what they can. The short-term priorities are preventing deaths from the disease and supporting each other, including but not limited to our students, in getting through the immediate economic and mental health consequences of the crisis. The majority of vendors would have very limited ability to help this term even if they come to the table with the purest of intentions and a near-perfect understanding of what is needed.
That said, there are good reasons to engage responsibly with vendors right now. For one thing, while some of them can’t help with the problems of this term, others can, in ways that might or might not have anything to do with products. A good friend of mine who is also the CEO of an EdTech company told me that he has smart employees who are sitting on the bench right now due to project delays caused by the crisis and would like to see them volunteering rather than sitting on their hands. Meanwhile, I’ve been having calls with a number of companies that are looking for guidance on how they can be more helpful. Also, even if a vendor can’t help with the problems of this term, there is time to organize a better response for next term, when there is a non-trivial chance that we will still be dealing with COVID-19 challenges. And even if we aren’t, we should be preparing for the next crisis as part of our response to this one.
Expanding the sponsorship base of EEP was always part of the plan. While I have been very careful in the early years to only accept sponsorship either from companies that have already demonstrated they are aligned with EEP’s goals and values or particular people at companies who I know and trust, my ultimate goal has always been to help a wide range of EdTech companies that want to be better servants to education learn how to do so (and be rewarded proportionately for good behavior). I have decided that the current situation calls for me to broaden the tent a little earlier than planned. If a company comes to me for support in learning how to be more helpful, if I perceive that they are coming to me with a baseline level of earnestness, and if I can see a way to help them do the right thing, then I will help them.
I’ll provide more detail on what “doing the right thing” means to me in the next section.
Medium-term: Fostering a resilience network
As I wrote in the previous section, we are likely to face continuing COVID-19 challenges in the next term and certain to face challenges from other crises into the indefinite future. While I mean no disrespect to the good people who are organizing the crisis response, we need to get further along than a bunch of separate Google Sheets floating in the ether. We need to get better at sharing effective practices and supporting policies and at distributing knowledge of such things more broadly. Where there are resource gaps, we need to identify what they are and advocate for filling those gaps as part of our coherent response plans. This is primarily about people, knowledge, and organization. The technology should be an enabler of practices that educators have reason to believe are effective. It is crucial but second-order. The first-order problem is the creation of a resilience network.
Right now, most educators I know are focused on the immediate problem and can’t spare much thought for next term, never mind the indefinite future. It’s all many of them can do to put their line in a spreadsheet or make a call to a friend who they think can help. But that will change—somewhat—and we need to be ready to make the most effective use possible of the work that will come with the after-action reviews and the prep for next term. I expect to engage the EEP network—and the wider academic world—in this conversation when they can come up for air.
In the meantime, as I talk to the vendors, my focus is going to be on supporting that network. I’m not interested in creating a laundry list of limited-time free offers (though I don’t pass blanket judgment on such offers either). Rather, I’m interested in finding ways that the vendors can support the academics in diffusing relevant knowledge more widely and supporting each other more effectively. For this reason, I will be mindful about offers to use platforms. I don’t eschew them by any means. Google Sheets is a platform, and it is better than having nothing at the moment. But I will be focused on using platforms when they support the network purpose and consider the tradeoffs carefully. Laura Czerniewicz has a good guest post on PhilonEdTech (POET) about her experience thinking through such tradeoffs from an academic perspective (among other relevant topics).
I know there is a wide range of feelings in the academic community and I don’t claim to have the best, one-size-fits-all answer. But I can play the role of an honest broker. My personal experience in the conversations I’m having with folks who work at EdTech companies right now is that they want to help. They feel that if they can, then they should. But it’s hard to figure out how to be helpful. If I can help to reduce the social distance between them and the people they want to support in this time of need, then I will. Some of that support will come in the form of help now, but a lot of it will be focused on preparing for the next round.
Educators need to lead on this, and they can’t do that effectively while also responding to the immediate crisis. Those of us who are not immediately engaged in addressing the crisis need to prepare to help those folks make the most of their efforts going forward.
I have some announcements and updates to share regarding the Standard of Proof webinars. All good stuff, which I’m sure we could all use more of these days. As a general reminder, Standard of Proof webinars are always about contributions that vendors have made to our general knowledge or capabilities for supporting students—i.e., contributions that don’t just help their own customers—and they always include academics who were meaningful partners in the work. Companies featured in the Standard of Proof webinars are all sponsors of the Empirical Educator Project (EEP), but sponsorship does not guarantee a webinar. Only a qualifying contribution to the educational commons, as judged by me, earns them a webinar. This is not just promotional fluff. So I’m pleased to share the latest with you, which you’ll see represents an uptick in our tempo.
First, the video archive of the webinar on the LXPathways learning architect and instructional technologist competency programs from iDesign is now live on YouTube. Or you can just watch it here, if you have the hour:
iDesign LX Pathways Standard of Proof webinar
Second, a reminder that we have a webinar tomorrow with Macmillan on their method for studying the effectiveness of their products early in the product development cycle. I serve as a (paid) member of the Macmillan Impact Research Advisory Council (IRAC), so I have had both a ring-side seat and some input on this process. The transparency they are attempting to provide regarding what they know about product impact, how they know it, and how confident they are in the results at any given moment is commendable. I’m looking forward to that conversation tomorrow, i.e., Wednesday, March 10th at 2 PM ET. Register for the Macmillan efficacy methodology webinar here.
Third, you may have seen an announcement go out earlier today about our webinar for next Wednesday, March 18th at 2 PM ET for our Driving Toward a Degree webinar with Tyton Partners. We still have a lot to learn about increasing graduation rates, particularly among first-generation, post-traditional, and underrepresented students. Tyton has made an important contribution with their finding of the value of a coordinated response that far too few colleges and universities even attempt to achieve for this sort of effort. Their methodological approach to studying the impact of different approaches is also really interesting. Register for the Driving Toward a Degree webinar here.
And finally, I have a hold-the-date pre-announcement for a follow-up webinar on the research AdmitHub has been doing. (You can see the video of our webinar with them on summer melt here.) This research has broader implications for helping students get to graduation. I’ll hold the details until we have the description nailed down with the participants—this is new research, so I always want to get the characterization right—but for now, save the date for a webinar on Thursday, April 9th at 2 PM.
I am just thrilled to post the archive of our inaugural Standard of Proof webinar:
Summer Melt Standard of Proof webinar
https://youtu.be/7NoYUa_VHfs
It is a near-perfect encapsulation of everything that the Empirical Educator Project aspires to promote and foster: front-line educators, academic researchers, and commercial vendors working together to measurably increase student success and to do so in a way that is both persuasive in terms of the rigor of evidence and repeatable by other institutions.
As a reminder, Standard of Proof highlights academic collaborations by the Empirical Educator Project’s commercial sponsors with academic researchers and academic practitioners. All projects must either contribute new knowledge to the commons or promote the diffusion of established evidence-backed practices.
In this case, we have a story of a small EdTech start-up—AdmitHub—that has a total of nine randomized controlled trials either completed or in process with academic partner institutions.
Nine.
How many of your EdTech vendors have nine randomized controlled trials either done or in the works? How many have one? How many have you even asked this question? (If you want better answers on the first two questions, then try to improve your answer to the last one.)
They were able to do this, in part, because institutions like Georgia State University had real leaders, like GSU’s Tim Renick, who refused to accept the proposition that disadvantaged students were already beyond help by the time they got to college. Instead, the GSU folks (and their colleagues at other academic institutions conducting research with AdmitHub) set out to study their own practices and identify areas that they were failing students in ways that could be fixed. Using data, with the help of their vendor and an academic researcher—in this case, University of Pittsburgh Professor and summer melt expert Lindsay Page—GSU demonstrated that we can make education better and more equitable. It is the perfect counter-example to the sad situation I described in my post on what I didn’t see at EDUCAUSE.
This is the future that we all should be reaching for, folks. And it is reachable, as all three of my guests made clear. The video is well worth an hour of your time. Standard of Proof webinar production values will improve as we learn our craft, but the conversation quality just doesn’t get much better than this.
We’re actively working on the spring webinar line-up and will have some juicy conversations scheduled by the time you return from your holiday break.
