e-Literate

Present is Prologue

Tag: Online Education Initiative

  • The Christensen Institute’s Calbright Position is Dangerously Dogmatic

    If your main intellectual response to the current health crisis is to double down on your previously held beliefs, then you are probably not thinking deeply enough. For example, suppose that we develop both treatments and a vaccine for COVID-19, but the immunity conveyed by that vaccine only lasts for a season (much like other flu vaccines). Suppose that flu strains become permanently more virulent and more deadly. Some of the changes in the healthcare system that we are seeing now—in point-of-care, supply chain, and research efforts—become permanent. The risk of getting together physically in groups is permanently altered. Not to the degree that it is now, but significantly higher than it was. Think about how that permanent change would impact all the many businesses that are being impacted now. The growth sectors of our economy would change. The geopolitical landscape would change. Everything would change. Much of what we thought we knew about the future of work would have to be thrown out and rethought. And much of what we thought we knew about the future of education—what jobs we need to prepare students for, how we should be teaching, what sustainability looks like for an educational institution, and so on—would also have to be thrown out and rethought.

    It is hard to think such thoughts. I find my mind desperately casting about for reasons to believe that this scenario is not plausible. That life will someday, somehow, return to normal. That the world has not changed.

    But it has. And we must all learn how to think the unthinkable. We have to find ways to let go of the comforting assumptions that enable us to glide through the day on autopilot, somehow not swerving off the road while we fiddle with the car radio or talk on the phone or think about what will happen when we arrive at where we are going. We have to look at the road with fresh eyes. See what is in front of us. Prepare our minds for unexpected obstacles. The adaptive cruise control and lane-assist and other comforting tools that we have built to protect us from the need to be in the moment will not serve us well on this road we have found ourselves on. In fact, they provide a dangerous illusion of comfort. We must seize the wheel and learn to be navigators again.

    Right now, as we respond to an unprecedented crisis, a lot of the safety mechanisms in the system are being thrown out even as we face massive uncertainty. Multi-trillion-dollar legislation is being passed through Congress incredibly quickly. The medical system is finding various shortcuts to get around the system of checks that are designed to make sure treatments that tinker with your immune system are safe. These safeguards were created for good reasons, one of which is to mitigate the impact of sloppy or reflexive thinking on consequential actions. If we are going to remove them, then we must think harder.

    Even under normal times, I would have been disturbed by the Christensen Institute’s argument that “California should double down on Calbright College—by leaving it alone.” The post author Richard Price asserts that California’s new, experimental online community college, which received a $100 million initial allocation and a $20 million annual budget from the California State Legislature, is already fabulously successful. And it will be even more successful if only it is freed from the shackles that the other 114 California community colleges manage to live by and State Senators would back off of their call for Calbright to be audited for its finances, progress, and its compliance with the conditions under which it was authorized. He makes these assertions without evidence or even much in the way of argument to back them up. While reading the piece, I can’t escape the feeling that I am in the back seat of a car with a habitually distracted driver who is reflexively trusting his GPS to guide him through territory that has not been mapped.

    I have been patient with the more careful proponents of disruptive innovation in education, despite my skepticism, because it is not their fault that the term has been appropriated by testosterone-addled Silicon Valley teenagers, and because they have made cogent (if unconvincing) arguments. But the opening line of Price’s Calbright post, while sounding completely anodyne, alarms me:

    As the COVID-19 outbreak increasingly prompts universities to move instruction online, California lawmakers may want to reconsider their deep skepticism of the state’s first fully online community college—the young institution is proving itself more vital than ever.

    California should double down on Calbright College—by leaving it alone

    There are just so many things wrong with this sentence. For starters, the author links to an article about lawmakers’ “deep skepticism” without mentioning anywhere in his own piece that the article describes the reasons why lawmakers are skeptical. He certainly doesn’t attempt to refute those reasons. To the contrary, he asserts that the “young institution is proving itself more vital than ever” without ever really explaining what that means. Second, the sentence conflates skepticism about Calbright with skepticism about online education. Reading it, one would never guess that the California Legislature has poured something like a quarter of a billion dollars into online programs for the California Community College System alone, plus hundreds of millions more into the Cal State and UC systems. Lawmakers are not demonstrating skepticism toward online education. Rather, they are demonstrating skepticism of Calbright’s implementation as the best application of the substantial investment they are making in online education.

    Worst of all, the introduction sets the tone for the article, which is to accuse others of dogmatism while itself making its case on pure dogma. In doing so, it links to another post on the Christensen Institute’s site by another author about another system. This article, unfortunately, also gave me the feeling of being an uncomfortable passenger sitting beside a driver who is barreling down the road toward a destination that he is determined to reach, heedless of what may lie between here and there.

    The end result is a specious chain of arguments made from a position that indirectly has Harvard’s imprimatur—because Christensen was a noted Harvard professor—arguing that the state of California should commit to throwing tens of millions of dollars at an unproven program. And that it should do so while declining to hold that program accountable or even expecting it to follow the rules that all the other community colleges manage to live under. The main justification for this position is a theory that was designed to explain what happened to steam shovels and old disk drive technologies and is now being applied to a dramatically different domain without attention to context or evidence.

    If you are going to argue for removing safeties at a time of historic emergency, then you must think harder.

    It is time for the more serious proponents of disruptive innovation in education to take a good look in the mirror and ask themselves if they have been thinking hard enough. As their predictions about mass closures of universities and other dramatic changes have failed to materialize repeatedly over many years, their assertions about the future have become increasingly strident and sloppily constructed. It is time for them to seriously consider the possibility that they are just plain wrong, and that their attachment to cherished ideas is doing more harm than good.

    My own prior assumptions

    If I’m going to be fair, I should lay out my own prior thinking about both disruptive innovation and Calbright so that you can decide whether I am the one who is stuck in old thinking.

    I am a fan of Clay Christensen’s work. I have three of his books on my shelves, all of which I have read at least once, and two of which I find myself occasionally dipping back into for reference or inspiration. I find his basic thesis to be clever, well-argued, reasonably grounded in evidence, and provocative. That said, his theory is interesting because of the inference chain he constructs rather than because any of his individual insights are particularly novel in and of themselves. For example, one central concept in disruptive innovation starts by “competing against non-consumption,” which is a fancy way of saying you should go to where the competition isn’t by figuring out who isn’t being served well. While this assertion is worth noting, it is the opposite of revelatory. It borders on the banal. Another stepping stone in his argument is that successful organizations will tend to resist ideas that are different than their historically successful ways of doing things. We nod at this without pausing. Christensen’s argument pulls us along not because the steps along the way stop us in our tracks but, to the contrary, we glide down an easy path and suddenly end up somewhere unexpected. As Price puts it in his article, “In the Innovator’s Dilemma, Clayton Christensen made an unsettling observation—CEOs of established companies that fell to disruptive entrants often did everything right, and for that very reason failed to save their companies.”

    Disruptive innovation theory boils down to the idea that companies that are fat and happy tend to result in products that are fat and customers that are less happy. At some point, Microsoft stopped building features into Office that were useful to everybody and started focusing on features that were useful to their most lucrative niche customers. Office customers who were not in those niches became less happy as the software became more expensive and harder to use. But the company was focused on maximizing revenues from its best customers.

    So they didn’t react much when Google came out with Docs, Sheets, etc. This was a perfectly reasonable business decision for them to make, since Google was not an immediate threat to their customer base and some very profitable corporate customers wanted specialized features. Google’s products were obviously inferior for mainstream word processing tasks. I tried and abandoned Google Docs a few times in the early years because it just didn’t meet my needs. But it got better over time, first satisfying needs for which Word wasn’t well suited, then nibbling at the simplest applications of Word, and eventually becoming sophisticated enough to challenge the dominant word processor for a substantial percentage of common tasks. Today there are a billion users of Google Drive. I am one of them. I have not abandoned Microsoft Office, but I have thought about it at times.

    Disruptive innovation is both a particular kind of disruption and a particular kind of innovation. It is important to understand this. When we learn to trust a theory, we begin to take it for granted like we take our cruise control for granted. We stop thinking about it. But we need to understand how our tools work so that we can know when we are safe to trust them and when we are not. As I put it in 2014,

    These days, “disruptive” and “innovation” seem to always come together in the same sentence. It’s a bit like “big galoot.” Theoretically, “big” is a modifier for “galoot.” But you never hear people talking about small galoots, average sized galoots, or galoots of indeterminate size. In modern common usage, galoots are always big. “Big galoot” has pretty much become an open form compound word, like “post office” or “real estate.” But “disruptive innovation” is not a compound word. Disruptive innovation is a particular kind of innovation, and a fairly narrow kind at that. Specifically, disruptive innovation is a phenomenon in which a new market entrant can overtake a leader in an established market by offering cheaper and simpler solutions. It’s important to remember that some of Clayton Christensen’s seminal examples of disruptive innovations were steam shovels and disk drives. This is not the same kind of innovation that produced the iPhone. It’s essentially about identifying the slow fat rich kid and taking his lunch money. To be fair, it’s not that inherently mean-spirited, because presumably one takes the rich kid’s lunch money (or market share) by providing solutions that consumers prefer. But the point is that disruptive innovation is generally not about solving new problems with brilliant out-of-the-box ideas. It’s primarily about solving old problems better because the old solutions have gotten overbuilt.

    Why VCs Usually Get Ed Tech Wrong

    The reason that I have been skeptical about disruptive innovation’s applicability to education has to do with differences in both demand and supply from the situations that Christensen analyzed in his books. On the demand side, I argued in 2013 that education doesn’t fit the disruptive innovation framework because it is not a product:

    Silicon Valley can’t disrupt education because, for the most part, education is not a product category. “Education” is the term we apply to a loosely defined and poorly differentiated set of public and private goods (where “goods” is meant in the broadest sense, and not just something you can put into your Amazon shopping cart). Consider the fact that John Adams included the right to an education in the constitution for the Commonwealth of Massachusetts. The shallow lesson to be learned from this is that education is something so integral to the idea of democracy that it never will and never should be treated exclusively as a product to be sold on the private markets. The deeper lesson is that the idea of education—its value, even its very definition—is inextricably tangled up in deeper cultural notions and values that will be impossible to tease out with A/B testing and other engineering tools. This is why education systems in different countries are so different from each other. “Oh yes,” you may reply, “Of course I’m aware that education in India and China are very different from how it is here.” But I’m not talking about India and China. I’m talking about Germany. I’m talking about Italy. I’m talking about the UK. All these countries have educational systems that are very substantially different from the U.S., and different from each other as well. These are often not differences that a product team can get around through “localization.” They are fundamental differences that require substantially different solutions. There is no “education.” There are only educations.

    Don’t Cry for Me, Argentina

    Students’ reasons for the college choices they make are much more complex and varied than their reasons for choosing a word processing program. So I’ve been skeptical about whether the nebulous cloud of ideas and ideals that we call “education” is a thing that can be disrupted in the Christensonian sense.

    There are also mismatches to the theory on the supply side. Christensen’s theory of organizational behavior fits best with organizations that conform to economist Ronald Coase’s theory of the firm. Command-and-control organizations that are optimized for profitability and growth will be particularly vulnerable to the pathologies that he identifies. Most universities don’t function that way, and no university systems do. While universities do have to worry about optimizing for sustainability, that goal is complicated by their strong mission focus and their shared governance structures.

    And university systems are even further from Christensen’s model because they are governed by political systems, which operate under entirely different decision-making processes than companies do. For example, with 114 physical community colleges grouped into 72 districts across California’s 58 counties, every state legislator has a local interest in protecting one or more of those colleges. There were mission-related reasons for this design decision, having to do with making education accessible in an era before modern online learning. Regardless of the degree to which those reasons may or may not still apply, decisions for the California Community College System are still driven by political representation of those county interests in the state’s legislature. Christensen wrote an entire book—The Innovator’s Solution—on how entrenched corporations can stay innovative. As far as I can recall, none of the example organizations in that book looked remotely like the California State Legislature. So at the very least, somebody would need to make the affirmative case that Christensen’s prescriptions for corporations can be applied in this very different context.

    And remember, it’s the chain of inferences that makes disruption theory interesting. If any of the links break, then we have to question whether the theory holds. Cherrypicking evidence for one or a couple of the individual common-sense assertions in the inference chain puts us on a fast road to Stupidville, however our GPS display may label the destination.

    I have historically been skeptical of Calbright for reasons that have less to do with disruptive innovation and more to do with math. COVID-19 has been giving us all a hard lesson on the mathematics of growth. If every person carrying the disease spreads to between two and three other people, then the number of people with the disease will double every few days. On the other hand, if the average infected person spreads the disease to at most one other person, then it will eventually die off.

    The challenge of Calbright has been that the math never worked for the kind of growth that the legislature expected it to achieve. If the main goal was to create an incubator that would eventually teach us better ways to serve underserved students, that would be one thing. But the goal was to actually serve millions of underserved students in a relatively short span of years. Calbright’s growth rate would have to be unprecedentedly high in order to achieve that aim. Phil Hill broke it down in a post in 2017:

    The maximum growth rate of [some] cherry-picked successful schools ranges from ~1,200 / year for Excelsior to ~7,700 / year for SNHU (note that Rio Salado at ~1,400 / year is the only public institution). Add to this the fact that all of these schools have been around for decades. No accreditation issues, no time-consuming establishment of core leadership team, etc.

    There is a big difference in dealing with institutional issues and statewide issues, particularly in California. One in five US community college students in the US do so in California, and the statewide issues tend to come in large numbers. Statewide issues tend to come in hundreds of thousands while institutional issues tend to come in tens of thousands.

    What this points to is that for a new fully-online institution to get to some meaningful level of enrollment (let’s say 20,000) in the same ballpark as these comparison schools, I estimate it would take a full decade at the least. This is the reason, by the way, that Mitch Daniels and Purdue University made the Kaplan University deal even though Kaplan’s enrollments are dropping. Daniels did not want to wait a decade to get to meaningful enrollment numbers for an online college serving working adults – if everything works out, within a year Purdue will have a fully-online institution serving 30,000+ working adults. That is a big if, by the way.

    Enrollment Implications Regarding Directive for Online Community College in California

    I have no problem with the idea of Calbright as an experiment in supporting underserved students. But the math doesn’t support the idea that it is going to have an impact at scale any time in the next decade, even under the most optimistic of scaling scenarios. So my concern with Calbright is that both California politicians and proponents of disruption theory have seized on it as a means for solving different problems than the ones it seems suited to solve.

    I have one more predilection that you should know about, which is that I believe theories should be disprovable if they are to be credible. You can’t say that disruptive innovation is going to result in 50% of schools going bankrupt—no, wait, maybe it’s 25%—and that it’s MOOCs that are the disruption—or maybe just online learning in general, or possibly CBE, or maybe adaptive learning—and continue to be taken seriously. There has to be a way to arrive at the statement, “data X would disprove my hypothesis.” It can’t be the case that any old data can support your hypothesis while no data can disprove it. That’s not a theory. It’s dogma. My patience with proponents of disruptive innovation in education has worn thin over time because I haven’t seen them demonstrate the willingness to ask themselves the question, “But what if we’re wrong?”

    Now that I have laid my own comforting assumptions on the table, it’s time to examine the assumptions that are evident in the arguments from the Christensen Institute.

    Is Calbright a disruptive innovator?

    Let’s return to the hyperlink in the very first sentence of Price’s post, which is attached to the words “their deep skepticism.” What is the skepticism described in the Education Dive article behind that link? First, the article emphasizes the question of whether Calbright would be best run in the existing structures rather than as an independent campus. That’s one surface resemblance to disruption theory. Again, you don’t have to believe in disruption theory to believe that sometimes a little autonomy from a large, entrenched bureaucracy might lead to fresh thinking. But at least it’s consistent.

    On the other hand, Price neglects to mention other salient facts. Like the fact that the legislative analyst’s report in 2018 said there is no evidence that Calbright will solve the problems that prevent Californians from enrolling in college (or, in Christensonian parlance, that it can compete against non-consumption). Here’s the relevant paragraph from that report:

    Unclear If Providing Online Offerings Will Solve Key Barriers for Target Student Group. One of the proposal’s goals is to increase educational attainment for adults who currently have no postsecondary credentials. Although this is a laudable goal, the administration has not provided any evidence that an online community college will address the key barriers for this potential student group. Although an online program can increase convenience, working adults may not be pursuing additional education for a number of reasons. The administration also has not provided evidence that those working adults who are interested in more education cannot access it through existing online or in‑person community college programs.

    California LAO report

    Is there reason to believe that the innovations of Calbright’s design will cause large numbers of students who were previously not enrolling in existing online offerings to enroll in Calbright’s? I don’t know. The California LAO’s office didn’t see such evidence, and Price fails to even mention it as a problem.

    The Education Dive article also mentions that Calbright has duplicated existing academic offerings of other community colleges in violation of state rules. Here Price does have a rejoinder, though he does not dispute that Calbright has violated state rules:

    [C]oncerns that Calbright’s courses may duplicate existing OEI offerings fail to acknowledge that competency-based, self-paced courses differ fundamentally from seat-time based courses covering the same material, and better serve time-constrained adult learners.

    OK, that’s a claim. Where is the supporting evidence? Such programs exist in the world. Western Governor’s University, SNHU’s College for America, and others implement the instructional approach that Calbright has chosen. I’m not arguing that evidence doesn’t exist. But I do think that if somebody is advocating for a $100 million initial allocation plus a $20 million allocation in support of a new and unproven institution while simultaneously advocating that rules should be relaxed and oversight waved off, then one should present any evidence so that we can examine it together. The action Price is calling for requires less reliance on cruise control and more on the blind-spot camera.

    Rather than focusing on making an evidence-based case for Calbright, Price chooses to make an argument against California Online Education Initiative (OEI):

    Consider CCCS’s Online Education Initiative (OEI), a course exchange program, which Calbright critics point to as obviating the need for a separate, online college. In theory, students can enroll in online courses offered by other state community colleges. In reality, CCCS’s enrollment-based funding model discourages school participation, since only the college that offers the course earns enrollment revenue. This has limited course offerings and engagement overall.

    Here is an example of how Calbright, operating under a different business model, could be an asset to CCCS as an autonomous institution. Free from the constraints of the funding model that has proven incongruent with expanding online course options, Calbright could work towards revenue and course enrollment models that best serve large numbers of job-seeking adult learners.

    This is factually wrong on several levels. First, the course exchange is just one component of OEI. Luckily, MindWires—the consulting practice that I used to be a partner in and that Phil Hill still runs—produced this convenient explainer on behalf of OEI (which is less than four minutes and is easily findable on YouTube):

    Intended Consequences

    (Video link: https://youtu.be/1DdlaIZYiDI)

    As Phil recently wrote, Calbright is currently serving about 450 students. In contrast, in the 2020-2021 academic year, OEI projects 99,000 enrollments in new academic pathways. (And word is that the OEI number is likely to be revised upward significantly based on more recent data.) Apples-to-apples, this translates into probably a couple of tens of thousands of students.

    The course exchange part of OEI is one component. It may or may not succeed at scale. But the broader OEI program is indisputably reaching many more students than Calbright. Worse than getting the incentive mechanism wrong for the course exchange, Price apparently failed to conduct even cursory research regarding the nature and impact of the program he was critiquing. Even if the course exchange portion of OEI fails utterly, the overall program is still tracking to increase enrollments by two orders of magnitude more than Calbright will in the next year.

    That doesn’t even count the ways in which Calbright’s progress toward (unspecified) milestones was accomplished, in part, by using resources built with OEI funding. Here is a screenshot of Calbright’s essentials course:

    The link at the bottom of that screen leads to this page:

    Of the Calbright students enrolled in October through December of last year, 449 enrolled in essentials courses, while 20 enrolled in a program pathway. In other words, the vast majority of students taking Calbright courses last term were taking courses that were using OEI content resources.

    Also, while Calbright’s mission was specifically to reach underserved students that aren’t being reached via the 114 existing community colleges, the results so far are not promising on that score. For example, despite specifically targeting Latinx students, they represent 16% of Calbright’s enrollments—in contrast to 45% Latinx enrollment system-wide. Calbright’s students are, in fact, disproportionately white:

    Calbright legislative review backgrounder

    In addition to these reasons for questioning Calbright’s progress toward and ability to accomplish the mission set for it by the State Legislature, there’s also the fact that Heather Hiles, former Calbright CEO, resigned abruptly without public explanation less than a year after being hired.

    So given these facts, is it reasonable for California to ask whether Calbright makes sense and is working as intended at this time? I think it is. While I don’t prejudge the outcome of that introspection, the legislators have a responsibility to at least ask the hard questions.

    But, worries Price, whether or not OEI may work in practice, does it work in theory? He seems to take a kind of libertarian position that any program that is regulated will inherently be inferior to one that is not. He is aghast at the LAO’s concern that Calbright will eventually have to comply with the system’s collective bargaining agreements and spend 50% of its budgets on instructional salaries.

    In other words, Calbright being embedded in CCCS subjects it to strictures that will eventually hamstring the school in its efforts to innovate for adult learners. The LAO report ironically makes the argument that even as an independent school, Calbright isn’t autonomous enough.

    Does it? How much does it cost to run a Calbright-style CBE program with quality? How much of that money goes into instructional costs? How does that compare to the requirements that it is currently required to comply with and that OEI online programs do comply with? Again, the data exist, but Price either doesn’t believe he needs to justify his position or doesn’t have the data in question.

    I don’t have it either. What I do have is a quote from the President of Southern New Hampshire University (SNHU), which the Christensen Institute valorizes without understanding—more on that in a bit—about the amount of human time and attention that is required to deliver quality CBE education:

    I think there is a general lack of awareness of how rich now the underlying data analytics are. We monitor our students 24/7. We know when someone hasn’t logged on. We know when someone has struggled with a project. We know when performance has dropped off. We actually have closer to a 360-degree view of our students than most traditional institutions do. Then, when those students are engaged in the work, they have ready access to qualified faculty if they’re really stuck. We’re never going to let somebody get stuck on a math concept, for example, and just say, well, just figure it out. We’re going to get you help.

    Why Competency-Based Education Stalled (but Isn’t Finished)

    SNHU is putting significant time and money into instructor support. Is it 50%? I don’t know, and Price apparently doesn’t either. Is there a case to be made for granting Calbright some flexibility based on evidence of results in successful programs? Possibly, but Price doesn’t make it. Instead, he simply declares,

    Without an autonomous Calbright, CA will struggle to properly serve its adult learner population. Innovators like WGU and SNHU will take good care of these learners, but CA will have ceded, rather than seized, the future of learning.

    With little in the way of argument and nothing to speak of in the way of evidence, he concludes that (a) only Calbright can “properly” serve California’s adult learner population, (b) Calbright can only do so if it is “autonomous,” (c) Calbright represents “the future of learning,” and (d) if the California Legislature doesn’t cede oversight Calbright while continuing to fund it, then out-of-state competitors may disrupt or otherwise somehow surpass the California Community College System. It’s not at all clear how Price got us here, or even what he means by “properly” serving California’s adult learner population, or making Calbright “autonomous,” “ceding” or “seizing” the “future of learning,” or what role he believes that “innovators” such as WGU and SNHU will play in California’s future should the State Legislature fail to follow his prescription. He does link to (without really explaining the relevance of) another Christensen Institute post by another author about how the PASSHE system partnered with SNHU. I wish I could say that I found that article more persuasive.

    Alas.

    Is SNHU disrupting PASSHE?

    Michael Horn’s post from February of this year bears the breathless title, “Why disruptive innovation is stealing Pennsylvania’s students.” The trigger for the article was an articulation agreement signed between Pennsylvania’s State System for Higher Education (PASSHE) and SNHU which enables PASSHE students to transfer up to 90 credits to SNHU and complete their bachelor’s degree at SNHU at a 10% tuition discount.

    Horn emphasizes his belief that state regulations and slow-moving bureaucracy have hobbled PASSHE’s ability to innovate in online education, leading it to the necessity of outsourcing. He does so mainly by analogy to SNHU. There are several problems with this approach.

    First, as I pointed out earlier, the idea of increasing autonomy to spur innovation was not newly introduced by disruption theory. Bell Labs embodied this strategy from its formation in 1925. Clayton Christensen wasn’t even born until 1952. Second, as I also pointed out earlier, the success of this strategy does not validate all of disruption theory, never mind its applicability to education. As Horn acknowledges in his post, PASSHE is not failing because its leaders did everything right in pursuing its most lucrative customers. (Not that we would want public education systems to do that anyway, which is yet another problem with disruption theory’s applicability.) To the contrary, PASSHE arranged for the articulation agreement because many of its (tuition-paying) students are not finishing their degrees, according to PASSHE Chancellor Dan Greenstein. Presenting those students with another reasonably priced online option was intended to increase student graduation rates, even at the possible cost of the system’s most lucrative customers. We have to follow Horn down a chain of several inferential steps—each of which is given with either thin factual support or none at all—to arrive at the implication that “disruptive innovation is stealing Pennsylvania’s students.”

    Horn relies on a loose comparison to SNHU to do most of the work from him. To give him the benefit of the doubt, let’s stipulate several points:

    • SNHU’s growth has been undeniably impressive at a time when PASSHE has been experiencing a slow-motion crisis.
    • SNHU President Paul LeBlanc, like me, is a fan of Clay Christensen’s work. He has thought about it a lot and credits it with influencing his thinking.
    • Specifically, in his 2015 interview with yet another Christensen Institute writer, he speaks at length about the value of creating some measure of autonomy among academic units as a critical success factor for SNHU, and he directly links that notion to Christensen.

    Even so, it’s not that simple. LeBlanc hasn’t applied the separation principle dogmatically. SNHU operates under one shared governance structure. Autonomy is not the same as freedom from rules or oversight. Second, he has not hesitated to revoke autonomy when it wasn’t working. SNHU’s competency-based education unit, the College for America, used to be autonomous but has since been pulled back into the larger online learning unit.

    Overall, Horn’s article is a confusing muddle. He readily acknowledges that the causes of PASSHE’s troubles look nothing like the causes of disruptive innovation described in The Innovator’s Dilemma. He also admits that better funding of the chronically underfunded system would help. He then swerves off into an explanation of disruption theory, connected only by the thread that SNHU employed the same basic strategy that Bell Labs used successfully nearly 30 years before Clayton Christensen was born. He ends with a call for state systems to create small incubators where new ideas can be piloted. I agree with that idea, but I don’t think it bears any relationship to the completely unjustified clickbait title of his post. I don’t think one has to believe in disruption theory to come to the conclusion that university systems have become sclerotic and need to find ways to get out of their own way. This could have been a perfectly good article if it weren’t trying so desperately hard to validate disruption theory. Horn’s rather modest call to action is not in any way proportionate to the dire pronouncement of the headline; nor does it justify Price’s call for Calbright to be exempted from oversight or collective bargaining rules before it has even proven that it can serve 1,000 students.

    This is all hauntingly familiar. I mentioned that I own three of Christensen’s books, two of which I occasionally dip into to this day. The other one is Disrupting Class, which is the book that Christensen co-wrote with Horn about the application of disruption theory to education. In stark contrast to The Innovator’s Dilemma and The Innovator’s Solution, it was a disappointing mess. Parts of it attempt to stretch disruption theory beyond recognition in order to fit educational examples. Other parts contain thoughtful, reasonable observations that bear no credible connection to disruption theory. It doesn’t hang together as a cohesive work. I found the book unpersuasive when it was published 12 years ago and do not believe it has aged well. And in subsequent publications by disruption theory proponents in education, the flaws that were apparent in that original work have been papered over rather than addressed. Christensen has passed away, but the people carrying his torch in this domain have not done his legacy a service, however diligently and earnestly they may have tried.

    Twelve years of trying is enough. It is time to think harder.

    Grieving our errances

    Alexander Pope wrote that to err is human, but he might have added that to err repeatedly due to habits of mind is particularly human. The disruption theory proponents love their idea. As a man who has written 7,000-word blog posts, I cannot credibly claim immunity from that particular siren song.

    It is not a propensity to be ridiculed. Our capacity to fall in love with a thing as aetherial as a concept has been a key to our survival as a species, and to creating lives for ourselves that are about more than just surviving.

    Sadly, we have not evolved to feel equal love for the process by which we produce, examine, and sometimes destroy ideas. We became toolmakers because tools enable us to accomplish more work with less effort. Ideas are tools that enable us to accomplish more work with less effortful thought. To err is human; to think is divine. These days, I am less inclined toward mocking people for being passionately wrong than when I was young enough to believe that only other people ever commit that sin. But I will attack their arguments without hesitation or mercy when I think they are harmful.

    It has become clear to me that all talk of disruptive innovation in the context of education is harmful. Not just when applied thoughtlessly by people who don’t understand the theory, but even when applied diligently by people who do. It is harmful to advocate for a failed idea as if the fate of education depends on believing it is true.

    Disruption theory in education is dead. Be sad if you need to, but move on. Seize the wheel and learn to be a navigator again.

    As for Calbright, I remain agnostic. I believe it is a good idea in principle for California to create a safe space to experiment with alternative means of reaching the millions of Californians who need education but aren’t getting it. I worry that this characterization is not fully aligned with the stated goals of the Calbright program. I do not know whether the experimental college is being run well, or whether now is the best time to prioritize it over other educational exigencies of the moment. I do believe it is not only fair but important to ask these questions and insist on arriving at fact-based, well-reasoned answers.

  • Insight into Community College Students and Challenges of Online Education

    Insight into Community College Students and Challenges of Online Education

    Inside Higher Ed reported today on a new survey report on community colleges and challenges that students face.

    Most community colleges are aware of the challenges students face if they are working, raising children or struggling to afford textbooks. But a newly released survey digs into the nuances of those challenges so colleges can pinpoint ways to lift barriers to college completion and prevent students from dropping out.

    Researchers at North Carolina State University designed and encouraged students to participate in the Revealing Institutional Strengths and Challenges survey. The survey found that working and paying for expenses were the top two challenges community college students said impeded their academic success. The researchers surveyed nearly 6,000 two-year college students from 10 community colleges in California, Michigan, Nebraska, North Carolina, South Dakota, Texas, Virginia, Wisconsin and Wyoming in fall 2017 and 2018.

    Of the top ten challenges listed, the category of online classes was tied for fourth along with parking in an ironic twist as lack of physical facilities is one of the drivers for the growth in online education. Interestingly for parking, it is not the costs. 86% of those listing parking stated it was “difficulty finding parking on or near campus” and only 10% listed parking as too expensive.

    Top Ten challenges for community college students

    The value of this survey, as described at IHE in interviews with the report authors, is in the nuance that can guide institutional planning.

    [Report authors] Umbach and Steve Porter, also a professor of higher education at the university, said they noticed a dearth of surveys that asked students about the barriers they face to completing college and wanted to provide a tool that colleges could use to eliminate those barriers and boost graduation rates.

    On the topic of online education as a barrier, one view of the results could be that only one out of five students have a problem with these classes, which is not problematic as we have long known that online is not for everyone. On the other hand, the nuance provided should give institutions some insight into how they can improve their services to students.

    Reasons for online classes as a challenge

    The big issue that I’ve seen in the field is not whether a school should offer online classes – in so many cases this is the only way for students to have access to degrees – but how well-designed the courses are and how much support is provided outside of the course. Throwing courses online with no real interaction or adequate support is a recipe for disaster here, as I described in one case last year. But the school in that example is not alone in this regard. In the many cases where community colleges make these mistakes, students should have difficulty learning and the 21% number should be problematic.

    But on the opposite side, when colleges focus on improving course design and extend meaningful support services, student outcomes improve dramatically. Consider the California Community College system and their improved outcomes, where their most recent distance education report shows system-wide closing of the achievement gap between face-to-face and online students. Online education can work for community college students and is an important part of student access, but there are no silver bullets.

    I was quoted in the article about these challenges.

    Hill said the California Community College System’s Online Education Initiative, which he worked on as a consultant, is a good example of a well-designed online learning system. It helped close the gap between the rate of students successfully completing traditional courses and online classes from 17 percent in 2006 to 4 percent in 2016.

    To be clear, the California Community College System in general has been improving their provision and support of online courses for years, and OEI is not the only driver of this change.

    CCCS improvements in gap of online ed

    I don’t think the California Community College System is the only example of improvements in online education support, but I do think their focus on improving course design as well as improving advising and support structures is worth considering.

    The problem of ‘difficulty learning material on my own’  and ‘difficulty keeping up’ issues can only partially be addressed – online education is not for everyone – but more engaging and well-designed online courses can help, or at least reduce barriers. The mixing of synchronous elements of a course along with asynchronous can also play an important role.

    ‘Lack of interaction with faculty’, ‘lack of interaction with other students’, and ‘difficulty using course technology’, however, are issues that should be addressed by the institution as part of the course design and support services. It would be naive to think that these issues could be eliminated, but there is no excuse for schools to not have a coordinated effort to make improvements across all online courses.

    Online education can work, and community colleges can improve outcomes by addressing the challenges students face.

    While this post focuses on the online education angle, the whole report is worth reading. The insights into issues outside the classroom, particularly for students trying to balance work and family commitments with their education, should provide valuable input into institutional- and system-level planning.

  • Toward Operational Excellence at Student Success: Double-Loop Learning

    Before I move on to my next case study in academic institutions moving toward operational excellence at supporting student success, I want to revisit a section toward the end of my last post on the California Community Colleges Online Education Initiative (OEI). I was looking at the alignment that has to be achieved at various levels in the academic organization in order to encourage all the stakeholders to embrace this collective mission, with all the changes to their day-to-day work and even professional identities it would entail. Much of the piece is about the work that had been done so far to get alignment at various levels within the administration. But toward the end of the piece, I speculated a bit on potential opportunities for fostering faculty alignment through a course peer review process using a common rubric:

    [T]o me, one of the most interesting vectors for culture-building is the course exchange course quality rubric. Every course on the exchange has to be evaluated against a rubric of evidence-backed effective online teaching practices. As the pace at which exchange courses are developed increases, OEI will not be able to keep up with demand to evaluate these courses using central staff. So they are creating a peer reviewer mechanism in which faculty on the campuses are trained on the rubric and presumably compensated to review courses that are candidates for the exchange.

    This opportunity fascinates me. We know that faculty who go through an expert-supported course redesign process often experience intellectually deep and emotionally moving shifts in their teaching strategies. Is the same true when faculty are trained reviewers of their colleagues’ redesigned courses? What effect will simply exposing faculty to more and different course designs have? How will their role as reviewers and critiquers shape or enhance that effect? Can a continuously improved and updated rubric become a vector for sharing new research-supported processes across the system on an ongoing basis? Will the impact be broad and deep enough to foster new kinds of intra- and inter-campus faculty dialogs about the scholarship of teaching and learning (SoTL)? Will these cultural changes help to foster alignment around continuous operational improvement for enabling student success? This is the last mile problem of higher education. Operational excellence at student success cannot be achieved unless it is infused in the daily operations in individual classrooms. That requires affirmative faculty buy-in, support, training, and embedding in a culture that invites them into the larger conversation.

    Unpacking this a bit, what does it really mean to build a culture of operational excellence in supporting student success? What kind of change would be necessary at the individual level to achieve change at the organizational level?

    Organizational learning

    There is a useful concept in organizational psychology called “double-loop learning.” I’ll give a simple example of a non-academic organization first to make the concept clear. Suppose your company manufactures smartphones. You want supply to match demand almost exactly as possible. If you manufacture too many phones, then you will sink expense into building units that will sit on the shelves and fairly quickly become obsolete. But if you manufacture too few, then you won’t have phones to sell at the moments that people need to buy them, thus encouraging them to buy a different (more available) phone instead. In a single-loop model, you have one lever to pull, which is how many phones you produce at a given time. It’s a like a thermostat: If the room is too cold, then turn on the furnace. If the room is warm enough, then turn off the furnace. If there are not enough phones on the shelves, then turn up production. If there are too many phones on the shelves, then turn down the production line.

    The problem is that there’s a significant lag between when the order is given to produce more phones and when they arrive on the shelves. During that time period, demand can change. Maybe by the time the new phones the company produces during a period of high demand actually land on the shelves during the beginning of a recession, or right after a competitor releases their hot new model. The single-loop, thermostat-like model doesn’t work very well.

    Of course, the people who run the company are smart enough to know this, so they come up with all sorts of work-arounds. They build warehouses to hold excess phones near where they are built, since holding onto the phones that way is cheaper than shipping them halfway across the world and negotiating with the retail stores that are selling them and may want to ship excess inventory back. They build sophisticated forecasting models that account for factors such as the economy and competitor behavior, so the chances of them being badly wrong are reduced. These are all work-arounds to a fundamental problem regarding the costliness of being wrong in your demand forecasts. And this is exactly the way manufacturers of all kinds of complex items, including smartphones, used to operate in the old days.

    But then somebody somewhere questioned a fundamental premise that drove so much effort and activity: Does it have to take so long from the time the company order new products to be manufactured until those products reach the retail shelves? Maybe there’s some part that often holds up the whole product; if we could only use a different part, or make the part ourselves, then we could get rid of a lot of the delays. Maybe the places where those component parts come from are farther away from our factory than they need to be; if we could just get them to move closer, then we could cut down on the lags. Maybe we have extra steps in our manufacturing process, or use outdated equipment; if we could only make some updates, then we can shorten the lag. And maybe if we do all of these things, as well as more generally finding and making changes anyplace where the process bogs down, then maybe we don’t have to put products on shelves at all. Maybe we can get the delay between order and manufacture short enough that we could start manufacturing the device when the consumer orders it and get it assembled and shipped fast enough that the consumer would tolerate the delay.

    This is double-loop learning. Organizations not only use processes that allow them to make adjustments but also regularly examine the assumptions behind those processes that may be unnecessarily getting in the way of achieving organizational goals. We assume that we have to develop processes to mitigate bad product demand forecasts because we assume that those costs will be high because, in turn, we assume that manufacturing the product will take a long time once we decide to do it. But what if we’re wrong?

    Double-loop thinking is a reasonably simple concept to understand but very hard to execute well and consistently. In the smartphone manufacturer example, think about all the many kinds of assumptions in the way things had always been done that would have to be identified, questioned, and replaced with a better-designed alternative. Particularly in the early days, when there weren’t models to copy or lessons learned elsewhere, no one person who could see all the changes that would have to be made. There would be many people across the organization—in manufacturing, product design, contract negotiation, shipping, retail relations, and so on—who would each be able to spot an individual sub-optimization in her daily work experience. And then more people would have to be involved in designing a solution to each sub-optimization, including accounting for all the ripple effects across other aspects of the organization. It would be an all-hands-on-deck sort of affair. Everyone would be needed to find problems, identify potential solutions, check those solutions for side-effects, and then implement them well.

    Double-loop learning in academia

    Now think about a few of the many questions that are starting to be asked about the operating assumptions about the education-related processes of colleges and universities:

    • Why must students stay in a course for a set number of weeks, regardless of how quickly or slowly they are capable of learning the material?
    • Why are students only able to register for and start a course at most a couple of set times in the year?
    • Why some very common teaching modalities based on the default assumption that all students learn roughly the same way and encounter roughly the same rough spots?
    • Why do we define the minimum math literacy for a college degree as basic algebra rather than, say, statistics?
    • We do we believe that professorial training requires at least five years of deep disciplinary education and at most one course in pedagogical education?
    • Why do faculty gain job security through research excellence far more than through teaching excellence?
    • Why do we assume that students know and understand everything they need to do from the moment they receive their college acceptance to the moment they arrive on campus for the start of their first semester of class?
    • Why do we assume we can know when individual students are in trouble and need help from the academic institutions when no employee of that institution sees the student for more than a few hours in the week—at most—and there is no good mechanism for sharing concerns and observations among the people who have contact with that student?

    Think about the people who were in a position to spot each of these assumptions. Think about all the people required to design, troubleshoot, and implement alternatives that arise out of questioning the assumptions. If we want to reliably create student-ready colleges, then we need to be able to identify many unwarranted assumptions and design many alternative ways of doing things in ways that will deeply affect the ways in which academic institutions—and the people employed by them—work. To change everything, you need everyone. That specifically includes faculty.

    A rubric as a vector for change

    Way back in late 2013, I wrote about Pearson using a rubric to try to catalyze this sort of broad-based organizational shift in (critical) thinking: ((Pearson is a sponsor of e-Literate’s Empirical Educator Project.))

    So if you’re the CEO of major textbook publisher and you want to unite the entire 45,000-employee company around a plan to transform the way the company does business, what do you do? Surprisingly, Pearson’s CEO John Fallon’s answer was, “I’ll create a rubric.”

    I’m not going to analyze Pearson’s rubric in detail here…. I’ll say this much about it: It’s nothing special. It’s not bad, but it’s not genius either. There are plenty of flaws and limitations you could find if you worked at it and applied it broadly enough. There is no magic in it.

    But here’s the thing: There is neverany magic in a rubric. The magic, when there is any, happens from the norming conversations that the rubric engenders. It happens when one colleague says to another, “What do you mean by ‘quality of evidence’?” Or “I scored that course a 2 on effectiveness. Why did you think it was a 4?” To the degree that the Effectiveness Framework proves to have any magic for Pearson, it will be in the norming conversations that it engenders across the company. Like our hypothetical Berkeley president, Fallon is working with diverse groups within an institution that has a culture of independence and Balkanization. Some of this is for good reason; conversations about effectiveness in chemistry education should look very different from conversations about effectiveness in fine arts education. Some of the fractiousness is about lack of a common culture and language necessary to discuss what otherwise arecommon challenges. And some of it is just human territoriality and self-interest. The first two challenges might be addressed by having a deep and wide ongoing norming conversation about a rubric that is general enough to cover a wide range disciplines and products but focused enough to provoke important discussions. The goal is for that conversation to become the basis for a new culture. The third challenge might be addressed by reinforcing that culture through your HR and other business practices.

    Since I wrote that post, Pearson has developed a set of rubrics for evaluating whether a given product supports research-backed learning design principles. They have rolled those rubrics out to every product team and trained their product teams on how to use them. They have released them under a Creative Commons license and are. (For more on both the resource itself and Pearson’s interest in working with academics to make them more useful to academia, see the talk given by Pearson’s Global Head of Efficacy and Reach at last year’s Empirical Educator Project summit.) So Pearson continues to use what is essentially an academic strategy, not that different from the one being rolled out by California OEI, to build a double-loop culture around designing educational content and software functionality that are more effective at impacting student outcomes.

    The rubric development, training, and norming processes are necessary but not sufficient. As I suggest in that last sentence of the Pearson post quote, other organizational processes need to be put in place as well in order to get the desired effect. It would be easy to get faculty thinking that the new practices are baked into the rubric, and as long as everybody is aligned with them, you’re good. The organization goes through the double-loop, but only until the norming process is complete. This is, in fact, what happens in many colleges and universities that adopt course quality rubrics. The institution has to mindfully employ the rubric updating, retraining, and renorming processes as methods for collaborative innovation. The rubric needs to be designed at a high enough level that it invites discussion and thought rather than rote implementation. The processes around it need to be collaborative rather than broadcast-only. And many other processes—like compensation for time invested or rewards for innovation, to take a couple of obvious examples—need to be created or modified to support and dovetail with the rubric processes.

    There are lots of organizations that implement course quality rubrics. Enough that we should be able to start gathering stories and effective practices for using them to foster continuous organizational improvement. If anyone has a good example, please let me know.

  • Scaling Educational Access

    Scaling Educational Access

    There’s an interesting piece in EdSurge about the potential access to education challenge that CUNY will face if Governor Cuomo makes good on his proposal to make college tuition free:

    According to the CUNY 2016 Student Experience Survey, 21 percent of the system’s community college students were not able to take required courses, most citing “lack of seat availability” as the reason. In an interview, George Otte, CUNY’s director of academic technology, described the “bottleneck” created by course shortages. Students were not able to take required courses because demand for space exceeded capacity—a problem that could get worse if more free tuition-seekers pour in. “We have always had lots of enrollment, which is one of the reasons I think we were late to the online course offerings. Our mission was never to gain students from the outside,” he explains, “but we are just beginning to realize that this has an enormous impact on capacity.”

    “The real challenge is that eligibility is predicated on full-time enrollment,” Otte says, citing a caveat community colleges cannot ignore. Without taking 15 hours a semester, which equals full-time enrollment, students are not elible for Gov. Cuomo’s free tuition program—the Excelsior Scholarship. However, at least half of community college students work full-time. It is unlikely that such a population can opt-in to campus-based instruction that requires them to commute and show up for courses offered only at, say, 9 a.m. or 4 p.m.

    It turns out that we’ve done a lot of work in this area, particularly in the state of California, and have some lessons learned that are worth sharing.

    (more…)

  • The Intended Consequences of California’s Online Education Initiative

    The Intended Consequences of California’s Online Education Initiative

    Last month we presented two explainer videos on the growing usage of course exchanges, where multiple institutions pool resources in creating or extending online courses. ((Disclosure: Our e-Literate TV series of video case studies and explainer videos is funded by a grant from the Bill & Melinda Gates Foundation.)) If online courses or programs breaks down the barriers of campus walls and enables anytime, anywhere education, then why not explore how collaboration can open up access and improve quality. While we tend to not write e-Literate about our consulting work through MindWires, in this case we have heard a general interest from other systems to learn more about what the California Online Education Initiative (OEI) at the community college system is doing.

    The first two videos explored the concept of course exchanges in general and the required infrastructure needed to create them. In our third explainer video from this series we go back to the OEI to look at how their investment in academic infrastructure should provide benefits that go well beyond the courses and students participating in the course exchange. This extension of benefits, however, was not a surprise to those creating OEI; rather, these broader benefits represent the intended consequences of their approach to collaborative online education.

    (Video source: https://youtu.be/1DdlaIZYiDI)

    (more…)

  • Explainer Videos on Course Exchanges and the Shared Infrastructure Behind Them

    Two months ago O’Neal Spicer and I wrote an op-ed for the Community College Daily describing our observations on the California Online Education Initiative (OEI). We don’t often write about clients of ours, but in this case we felt it would be useful to share our thoughts outside of the consulting context, and the OEI staff agreed.

    While some argued that a parallel system of MOOCs, unbundled course and third-party operators should step-in, the state opted to re-invest in its existing faculty and infrastructure. At the heart of what became known as the Online Education Initiative (OEI) was a system for an online course exchange that could help to improve access to courses that were oversubscribed or not available at local campuses.

    California was not alone in making the move toward a system where a consortium of campuses pooled resources to offer online courses for students across multiple campuses. North Carolina has done so, as has Colorado and Mississippi and several others. Part of the attraction of course exchanges comes from the acknowledgement that with online education, the campus walls do not need to be an artificial barrier.

    Ryan Craig wrote a separate piece for Forbes on the upcoming release of the OEI course exchange as did Ashley Smith at Inside Higher Ed.

    We thought it would be useful to explore this concept of course exchanges more broadly. ((Disclosure: Our e-Literate TV series of video case studies and explainer videos is funded by a grant from the Bill & Melinda Gates Foundation.)) OEI is not the first course exchange – Colorado Community College System, University of North Carolina SystemMississippi Virtual Community College, Kentucky Virtual Campus, to name a few other initiatives ((Disclosure: Besides OEI, Colorado Community College System and Mississippi Community College System are past clients of either MindWires or its predecessors.)). What is the concept of course exchanges and what problem is being addressed? In a nutshell, you can think of this as removing the traditional boundaries of higher ed, as described in our first video.

    (Video source: https://youtu.be/HBCVAau8hK8)

    (more…)

  • From Cockroaches To Miniature Elephants: Webinars on Not Your Mother’s Old LMS

    We just finished the third of our three-part webinar series on“The Modern Learning Platform: Not Your Mother’s Old LMS”, hosted by our friends at NobleStream. In the first discussion, Michael and I laid out our vision for why we are analysts for the LMS market and what we hope to achieve, capped off by Michael’s comment “Depending on how you define it, I have a feeling that, for better or for worse, long after humanity is dead and gone, cockroaches will still be using the LMS.”  In the second discussion, we took the institutional view and had Pat James and Anna Stirling, from the California Community Colleges, talking about how they approached LMS selection and in particular how they focused on the academic and strategic needs first, treating the LMS as plumbing to enable the house to be built. In the third discussion, we took the vendor view and had legendary Ray Henderson take a break from his well-deserved steelhead fishing excursions and describe how vendors understand and manage the LMS vendor selection process. This conversation was capped off by my mention of “the miniature elephant in the room”, where strategically-beneficial LMS decisions are typically made in spite of the RFP process, not because of it.

    Michael and I thoroughly enjoyed these discussions and we hope you will, too.