I’m going to be facilitating an Empirical Educator Project-relevant panel at OLC today at 11:15 AM in Oceanic 1, followed by an EEP and EEP-curious meetup at Soomo booth (#226) at 12:15 PM in the Expo Center. The rest of this post is just a little extra information on each of the SoTL work of the panel participants’ home institutions, for those who attend the session.
CMU Eberly Center
At the intersection of faculty research, teaching, and service, the Eberly Center supports Teaching as Research. We help faculty answer compelling research questions regarding which teaching strategies are more effective at promoting learning, increasing engagement, and enhancing the learning environment. Our services provide the tools and expertise to help instructors develop research questions and study designs, identify valid and reliable data sources, analyze and interpret educational data, and present and publish research results. Read more about our research processes and findings in this site:
Support is provided by the UCF Faculty Center for Teaching and Learning and the Research Initiative for Teaching Effectiveness. RITE assists faculty, free of charge, with any SoTL activity within the research design to dissemination continuum.
CTU
CTU is a career-focused university encouraging the use of educational technology and SoTL research in the areas of professional scholarship and adaptive learning. Faculty (including adjunct faculty) can apply for funding through an internal website and faculty are encouraged to share their research and scholarship work with the university. Additionally, research collaboration with other institutions is supported and encouraged as demonstrated by the work with CTU and UCF.
I’m going to be facilitating an Empirical Educator Project-relevant panel at OLC today at 11:15 AM in Oceanic 1, followed by an EEP and EEP-curious meetup at Soomo booth (#226) at 12:15 PM in the Expo Center. The rest of this post is just a little extra information on each of the SoTL work of the panel participants’ home institutions, for those who attend the session.
CMU Eberly Center
At the intersection of faculty research, teaching, and service, the Eberly Center supports Teaching as Research. We help faculty answer compelling research questions regarding which teaching strategies are more effective at promoting learning, increasing engagement, and enhancing the learning environment. Our services provide the tools and expertise to help instructors develop research questions and study designs, identify valid and reliable data sources, analyze and interpret educational data, and present and publish research results. Read more about our research processes and findings in this site:
Support is provided by the UCF Faculty Center for Teaching and Learning and the Research Initiative for Teaching Effectiveness. RITE assists faculty, free of charge, with any SoTL activity within the research design to dissemination continuum.
CTU
CTU is a career-focused university encouraging the use of educational technology and SoTL research in the areas of professional scholarship and adaptive learning. Faculty (including adjunct faculty) can apply for funding through an internal website and faculty are encouraged to share their research and scholarship work with the university. Additionally, research collaboration with other institutions is supported and encouraged as demonstrated by the work with CTU and UCF.
For a couple of years now, we’ve been saying that higher education is at the beginning stages of a long transition from a philosophical commitment to student success toward an operational commitment to it. In other words, colleges and universities are beginning to grapple in earnest with how to rewire themselves so that their culture and processes are deliberately optimized and continuously tuned to support their students in getting the best education possible. This is a profound shift. It will require major changes to the ways in which academia works and the ways in which ed tech designs and markets its products. It will be very hard and take a long time. But the drivers of this change are in place.
Recently, I wrote about how our concept of Empirical Education has built into it a theory of change. The implication is that it has the backbone for a methodology of change. Our work as both analysts and consultants shown us that the increasingly aligned strategic priorities throughout the sector, when combined with the knowledge that is scattered across it, can be distilled down into a powerful yet flexible methodology for system change in education analogous to Design Thinking or one of the Agile software development methodologies. It can be a set of processes, built on a fairly small set of fundamental principles but supported by a lot of detailed craft knowledge and a rich ecosystem of supporting tools. It can be owned by no-one, although there would likely be some premiere practitioners of it. Colleges and universities could use it to redesign themselves to be more student-centric and, in the process, also more educator-centric. Product and service companies could design their offerings around it and compete based on their ability to help their academic customers better implement it.
This sense of possibility has been the animating impulse behind the Empirical Educator Project (EEP). We started with only a hazy idea of what we were building. Over the last twelve months of working with academics and ed tech product people, some aspects have become clearer. I have grown more confident in the potential of the idea even as I have grown more overwhelmed with clearer understanding of the size of the undertaking.
I am going to articulate my latest thinking about it in this post.
The time is now
There is a saying among consultants that potential clients won’t hire a consultant until and unless they both realize that they have a serious problem and come to accept that it is not a problem they can solve on their own. That holds equally true for a wide range of difficult changes that require help or cooperation, from coping with an addiction to building a functioning government to changing an institution. Higher education has been an incredibly stable system. As in, remarkably consistent over a period of about a thousand years. Historians of education tend to write about changes that take place over decades or half-centuries. There has been a looming question of whether such a slow-changing institution can adapt to such fast-changing times. Unsurprisingly, this debate has been raging for a few decades now, with relatively little sector-wide change to show for it. Is the system terminally rigid, or is it in a state of punctuated equilibrium that will shift in an appropriately dramatic amplitude once it reaches an inflection point?
I believe the latter is the case, and I believe that we are at that inflection point. Access-oriented institutions—particularly publicly funded ones—have already been under pressure for some time now to show better outcomes for students in terms of rough measures like graduation rates and time to graduation, as well as some more meaningful but difficult measures popping up on the margins such as employment and career success. On the other end of the spectrum, the elite institutions whose brands have popularly defined excellence in education for the last century or more are starting to realize that they need to adjust to changing student expectations if they are going to continue to be considered the gold standard for the next century or more. The MOOC craze was complex and problematic, but it woke the elites up to the potential for use of technology-enabled approaches to enhance their teaching practices rather than detract from it, even as their students show up on campus with increasingly high expectations for the kinds of access to knowledge, interactive experiences, and high-touch communication that technology can enable. And in the middle, private universities with decent regional reputations and tuitions that approach those of Ivy League schools are increasingly under pressure to justify their tuition with something of more permanent value to students than climbing walls and dining halls. More and more, the buzz is about innovative partnerships with employers, or about learning analytics, or about student success systems supporting better guidance counseling. In other words, we are seeing colleges and universities grope toward approaches that enable them to more reliably support student success. And they are looking for help to do it.
The focus of that previous paragraph is primarily on undergraduate education, but it also increasingly applies to graduate education. We sometimes see this problem manifest itself in financial terms, where it gets somewhat obscured by the current conversations around Online Program Management (OPM) companies. Universities often launch career-oriented graduate programs such as MBAs and MSWs because (a) they are looking for more revenue to make their institutions more sustainable, (b) online programs can scale without scaling costs like real estate and physical classrooms, and (c) they know there is a market of people who are inclined to sign up for online graduate programs that can fit with their work and family schedules while also giving them credentials that will help them advance in their career ambitions. But as the online MBA market gets saturated, universities increasingly have to find differentiators. And they can’t use climbing walls or dining halls. In the end, the only effective and durable differentiator for an online career-oriented graduate degree program is its effectiveness at helping the students achieve their goals. In this space, the immediate university driver is revenue and the immediate student goal is career advancement. So the sector tends to view this change narrowly. But if you zoom out a little, it becomes clear that the trend with graduate programs and OPMs is just one particularly clear example of where the academic institution’s financial sustainability issues are driving it toward a sharper operational focus on its mission.
Let’s turn now to the educational vendors, who are also at an inflection point across product categories. All of these companies—curricular materials providers, LMS vendors, SIS vendors, analytics vendors, and so on—they are all looking to move up the value chain and argue that their products can directly, meaningfully, and provably impact student outcomes. ((I am use phrases like “student outcomes” and “student success” interchangeably and broadly for the purposes of this post, even though I know that they can have different connotations.)) They have to, because most of the major ed tech product categories are either in danger of commodifying or in danger of failing (in the case of established product categories) to achieve meaningful market penetration (in the case of new ones).
The textbook companies hit the wall first. As students increasingly found ways to avoid buying new books (or any books), the textbook publishers raised their prices, which started a vicious cycle of reduced sell-through followed by price increases followed by further reduced sell-through followed by further price increases. This was ultimately unsustainable, particularly since the internet has made obtaining basic factual information and focused educational supplements—think YouTube—easily obtainable and free. Increasingly, publishers had to make the case that their content is somehow better than the commodity content. But better how? For a long time, the “better” publishers worked on was instructor convenience. But there’s only so far that slides, extra problem sets, and auto-graded homework can compensate for the vicious pricing cycle, particularly since the commodity materials get more organized and feature-rich over time. Eventually, the major publishers came to the conclusion that the only sustainable “better” they could shoot for is more educationally effective.
Pearson was the first out of the gate with a massive push for “efficacy.” ((Disclosure: Pearson is a sponsor of EEP.)) They have bet and are still betting the company on that strategy. But as I have written about here before, the fundamental problem is that products can’t really be “efficacious” in and of themselves unless the educators in whose class the materials are being used (a) agree with the efficacy goals that have been defined by the product developers and (b) change their teaching to work with the educational strategies designed into the products. More fundamentally, the educators have to trust the research claims of the vendors in order to even think about the product-defined efficacy goals, much less adjust their teaching strategies. Pearson’s original articulation of efficacy failed to account for any of this. They have since adjusted their course, and other curricular materials developers—most notably McGraw-Hill Education and Macmillan among the larger players—have followed suit by also focusing more on encouraging faculty to buy into research-backed teaching practices and then, having obtained that buy-in, show how their products support and implement those practices. ((Disclosure: McGraw-Hill Education is a sponsor of EEP and subscriber to our Trusted Advisor market analysis service. Macmillan is a sponsor of EEP.)) But for all their good efforts—and they are generally, good, honest efforts—these vendors are pushing string. Most academics will never take them seriously as a source of advice for considering deep and scary changes to their teaching practice.
Meanwhile in the LMS space, the developed markets have saturated and are stabilizing. New adoptions appear to be down. There are many developing markets to plumb, but they are slow and expensive to develop. So LMS vendors too have been trying to move up the value chain by talking more and more about student success. D2L has focused for some time now on the course design process and has been adding tools to its portfolio like LeaP, which is a tool for recommending personalized supplemental curricular materials. ((Disclosure: D2L is a sponsor of EEP and a subscriber to our LMS market analysis service.)) Blackboard has gone so far as to promote themselves as “your partner in change,” to the point of deprecating their flagship LMS project as “not enough.” ((Disclosure: Blackboard is a sponsor of EEP and a subscriber to our LMS market analysis service.))
And yet, the LMS companies face the same uphill battle with credibility that the textbook publishers do. By and large, academics are not going to look to their LMS providers for guidance on how to change their teaching practices. The same goes for the upstart product categories like learning analytics. Vendors will struggle to convince academics to change their teaching practices, but their products will mostly fail to demonstrate meaningful learning impact until the academics adopt practices that take full advantage of the products. All these vendors need to climb a wall of credibility with academics, but they can’t do it unless somebody throws them a rope. (Companies with significant faculty-facing service components have the best chance of swimming upstream, but that’s another post for another time.)
All the institutions in the sector—all types of colleges and universities, all types of ed tech vendors—have realized that they have a problem and are starting to realize that they can’t solve it on their own. They recognize that the core problem is that colleges and universities need to get much better at supporting student success, however their particular students may define it. They all want to get there and are starting to look to each other for help. But they don’t know how, and most of them can’t do it alone.
There’s only one stakeholder group in this picture that has not gone through the process of seeing that they have a deep problem and accepting that they need help solving it yet. Have you spotted who they are?
The people who can actually solve the problem
While the shift in incentives has reached a tipping point for the institutions, the same cannot be said for the faculty. Their graduate training is largely unchanged. Their tenure and promotion criteria are largely unchanged. The rewards and accoutrements of professional accomplishment are largely unchanged. Faculty have been given no reason to change; therefore, they don’t. Everybody knows this is true.
Or not. There are several vital aspects of this story which everybody “knows” that are either misleading or flat out wrong.
First, faculty do change. Anybody who has significant experience with the development of online learning programs or other course redesign efforts has seen it happen. They have faculty say that their experience in the redesigned class has changed the way they teach in other classes. They have watched skeptical faculty turn into preachers of the gospel. There are converts. Despite a dearth of incentives and a plethora of disincentives, despite uneven support, despite the fact that most will earn no glory for it on the other side of the closed doors of their respective classrooms, faculty do embrace pedagogical change when they have the right sorts of experiences that enable them to see the benefits.
Where are these amazing faculty members? They are everywhere and nowhere. They tend to be invisible on their home campuses, although if you ask around in different departments, you might be lucky enough to catch sight of one or three. (Or a dozen.) They have often learned the hard way that there is little benefit and significant pain involved with preaching on their home campuses, so many of them keep quiet and quietly work their magic in their own classrooms. If you want to see them in numbers, you usually have to go to one of the conferences where they congregate. I am going to one this week. One of the main activities of the participants will be crying on each other’s shoulders about how under-appreciated and under-resourced their efforts are on their respective home campuses.
It is also untrue that incentives for faculty to excel in their teaching craft remain rare. It’s still early days, but there are green shoots everywhere. Most of the time, we only hear about a small number of schools that are doing remarkable things. Arizona State University, Southern New Hampshire University, and Western Governors University, over and over again. If you’re a little more knowledgeable, you might have heard about work at University of Central Florida or Georgia State University. And if you’re paying attention to formal scholarship, you might a little about work coming out of places like Carnegie Mellon University, Duke, and Stanford. We could look a little further down the publicity pyramid at places like the University of Maryland Baltimore County. You very likely haven’t heard about the amazing work happening at diverse schools ranging from James Madison University to Coppin State University. I wouldn’t have known anything about the accomplishments of either of these institutions if I hadn’t stumbled upon them through my various travels in this very odd job of mine.
And because the news tends to focus on a few exceptional institutions, it also focuses on three contributors to success that are among the hardest to change: leadership, governance, and money. It is simply not true that the only institutions making real change have once-in-a-generation presidents, an iron grip on the faculty, and/or tons of funding. We see innovation everywhere. And everywhere it happens, it happens because institutions are finding new ways to draw on their most precious yet plentiful resource: their faculty.
There is an old term of art that deserves reviving and refreshing: the scholarship of teaching and learning (SoTL). SoTL is often seen as a grassroots effort by faculty who care about teaching to wrap it in the cloak of academic validity. If the only way that excellence in teaching will be valued by the institution is to get it into peer-reviewed journals, then let’s find a way to get it into peer-reviewed journals. In the past, institutions generally didn’t take the bait. Many treated SoTL as a pat on the head to faculty who were slaving away carrying the heaviest teaching and advising loads. “Here, you care about this teaching stuff. Have a workshop. You can pretend what you’re doing is scholarship for a while. And we’ll give you a certificate!”
That is changing. More and more institutions are realizing that faculty aren’t the problem; they are the solution. But that grassroots energy that comes from SoTL and other faculty empowerment efforts must be aligned with institutional efforts through support, incentives, and research. More and more institutions are making that connection. For example, here’s a graphic illustration of the dynamic, taken directly from Georgetown University’s Designing Our Future(s) web site:
Here are some lessons learned from Georgetown’s white paper about the progress the initiative has made so far:
A few core rules for this innovation work have emerged. First, every project has to push against some structural constraint (the 15-week semester, the credit hour, the nine-month calendar, etc.) and test variations of it. Second, projects cannot be idiosyncratic or depend on the particular interests of one talented faculty member; they have to be pilots from which we can generalize and which we might apply to other scenarios or problems. Lastly, we only fund a Red House project for one year (or the equivalent); after that, if a project is to survive, it has to be absorbed into the curriculum and faculty workload.
Beyond these basic rules we have also learned some valuable lessons about the viability of experimental and creative curricular work in a culture designed for deliberative shared governance and slow change:
We developed strong stakeholder involvement as part of our iterative design process—one that frequently included associate deans, the registrar, compliance officers, and financial aid representatives—early in each project’s development. Likewise, we communicated well and regularly with our board, alumni, and donors
We do not give ourselves as good a grade on continuous communications with faculty. Early on there were many open invitations and speaker events, and a drumbeat of updates. As the work became more intense and demanding, we focused inward, and neglected to continue to reach back out to this important community. We learned it is absolutely critical to spiral communications outward, and to be as inclusive and open as possible, especially as the work takes specific shape within a core group.
Very early on we should have established a formal faculty review and approval process for Red House pilots. We assumed we would work within the Curriculum Committee approval structures, long established for important reasons, but which do not in the end benefit a research and development initiative. Last year, a Designing the Future(s) Advisory Committee was created, with the sole mission of approving and monitoring innovation projects. This system is now working very well; it might have accelerated progress if it had been instituted earlier.
These are very early lessons, and they are somewhat Georgetown-specific. But it’s easy to see some more general principles emerge that could be useful across a wide range of educational and cultural contexts. And some of the most fascinating and remarkable changes are happening at institutions that you never read about, including some that have traditional faculty governance, few financial resources, and leaders who are extraordinary in the “normal” sense that many committed, hard-working, people-oriented academic leaders are in colleges and universities of all shapes and sizes.
I am going to write about some specific examples of this sort of organizational alignment in upcoming posts. For now, I want to spend a little time on the characteristics of a good methodology.
Toward a methodology of Empirical Education
When I think about general methodology that can be adopted and adapted across a wide range of contexts, the two models that come to mind immediately are Agile software development and Design Thinking. I’ll focus on Agile (and particularly Scrum) for the moment because I know it better, but as far as I can tell, the same basic principles apply to Design Thinking.
First, the methodology should be designed to unleash the creativity of the knowledge workers involved in the critical processes. All too often, we take really smart people and put them in a strait jacket of process. We tend to design our mission-critical processes to get us predictable results, often by controlling the human element through various management techniques. The problem arises when we ask for predictable results in an unpredictable environment, having handicapped the very smart people who are best able to minimize the problems that arise out of unforeseen circumstances while maximizing the benefits of unforeseen opportunities. There is no knowledge work I know of that has more frequent and dramatic unforeseeable challenges and opportunities than education. We wrap a lot of process around education, but it’s not the right kind of process to promote excellence by getting the most out of talented educators, just as using Gantt charts was not the right sort of process to promote excellence in software development by getting the most out of talented engineers.
At the same time, empowering knowledge workers is not the same thing as letting them do whatever they want. I have been in an Agile software development environment where the engineers interpreted Agile to mean that they decide everything. The results were not good. All Agile methods that I am familiar with have multiple roles, with each role having certain authority and responsibilities. These roles are designed to be mutually supportive, and the success or failure is very much a success or failure of the entire team and its teamwork. This is a big cultural change for many institutions, where “academic freedom” has come to be used reflexively as a shield from any demands, sometimes because some of those demands are unreasonable or unwise. There has to be a well-defined process by which student success is understood to be the collaborative responsibility of the academic team, working together as an ensemble.
These two basic principles—empowering individuals and working as teams—can generally be captured in a fairly small number of rules and roles, regardless of the flavor of Agile being practiced. And most Agile teams that get them right will function adequately while getting more satisfaction from their work—under relatively unchallenging circumstances. They may even feel that they are doing Agile well. But then there is a whole world of craft that is all about handling context-specific challenges. How do you balance functional versus non-functional requirements? How do you prioritize aging aspects of the software, a.k.a. “technical debt”? How do you manage large projects that require many Agile teams? How do you deal with extrinsic constraints on release timing (like the start of an academic term)? Agile practitioners can always improve their craft, both as individuals and as teams. Entire industries of tools and consulting have grown up around supporting excellence in that craft.
Which is utterly unlike the way in which the industries that surround education function (or fail to function) today. There is a reason for that. An industry designed to promote operational excellence of knowledge workers cannot succeed in absence of a shared understanding among the knowledge workers about what operational excellence looks like. Agile software development is a craft with a lot of consensus around the principles, a track record of results, and enough expert practitioners that knotty problems, along with their solutions, can be shared fairly efficiently across a very large and loosely organized profession. There is a lot of debate too, which is the sign of a healthy ecosystem of knowledge workers advancing the leading edge of their craft. But that debate occurs within the context of a common understanding that is woven into the culture. Practitioners in those debates are rewarded with recognition of their expertise and contribution to the field. And their employers love having these experts and reward them appropriately because their excellence at creatively applying and innovating with the methodology advances institutional goals.
With that cultural substrate in place, a tool or service vendor can come in and say, “We help you solve X sort of problem in your Empirical Education process,” and the prospective customers will, understand what is being offered, be capable of evaluating its utility, and place (monetarily quantifiable) value on that utility. That’s what we need for learning analytics, adaptive learning, or just about any whizzy, trendy ed tech thingamabob you can think of or will be thought of.
Most or all of the elements for a methodology of operational excellence in education exist in the world today. They need to be gathered, distilled, and refined into a learnable, repeatable, and adaptable practice. That is the outcome we aspire to achieve in collaboration with the participants in the EEP, not to mention support from the collective wisdom and will of higher education writ large.
Moving forward
As I wrote earlier, I will be blogging about relevant examples we are seeing, on both the institutional side and the vendor side, in the coming days. And EEP will soon be announcing the first release of some tools that can help form a foundational layer of the institutional infrastructure for Empirical Education. In the meantime, if you are going to be at the Online Learning Consortium Accelerate conference, I will be moderating an EEP-relevant panel discussion of SoTL on Thursday at 11:15 AM in Oceanic 1. From there, some of us will head to the exhibition hall, where we will have an EEP meet-up at the Soomo booth (#226) at 12:15 PM. You don’t have to be a member of the current EEP cohort to join us; the EEP-curious are welcome.
Assuming [new Instructure President Dan Goldsmith’s] trial period goes well, I think it likely that he will be promoted to the top job within 9 months. The reason I pick this time frame is anything too close to InstructureCon 2019 poses the danger of being a distraction during the most important event of the year for the company.
Today at 4:00 PM ET, the company announced,
Instructure, Inc.(NYSE: INST), a leading software-as-a-service (SaaS) technology company in education, learning, and employee development, today announced that the Board of Directors has appointed Instructure President, Dan Goldsmith, as Chief Executive Officer, effective January 1, 2019. On that date, Josh Coates will transition from his role as CEO to Executive Chairman of the Board. Goldsmith has also been appointed to the Board.
So the transition actually happened a little less than three months after Mr. Goldsmith’s Big Top début. Instructure is not wasting any time.
In my original post, I wrote about Dan’s coming on board as part of a larger set of changes that the company is going through. I referred to the company as entering “those awkward teenage years” because it is in the beginning of a transition to becoming something else:
Instructure’s unbelievably long age of innocence may finally be coming to an end. That doesn’t mean that it is going to fail or to become the next EdTech company that everybody hates. It does mean that it is beginning to go through some changes, that some of those changes will be awkward and hard, and that the company will eventually grow up to become somewhat different than it has been. Not necessarily better or worse. But necessarily different.
We tend to write a lot about the short to medium term changes—the “adolescence” in this case—because one of our primary audiences is the group of folks at colleges and universities who may see changes in the behavior of a vendor that they depend on and need to understand the drivers behind those changes in order to make good decisions for their institutions. And those changes, in turn, are at least partially driven by finance and markets and other business stuff. As I write this post, we are less than an hour away from Instructure’s quarterly earnings call. Many of the people listening to that call are concerned, not because the company is in financial free fall, but because it might not grow as quickly in the next couple of years—or even in the next couple of months—as it has in the past.
The pathology of investor short-term thinking is, unfortunately, part of what university folks need to understand in order to understand the behavior of these companies. That said, while we’re going to continue writing about the short and medium term, Phil and I are going to take a step back from the serpent-eating-its-tail obsession with quarterly performance and write some pieces about the long-term prospects for the LMS, both as a product category and as business. Neither of those aspects are a static as they appear to be. In fact, while some of the behaviors of the various providers are motivated by those short-term demands of the markets, others have to do with tectonic shifts that aren’t yet obvious but may be far more consequential in the long run. The LMS continues to have a future, and it’s a surprisingly interesting one in some ways. We’ll have more to say about it in the coming weeks.
One piece of news we never circled back to after the crush of LMS conference season updates was the ending of the Blackboard’s membership in the Moodle Partner program. To recap, Moodle Pty., the company that runs Moodle development and owns the Moodle trademark, suddenly announced right around BbWorld that it was ending Blackboard’s membership in the Moodle Partner program. Blackboard scrambled to put out a press release saying the decision was mutual. What really happened, and what will happen next?
The decision was mutual but the messaging wasn’t
Blackboard’s partnership agreement was up for renewal. From what we can tell, both sides recognized that the discussion around terms wasn’t going well and were starting to contemplate the contingencies in the event that the negotiations failed. Moodle creator Martin Dougiamas made a unilateral decision to call it and announce the break-up, but I think the handwriting was on the wall already.
The timing was clearly bad for Blackboard from a publicity perspective. Coming at the end of BbWorld, it basically stepped on any announcements they had. That timing could have been deliberate or coincidental; the contract renewal date was set, so the timing was already set to a certain degree. That said, the fact that Moodle did not warn Blackboard or work with them on a joint statement suggests that, at the very least, it was not as amicable a breakup on Moodle’s side as their press release and both sides’ public comments suggest. Which makes the timing of the announcement look a little more likely to have been planned. To be clear, (a) that’s speculation on my part, and (b) I really don’t know enough of the details of the negotiations to piece together exactly what was said or done by whom at what point for what reason. These sorts of negotiations are always complex, and the Blackboard/Moodle relationship was particularly fraught for a number of reasons. But partly for that exact reason, you should take the amicable language on both sides with a grain of salt. Just because somebody doesn’t want to talk trash in public about their ex doesn’t mean that there aren’t…feelings.
The tick-tock and emotional valences of the break-up are not ultimately consequential. The real question is what happens next for both organizations. On the Blackboard side, Moodle has been an engine of international growth for them. Over the years, they have acquired major Moodle hosting providers in North America, South America, Europe, and Australia and rolled them into their Moodlerooms business (which was itself an acquisition). While the ending of the relationship doesn’t prevent Blackboard from continuing to use the open source Moodle software (or acquire more Moodle service providers), it does raise branding concerns for them in the immediate term and risks of diverging—forking—from that code base in the longer term.
On the Moodle side, Blackboard’s acquisitions meant that, increasingly, Moodle Pty was financially dependent on Blackboard. Historically speaking, the primary revenue model for the company has been to collect a percentage of Moodle-related revenues from Moodle hosting and support providers in their Moodle Partners program. As Blackboard acquired the larger and more successful Moodle Partners, they also acquired major sources of Moodle Pty’s revenue. At one point, we estimated they accounted for half or more of the company’s total revenues (although Moodle Pty has not publicly disclosed enough financial details for us to make this sort of estimation with a high degree of accuracy).
So what happens to Moodle and Blackboard post-breakup?
Short term: Probably not much
The most immediate short-term consequence for Blackboard is that they have had to change their product name. While they can continue to use the Moodle source code under the terms of its open source GPL license, Moodle Pty owns the trademark to the Moodle name. So Blackboard has had to change its product name to Blackboard Open LMS. They are able to say things like “Blackboard Open LMS is based on Moodle,” but they can’t actually call their product Moodle. That’s a tricky messaging problem for them in a couple of ways. First, a big part of the company’s sales strategy is to convert self-hosted Moodle customers to Blackboard’s SaaS product, arguing that such a move provides customers with an easy migration and all the benefits of Moodle plus the stability of SaaS and the value-added features that Blackboard bundles with the product. With the product name change, the company has taken pains to emphasize that they remain committed to “an easy on-ramp and an easy off-ramp” for Moodle schools through continuing compatibility.
The second question is the degree to which Blackboard’s customers have specific brand loyalty to open source, Moodle, or Martin Dougiamas’ leadership. Blackboard reports some customer push-back in Southern Europe and little customer concern about the transition elsewhere. We have not yet seen evidence of large-scale concern from Blackboard’s MoodleRooms customers about the transition, although such concerns would be not necessarily be visible to us this quickly if they exist. Blackboard’s Moodle-derived business—I think I can still call it that—is not likely to contract in the short term as a result of the break-up and may or may not experience a slow-down in growth. We don’t see any indicators of a slow-down at this time, but we’ll keep an eye on it. (We’re getting better at detecting switches from self-hosted Moodle to Blackboard Open LMS, so our ability to track Blackboard’s growth on this platform will continue to improve.)
On Moodle’s side, Moodle Pty. received $6 million AUD in investment money in the recent past. We don’t know how much revenue they company lost with the ending of the Blackboard partnership, but the company has cash to burn if it needs to do so. This leads to at least two significant consequences. First, Moodle Pty’s ability to pay developers to work on the platform is unlikely to be disrupted in the medium term. Second, unless the company changes its disclosure policy, it will be a while before we know how much the loss of Blackboard’s partnership revenue impacted Moodle Pty and how well they have been able to compensate with new sources of revenue. If the company performs well, we may never know. If they are burning cash to cover for the loss of revenue, we won’t see evidence of that until the cash runs out. Which could be a couple of years, even if things are not going particularly well.
Any visible impacts are likely two or three years out
For Blackboard, there are a few longer term risks. First, the rebranding and Moodle relationship may complicate their story enough that it creates more of an opening for competitors when self-hosted Moodle schools decide to move to external hosting. Second, there may be a quiet dissatisfaction with the rift among current customers that won’t be visible until contracts come up for renewal. It’s hard to gauge the size of these risks because there wouldn’t be many visible signs of them this early. A lot will depend on the strength of Moodle’s brand versus Blackboard’s marketing and customer service execution. The longer term threat is that it becomes harder for Blackboard to retain Moodle compatibility as their code bases drift apart. That risk has more like more a four- or five-year time horizon, and a lot can happen in that time to change the potential impact of that risk. For Blackboard, the breakup may not have a major impact on their business. We’ll see.
For Moodle, everything rides on their ability to grow alternative sources of revenue. The company has been touting newer offerings such as MoodleCloud, MoodleNet, LearnMoodle, and MoodleServices. Since we don’t have any external evidence that these are material sources of revenue for the company, and since the company itself has not shared numbers that we can independently evaluate, it’s very hard to tell what their chances are. Moodle has a huge installed base, which gives the project a lot of momentum. But the company that drives most of the core platform development has a business model that has not aged well and is in the process of diversifying into business models that are as yet unproven. I remember enough physics to know that momentum and acceleration are not the same thing. I think the risks are probably greater for Moodle Pty. than they are for Blackboard. But both sides of the equation bear watching.
This story feels like it’s significant. But at this point, there’s little hard evidence to show whether it will be, and if so, how. We’ll just have to wait and see.
A while back, I wrote a post about the four levels of Empirical Education. To recap, they are as follows:
Intuitively empirical: Intuitively empirical educators are curious about their students and try to figure out how to help them when they see them struggling. They try different things and pick up tricks in their teaching as they become more experienced in the classroom.
Mindfully empirical: This means that the educators try to create as many opportunities as possible to evaluate how their students are doing and make little (or big) adjustments to her teaching strategies constantly as they get to know their students better. Intuitively empirical educators are empirical in the moment. Mindfully empirical educators are empirical by design.
Metacognitively empirical: Metacognitively empirical educators have made the leap from assessing their students to assessing themselves. When a class struggles with a concept semester after semester, metacognitively empirical educators don’t just accept that the topic is hard. They ask whether their teaching strategies might be part of the problem. They challenge their own beliefs about good teaching.
Socially empirical: When educators reach the level of being socially empirical, it means that they have begun to see that testing teaching strategies and learning to improve can be a shared endeavor. It can be a discipline, with common language and standards of evidence for effectiveness. Socially empirical educators see teaching not just as an art that is personal and ineffable but as a craft that can be taught and learned, and maybe even as a science that can be advanced through shared research and peer review.
This isn’t just a taxonomy. It’s a theory of change. Anyone who has participated in course redesign efforts likely recognizes the milestones of this progression. That’s not an accident. Course redesign, particularly when properly facilitated by experts who are themselves practitioners of Empirical Education, can move the instructors who are going through the redesign process at least partway up the ladder. In fact, it often happens spontaneously.
Let’s take the example of launching a distance learning program. In most cases, these programs are going to attract intuitively empirical educators. People who aren’t curious about the process of educating students usually don’t volunteer to teach using a method that they’ve never tried before. (And when they are forced to do so, it is often a disaster.) So distance learning programs are often lucky to get a preponderance of instructors who are already on the first rung of our ladder.
Teaching online with any degree of quality almost always forces a course redesign. Educators moving from a physical classroom to a virtual one lose some senses while gaining others. Likewise, certain strategies they have come to rely on in the classroom won’t work online and have to be translated to or replaced by strategies that will. And the educator that is new to online teaching isn’t always going to know what will work and what won’t. They are almost required to become mindfully empirical in order to figure out how to replace the bandwidth they lost in the transition.
Sometimes, the process of trying new strategies because of the redesign provokes the empirical educator to rethink some of her basic assumptions about teaching. Anyone who has helped educators get courses online for more than a couple of years has likely heard at least a couple of them say that the experience of teaching online has caused them to change the way they teach in the classroom. That’s the beginnings of metacognitively empirical education.
And this is where things start to get hard, for several reasons. First, it’s very difficult to get faculty to make the leap to metacognitively empirical education without a conversion experience that happens when they have to teach their course differently. Both experience and research tell us that many educators—and probably most humans—have deeply held beliefs about what makes for effective teaching based on their own formative experience as learners. No matter how otherwise rational and open-minded they are, they are not likely to be persuaded by research studies. They have to live the change. Outside of distance learning programs, it’s hard to programmatically create new opportunities for formative experiences.
Second, even when they do live the change and bring new ideas back to their other classes, that doesn’t mean that they have reached the point where they are mindfully or systematically examining their own teaching strategies. Doing the latter requires both another step of self-reflection and some new skills. So a key challenge for Empirical Education is to make it easier to reliably give educators the right experiences, prompts for further reflection, and tools for investigation, in the proper order. Universities aren’t currently set up to do this. But they could be.
The third challenge in climbing the ladder is moving from metacognitively empirical to socially empirical. Educators who have made the metacognitive leap tend to get excited. It can be a life-changing experience. So they often want to evangelize. But where? If their home institution isn’t already creating a fertile environment for them to share teaching strategies with their peers—which it usually isn’t—then the metacognitively empirical educators tend to go to discipline-specific conferences, where they end up swapping ideas with a small circle of people who are already converted and largely share their background and training. This is far from ideal for cross-fertilization. So another great challenge of Empirical Education is to create new opportunities for metacognitively empirical educators to find each other and learn from each other across normal silos in higher education, like discipline or school type.
Phil and I believe that colleges and universities in all segments across the sector are beginning to make the transition from a philosophical commitment to student success toward an operational commitment toward student success. What that looks like in the end, and how long it takes, is up for grabs. Empirical Education is, among other things, a theory of change that is intended to accelerate the progress of that transition while strengthening the institutions of higher learning that have served civilization so well by empowering and motivating educators to be change agents.
Almost five years ago, when Pearson announced that the company would reorganize itself around efficacy, I was impressed by the degree to which the company was going “all in” on a very much unproven strategy. It was incredibly bold. I wrote,
In all my years of covering the ed tech industry, I have never seen a company be so explicit and detailed about their strategy as Pearson is being now with their efficacy publications. Yes, there is plenty of marketing speak here. But there is also quite a bit about what they are actually doing as a company internally—details about pilots and quality reviews and hiring processes and M&A criteria. These are the gears that make a company go. The changes that Pearson is making in these areas are the best clues we can possibly have as to what the company really means when they say that they want efficacy to be at the core of their business going forward. And they have published this information for all the world to see.
These now-public details suggest a hugely ambitious change effort within the company. Phil and I have consulted for a few textbook publishers, including Pearson, and I worked for Cengage for a year and a half. We have a pretty good idea of the magnitude of the change management challenges these companies face right now and the strategies that various publishers are bringing to bear in an effort to meet them. I can say with absolute conviction that what Pearson has announced is no half-hearted attempt or PR window dressing, and I can say with equal conviction that what they are attempting will be enormously difficult to pull off. They are not screwing around. Whatever happens going forward, Pearson is likely to be a business school case study for the ages.
Pearson bet the farm on efficacy. As far as I can tell, they are still betting the farm on efficacy. If anything, they have doubled down in the five years since I wrote those words.
There are a number of reasons why this bet is remarkable, not least of which is that we still don’t know if a curricular materials company can be successful in the long run by promoting efficacy as a primary value proposition. In addition to all the hard work that Pearson needs to do to credibly claim that their products support some sensible definition of efficacy, they also have to tackle the equally hard challenge of convincing faculty that the company’s vision of efficacy, as delivered in their products, is a good reason to pick their product instead of the (many) alternatives. They have an enormous customer communications challenge. That was one of the main points of my original 7,000-word post on the company’s strategy.
Which is why I find it mystifying to see an article in Forbes which seems almost designed to distract from or even directly undermine the efficacy message that the company has been carefully honing over the last half a decade. The article, “How 174 Year Old Pearson Is Developing The Netflix Of Education” is a longish interview with Albert Hitchcock, Pearson’s Chief Operating Officer and Chief Technology Officer. The jarring title refers back to Mr. Hitchcock’s previous use of the analogy in an interview entitled ‘Pearson aims to become the ‘Netflix’ of education.‘ In responding to that first piece two years ago, I wrote,
Given his profile on LinkedIn, Mr. Hitchcock appears to be new to education except for whatever memories he has of his own days as a student. Let me offer a couple of suggestions on how to get along in education for him and the many vendor employees who are in a similar situation:
Never, ever, say that you want your company to be the Uber of education, the Airbnb of education, the Pokemon GO of education, or the [insert name of tech darling] of education unless you really enjoy being a recluse (or you are secretly a double agent for your employer’s direct competitor).
If you absolutely must say something like the above, then do not say you are the Netflix of education. Honestly, Netflix isn’t even great at being the Netflix of movies. The last time they recommended a movie that I actually wanted to watch was…uh…never.
There is a recurring cultural fantasy that “solving” the education “problem” consists of creating a customized playlist of little content bits. So really, more like the Spotify of education, if you want to play that game. This idea enrages educators because it trivializes what they do. Nobody who has taught believes that proper sequencing of content chunks is the hard part. (For a fully fleshed out prior example—or a “worked example” in teaching parlance—of how the sorts of comments Mr. Hitchcock have made typically play out in educational corporate branding over time, see my post-mortem on Udacity’s pivot away from higher education.)
In the more recent Forbes piece, the interviewer picks up on the Netflix analogy and asks Mr. Hitchcock about it. Not only does Mr. Hitchcock choose not to disavow the analogy; he expands on it by adding Spotify and Amazon. I’ll parse his exact language later in this post in the interest of fairness, but my point from the previous post stands. Regardless of the intention behind the analogies, they are toxic and should be avoided at all costs. This is doubly true for Pearson because, as I will explain in the next section of this post, they also run directly counter to the more credible, more refined articulation of the efficacy strategy that the company is promoting in 2018.
The fact that the analogies did make it into an interview with a mainstream publication like Forbes raises some larger questions. Is Pearson less committed to efficacy than I have believed them to be? Is there poor alignment in the company around what efficacy means? Are they just really bad at messaging? Did Mr. Hitchcock’s intent somehow get represented unfairly by a few poorly chosen words that were then taken out of context by an editor?
To find out the answers to these questions, I spoke with Pearson’s President of Global Product Tim Bozik, SVP of Efficacy and Research Kate Edwards, and CEO John Fallon. Here’s the short version of what I have concluded, based on those interviews and other evidence:
I am mostly still convinced that Pearson is fully committed to efficacy as a primary value proposition for all of their products and services going forward.
I do believe that the Forbes interviewer was bad, as were some of the editorial choices made by the publication. (Particularly the headline.)
Even granting the benefit of the doubt on the previous point, Mr. Hitchcock made a number of very bad choices that no executive in his position should make and that cannot be explained away by blaming the editor or the interviewer.
Mr. Bozik and Ms. Edwards seemed fully aligned on what Pearson means by efficacy, how to talk about it, and how central it is to the company. Within the rules that they were bound to follow in an on-the-record interview through formal PR channels, they did the best job they could to articulate the most attractive and compelling version of Pearson’s efficacy strategy while diplomatically establishing some distance from Mr. Hitchcock’s questionable analogies.
Mr. Fallon chose a different path. He went to some lengths to justify or explain away the analogies. In doing so, he re-opened questions that Ms. Edwards and Mr. Bozik had all but closed for me regarding whether Pearson has the sensitivity and message discipline they will need to earn their customers’ trust regarding their commitment to efficacy.
For the long version, read on.
But before you do, you should be aware of our conflicts of interest, which I’m going to describe in more than the usual detail. Pearson is a current sponsor of the Empirical Educator Project. In addition, they have periodically engaged us in consulting projects over the years (although we do not have any current engagements with them.) Some of these projects have involved their efficacy work, either directly or indirectly. We received prior permission from the company to blog publicly about the first such engagement. Most recently, we were hired to review the company’s public efficacy reports, provide the kind of feedback that we might publish in a blog post, and offer suggestions for improving the work. You can judge for yourself whether these engagements make us more or less trustworthy in our assessments.
Netflix and Spotify analogies are fundamentally incompatible with Pearson’s view of efficacy
In order to understand why these analogies are particularly bad for Pearson’s efficacy effort, it’s important to understand how the company’s position on efficacy has evolved. In that original post five years ago, I wrote,
Let’s think some more about the analogy to efficacy in health care. Suppose Pfizer declared that they were going to define the standards by which efficacy in medicine would be measured. They would conduct internal research, cross-reference it with external research, come up with a rating system for the research, and define what it means for medicines to be effective. They would then apply those standards to their own medicines. And, after all is said and done, they would share their system with physicians and university researchers in the hopes that the medical community might be reassured about the quality of Pfizer’s products and maybe even contribute some ideas to the framework around the edges. How confident would we be that what Pfizer delivers would consistently be in the objective best interest of improving health? This is not entirely hypothetical; much of the drug research that happens today is sponsored by drug companies. Unsurprisingly, this state of affairs is viewed by many as deeply problematic, to say the least. It certainly doesn’t help the brand value of Pfizer. But at least much of that medical research is conducted by physicians and academic researchers and is subject to the scientific peer review process. Pearson is creating their framework largely on their own, selectively inviting in external participants here and there.
I get why they had to do this. The company is bleeding money, it will take some time to stop the flow of blood, and they couldn’t wait to build consensus before they tied the tourniquet. But it is not going to get them where they want to go. While there are obvious concerns about ethics and about whether such a company-driven approach is fundamentally compatible with progress on complex questions such as defining what an education is good for and how we know when we have achieved these ends, I want to focus on the business aspects of the problem. I want to focus on why continuing down this path is bad for Pearson. Or rather, why driving hard toward becoming facilitators rather than owners of efficacy research is good for Pearson.
I could give a number of examples, but one should hopefully suffice. In preparing to write this post, I asked Annie Cellini, Pearson’s Senior Vice President of Marketing and Strategy, whether Pearson intends to share the completed rubrics for their products with customers and prospects. This was her reply:
Though we don’t plan to share product efficacy scoring as part of our sales and marketing materials per se, where a product has a strong research and evidence base, we will communicate that to customers. It’s also worth saying that the most important output of an efficacy review isn’t a rubric score. We believe that much of a review’s value comes from the conversations that it prompts teams to have, which focus on the path forward, and on how to improve the product or service from a learner perspective. A poor score does not mean the product doesn’t work well. It often means that teams are not collecting the type of data needed in order to get a sufficiently robust view of the product’s efficacy, or they may not have a sufficiently practical plan to continuously enhance the product based on data. Their improvement plan will encourage them to start gathering new information, to start working in new ways, and to make sure that their customers are aligned with the outcomes they plan to achieve and understand their role in the product’s path to efficacy.
This is a perfectly sensible and responsible reply if you believe that the main value of the Efficacy Framework to customers is in the data that results from the work a product team does after an efficacy review. But remember, the magic of the rubric is in the norming conversations. Annie’s reply suggests that Pearson understands this in terms of the Pearson-internal processes but not yet in terms of their relationships with their customers. If Pearson were to say to faculty, “Here’s what we think we know about the efficacy of this product, here’s what we don’t know yet, and here is how we are thinking about the question,” they might get a number of responses. Maybe they would get, “Oh, well here’s how I know that it’s effective with my class.” Or “The reason that you don’t have a good answer on effectiveness yet is that your rubric doesn’t provide a way to capture the educational value that your product delivers for my students.” Or “I don’t use this product because it has direct educational effectiveness. It frees me up from some grunt work so that I can conduct activities with the class that have educational impact.” Most of all, if you’re John Fallon, you really want faculty to say to their sales reps, “Huh. I never thought about the product in quite those terms, and it makes me think a little differently about how I might use it going forward. What can you tell me about the effectiveness of this other product that I’m thinking about using, at least as Pearson sees it?” And you really want your sales reps to run back to the product teams, hair on fire, saying “Quick! Tell me everything you know about the effectiveness of this product!”
Pearson won’t get that conversation by just publishing end results of their internal analysis when they have them, which means that they have a high risk of failing to align their products with the needs and desires of their market if they think about the relationship between their framework and their customers in that way. I don’t think Pearson fully gets that yet. While the authors of The Incomplete Guide frequently invoke terms like “community” and “leaders” in the document, they generally seem to mean the community and leaders within Pearson. The company’s efforts to reach out to the academic community for feedback and participation are generally framed as an extension of their efforts rather than the very heart of them. And yet, Pearson’s brightest possible future is not as a company that designs educationally effective products, but as one that facilitates conversation and research about efficacy within the broader academic community (and in so doing is able to design products that their customers agree are effective for important educational goals as determined by meaningful measures).
The Netflix, Spotify and, to a lesser extent, Amazon analogies all speak directly to the question of whether Pearson intends to define efficacy for educators or with educators. Netflix famously just deleted—not just removed, but deleted—all user reviews and switched from a five-star review system to a simple “thumbs-up/thumbs-down.” The subhead on the Vanity Fair article I linked to in the previous sentence is “A step closer to a wholly ‘because you watched’ world.” The implication is that Netflix trusts the algorithm more than the humans to evaluate quality. Equally famously, Apple CEO Tim Cook has highlighted the company’s decision to use human curators of music, in contrast to Spotify’s decision to rely completely on its algorithms:
We worry about the humanity being drained out of music, about it becoming a bits-and-bytes kind of world instead of the art and craft.
Whenever Pearson makes any mention of Netflix or Spotify—or Amazon, which is known for its recommendation engine as well—the company risks appearing to come down on the side of the algorithm over the human. Educators worry about the humanity being drained out of teaching, about it becoming a bits-and-bytes kind of world instead of the art and craft. In 2018, is that where Pearson has come down on their definition of efficacy?
Not as far as I can tell.
Pearson, in fact, recently made a high-profile hire away from Intel of artificial intelligence expert Milena Marinova. Here’s how she described her work in an interview with The Bookseller:
Right now, I am working on developing human-centric AI – this means making the learning experience better for students and teachers; enabling lifelong learning through more accessible and affordable products; and building better products and solutions using new technology.
What does “human-centric AI” mean in this context? Mr. Bozik commented on this directly in our interview. He described it as an effort to help teachers and students get a better view into their own learning. He said the company is placing a strong emphasis on early intervention and formative assessment, and providing “feedback and insights.” He also said,
Pearson strives for respect for both the ambition and the humility of [its efficacy strategy]. The ambition is that efficacy means outcomes. Full stop. It’s the potential to help people live better lives. That is our purpose. The humility part is that it’s hard. We don’t underestimate the difficulty of it. It starts with having an empathy for teachers and learners, rather than just teaching and learning.
For Ms. Edwards’ perspective, while I could quote her from my interview, I think you’ll get a better sense of her from her lightning talk at the Empirical Educator Project summit in February:
Here’s the part of her talk that jumps out as relevant to the question at hand:
How do we maximize the uniquely human attributes that educators—faculty—bring to the table, and combine them with all the productivity enhancements that come from things like data science, AI, and the improvements we’re seeing as the result of technology. All with the emphasis on helping students achieve outcomes that really matter to them.
Ms. Edwards then goes on to talk about the resources that Pearson either has already contributed or will contribute under a Creative Commons license. These include a set of rubrics for evaluating curricular materials (or course designs) based on a set of academically accepted and empirically verified learning science principles and a set of tools for instructors that want to conduct action research. It’s these contributions, these public actions, that speak the most persuasively to Phil and me when we evaluate a company’s intent. (For more on Ms. Edwards’ views and her characterization of Pearson’s efficacy work, you can read her recent posts on LinkedIn.)
Taken together with Pearson’s other work, such as their first publicly released efficacy reports on their products, the message conveyed by the actual work that I have seen from the company is clear: They aspire to define efficacy with educators and students. As I wrote earlier in this post, I have had opportunities to view this work from inside and out over the past five years. I honestly believe that the people working on the efficacy and product teams—Ms. Edwards’ and Mr. Bozik’s teams, respectively—buy into that goal and are actively working toward it.
Pearson’s intent doesn’t matter if nobody believes them
The problem is that intent of the people working at the company, level of commitment to that intent by the CEO and the small circle of people that make decisions within a company, and customer perception of intent are three different things. When I published my post the first time Mr. Hitchcock was quoted talking about Netflix, some of the people who worked on product design and efficacy at Pearson were quite angry with me. They felt that I had misrepresented Pearson’s position, because—and this is key—they also felt that Mr. Hitchcock’s comments were not representative of Pearson’s position. I replied that I wasn’t sure there was any way to determine what Pearson’s position is. Mr. Hitchcock is quite senior. He reports directly to the CEO. I could ask other executives for their own responses, but they wouldn’t be any more authoritative than Mr. Hitchcock’s. Unless I could speak directly to Mr. Fallon—which I didn’t believe would be possible at the time—I had no way of definitively determining which beliefs the elusive entity known as “Pearson” holds.
What I did know is how academia would receive the phrase “the Netflix of Education.” I knew this because Mr. Hitchcock was not the first person to use it, as was acknowledged by a Pearson employee in a 2016 EdSurge story entitled “Why We Don’t Need a ‘Netflix for Education.’” Its toxicity was widely known inside Pearson, at least within the product and efficacy groups.
Last month, five years into the massive effort to align the company around efficacy, the analogy shows up again, expanded upon, by the same senior executive. And this time, it’s not published in a specialty IT outlet. It’s in Forbes. Interviews at that level do not happen at big companies without being vetted. If you’re a senior executive at a company like Pearson, you go through formal media training, and you are often accompanied on interviews by a PR professional. These tend to be usually serious, orchestrated affairs. There are rules. How was it possible that this interview was approved, went through the proper channels, and went off the rails? Do Pearson executives see it as being as far off-message as I do? Is there conflict or confusion about the efficacy strategy at the highest levels of the organization that isn’t obvious to the casual observer?
Let’s take a look at what Mr. Hitchcock actually said and then turn to the executive on-the-record reactions to it.
Mr. Hitchcock’s comments were bad and cannot be explained away by bad editing
Here’s the part of the article where Netflix came up:
High: You described the vision of delivering Pearson’s education, content, and services through a single platform as creating the “Netflix of Education.” Could you talk about this long-term vision?
Hitchcock: The intention of the message was to have the viewpoint that we needed to move to a platform type of model where we have all our products, services, and capabilities that we deliver to our customers in a single ecosystem. A great deal has been written around this model at Pearson, and that is especially relevant as our company grew through acquisitions. Ours is a diverse business that is over 170 years old and has had many different types of companies under the umbrella of Pearson for many years. Recently, education has become the primary focus, but that complexity that was acquired over those decades was inherent within the company. The way we serve our customers had been across many different brands and many different types of digital products. We sell millions of books and material on the digital side. We had to figure out how to transform that to be a consistent, high-quality branding experience and one that is comparable to the best customer experiences out there.
Silicon Valley companies create the benchmark for the digital experience by being platform businesses. Our vision is to leverage the opportunity to transform along similar lines in terms of having a single platform globally that could deliver all our educational content and courseware. Furthermore, this would allow us to move into a more personalized experience that delivers high-quality education outcomes. It would be game-changing for not only Pearson, but for the entire industry if we could create that single platform, similar to Netflix, Spotify, and Amazon. [Emphasis added.] This platform would be highly scalable, global in nature, high-quality, and a platform that could deliver all our experiences around the world to millions of learners. We are currently working to deliver high-quality courses to students that are proven to help them learn and progress their lives through education and ultimately to their professional lives.
In the interest of fairness, I’ll enumerate the extenuating circumstances:
The interviewer, not Mr. Hitchcock, brought up Netflix. And a Forbes editor, not Mr. Hitchcock, chose to feature Netflix in the headline.
Mr. Hitchcock seems to be making a point about the need to support a seamless, end-to-end, born-digital customer experience for students. Further, he is absolutely correct that Pearson, like all of the larger textbook publishers, grew through acquisition and have had a patchwork of many different systems that were never designed to work together. These are entirely valid and important issues for Pearson’s CTO and COO to discuss.
He does not take the last, most toxic step (this time) of describing Pearson as the “Netflix of education.” In fact, he seems to make a modest effort to narrow the scope of the analogy in response to the question: “The intention of the message was to have the viewpoint….”
Is it possible that Mr. Hitchcock meant something innocuous and unobjectionable by the analogy? Yes, it is. Is it possible that he honestly tried, in his own way, to communicate the narrowness of his intent with the analogy? Absolutely. Is this a relatively minor and understandable slip that would have been fine if it had just been worded a little more clearly?
Absolutely not.
I reiterate: There are no circumstances under which any EdTech executive should ever make an analogy between their company or offering and Netflix or Spotify. Period. Full Stop. It is not a mistake that anyone in Mr. Hitchcock’s position should ever make. In fact, if the analogy is brought up by an interviewer, as it was in this case, the first words out of the executive’s response should be “To be clear, Pearson does not aspire to be the Netflix of education.” That goes doubly for Mr. Hitchcock because he actually said that once before and should have taken the opportunity to correct the record, and trebly because the analogy can easily be interpreted as being in direct contradiction to the company’s bet-the-farm strategy, around which it has taken them five years to arrive at their current calibration in message and in action.
Nor can this easily be chocked up to bad editing. In fact, I question whether there was any significant editing whatsoever. This isn’t one of those tightly written articles in the reporter’s own voice with a one- or two-sentence quote sprinkled in here or there. This is an interview, with short questions followed by long answers that do not appear to be heavily edited for clarity or brevity.
Even worse, there is a pattern to Mr. Hitchcock’s answers which leave the distinct impression that he, the CTO and COO, is directly responsible for or taking credit for the company’s strategy on teaching and learning. He deftly pairs “I believe” statements with “Pearson has done” statements. It’s very easy to read these statements as Mr. Hitchcock making a causal connection between his beliefs and Pearson’s strategy decisions, which is an impression that is only reinforced by the sycophantic fawning of the interviewer.
Here’s an example:
High: How is the process of experimentation around new technologies done? Specifically, as new technologies emerge, how do you encourage your team to begin to experiment and to decipher more of the specifics of how that might be leveraged?
Hitchcock: […] Overall, I believe it is a combination of aspects. I believe it is about us looking at other ideas that other companies are implementing and seeing where we can bring these concepts together to deliver with the lasting initiatives that we are trying to put in place at Pearson. Then, as we create the single platform, it is about how we start thinking about future R&D and looking forward towards a five-year horizon. It is about seeing the ideas that are coming that we can then bolt into the platform to take us to the next level. More specifically, with AI, we recently hired a senior leader from Intel to help us to bring the entire AI area to focus in terms of what it is going to do for our business. We are building AI and machine learning capabilities and technology in other areas of our company to look at how we transform all aspects of our business using AI and machine learning.
As I mentioned earlier, it is true that Pearson has hired a senior AI leader from intel. She reports to Mr. Bozik, not Mr. Hitchcock. As President of Global Product, Mr. Bozik’s part of the company makes product design decisions. Mr. Hitchcock’s group is in charge of the software development required to build the product. Both men report directly to Mr. Fallon. Ms. Edwards reports to Mr. Bozik. I don’t know what role Mr. Hitchcock may or may not have played in the new hire, but she definitely does not work for him. That is not at all clear from the interview.
Here’s another example:
High: With new technology and innovation, you underscored how the pace of change is faster than ever. Additionally, fostering learning agility for individuals, enterprises, and the people who work within companies has become an important topic especially given the fact that a skill that one has today may be rendered obsolete quickly. How do you work to stay agile and ensure that your organization is doing the same thing?
Hitchcock: We recently released a report with the Oxford Martin School on the future skills and learning that will be required in the workplace. One of the things we focused on is the aspect of employability and equipping students with the skills that are necessary for future employment and progression within employment. One of the areas I believe is a great opportunity for Pearson is to increasingly serve education, not just through school and formal education, but with employers to transform their workforce. When I connect to our suppliers in the technology industry or my fellow CIOs, one thing that is evident is that nearly every company is faced with the challenge of re-skilling their workforce. This must be done to cope with the demands of the digital revolution that we are going through and the impact of AI, robotics, and other emerging technologies. I believe Pearson is exceptionally well positioned to assist companies in transforming the skills they need to acquire and develop.
I started out as an engineer, and I became incorporated and then chartered through the Institute of Engineering and Technology. An engineer is no different than a surgeon or an accountant in the sense that the profession demands ongoing development certifications. I believe people want to continue the learning part. Because of this, the opportunity is there for us to become the delivery of lifelong learning for both individuals, education institutions, and government to achieve that. In terms of how we do that internally, at Pearson, there is a big focus on internal employee development.
We started the Technology Academy shortly after I joined to try and take the technology workforce to the next level in terms of their abilities to equip them with the skills needed for our own journey around the digital change. That is something we will continue to work on and sponsor. We work very closely with institutions and universities to do that as well as our own people. It is a fascinating journey and an incredibly important one for society over the next few years. As society changes, we need to evolve to support the future digital requirements.
One could be forgiven for assuming, based on this answer, that Pearson’s focus on workplace skills and continuing education was Mr. Hitchcock’s idea, which he arrived at on his own based on his personal biographical experience. But again, Mr. Bozik and Ms. Edwards have far more authority to set this kind of product strategy than Mr. Hitchcock does.
The net effect of Mr. Hitchcock’s answers is a real problem for Pearson’s brand. What is a typical academic’s nightmare vision of Pearson’s worst possible future? A company whose ideas of education come from its CTO; a man whose previous job was being CIO for a phone company, who thinks that Spotify and Netflix are positive examples of the company that Pearson wants to become and isn’t afraid to say so, repeatedly, in public, despite previous criticism for having done so.
Also? No mention of efficacy in the article. Anywhere. Not once. Not a word. Not a whisper. In an wide-ranging, open-ended interview about Pearson’s future.
Pearson’s senior executives tried their best to clean up the mess
As I mentioned earlier, it’s hard to figure out Pearson’s official position on…well…anything. But Pearson’s official position, or what the organization collectively “thinks,” is a critical question in this case. Given that the interview of somebody who directly reports to the CEO, and who repeated (and arguably expanded upon) an analogy for which the company had taken flak, the question is whether the root of the problem is a failure of corporate communication or a failure of alignment of senior management. This is a high-stakes question. And it’s one that can only be answered definitively by Pearson’s most senior executives.
Given that, I wanted to get as close to a definitive answer as possible. So I did something that we don’t typically do at e-Literate. I went through proper PR channels. There are a number of reasons we don’t like gathering information this way. One is that the people we are talking to are chaperoned and constrained to hew as closely as possible to the company line. When we approach people informally, they are freer to give us information off the record that provides further validation or nuance to their on-the-record comments. We also generally try to cultivate an approach to our analysis that doesn’t depend on executives continuing to grant us access because we believe that doing so minimizes one kind of pressure on our objectivity.
But for this particular interview, the constraints of the formal channel were helpful. I could already make pretty good inferences about what Ms. Edwards and Mr. Bozik likely think as individuals. I have gotten to know Ms. Edwards through the Empirical Educator Project and, while I haven’t had much direct exposure to Mr. Bozik, I know quite a bit about the work that has come out of his shop. Further, neither Ms. Edwards nor Mr. Bozik were responsible for—or could be individually responsible for—Mr. Hitchcock’s comments. As his peers, they were not in a position to make statements that were more definitive than his.
By going through the formal PR channel, I could speak not just to them but through them to the aggregate entity known as “Pearson.” I wanted Mr. Bozik and Ms. Edwards to have the kind of pre-interview huddle orchestrated by PR that occurs when these media requests are made, and I wanted them to be constrained by whatever had been decided in that huddle. By doing so, they could provide answers that are official in a way that they wouldn’t be through less formal channels. Their job in this kind of an interview is to give their best, most honest version of the official company line.
(By the way, I didn’t ask specifically for these two executives. Part of the experiment was to see who Pearson decided to send. They did not decide to send Mr. Hitchcock. I don’t know if that was purely due to scheduling conflicts or if there were other considerations.)
Let’s be clear: 90% of my interview with Mr. Bozik and Ms. Edwards was kabuki theater. It was highly ritualized and scripted, with moments of drama that were entirely predictable. That was by design and in no way their fault. They were constrained by the script. I even gave them my basic line of questioning in advance so that the “company” could formulate “its” answers in advance. My questions consisted of four parts:
Describe Pearson’s current view of what “efficacy” means, preferably covering specific examples of efficacy work
Clarify Mr. Hitchcock’s role in the company specifically as it pertains to decision-making around the efficacy initiative and its integration with product design
Review Mr. Hitchcock’s comments in Forbes, in the context of the first two parts of our discussion
Describe Pearson’s official position on whether and in what sense it aspires to be the Netflix, Spotify, or Amazon of education
The answers to the first two sets of questions were entirely predictable. I wanted to make sure that the official characterizations in the interview were consistent with what I thought I already knew. They were. Mr. Bozik and Ms. Edwards did their job. They faithfully described what I had come to understand as the company’s official positions on those questions. The third and fourth sets of questions were the test. In the third, Mr. Bozik did his best to provide the most charitable honest reading of Mr. Hitchcock’s comments that he could. Again, he did his job.
When I asked him the fourth question, using more or less the same words as above, he gave the the only correct answer: “Let’s be clear: Pearson does not aspire to be the Netflix or Spotify of education.” Those were the very first words out of his mouth. Yet again, he did his job.
At that point, I went off-script. I asked him how, given that statement, Mr. Hitchcock could use that analogy for a second time. In the entire ninety-minute interview, this was the only moment when Mr. Bozik seemed to struggle for words. As he should have. Because there is no good answer to that question.
I had put him in an impossible situation. He was bound by the rules of the kabuki play we were in to give answers that support the company line (and his colleague). He didn’t have the option to say, “Off the record, it was a dumb thing to say.” He had only two options he could take without breaking the rules of engagement: rationalize that which I believe he knew was wrong, or flail around for the best honest answer he could give.
Mr. Bozik chose to flail. I think better of him for it.
But their CEO blew it
At the end of the interview, I found out that Pearson was going to grant my request to speak directly to the company’s CEO. I honestly didn’t expect that request to be met. Pearson is at least several times as large as the next largest ed tech company that we cover. By some measures, they are an order of magnitude larger. So I was surprised and gratified that Mr. Fallon was willing to make time for the conversation.
And it was an important step for the specific purpose at hand. Mr. Fallon is the only person at Pearson who is empowered to fully repudiate statements by the man who reports directly to him. (This isn’t true at all companies, but it tends to be true at companies as large as Pearson.) Once again, I telegraphed my line of questioning in advance, which was essentially an abbreviated version of the same approach I had used with the previous interview.
To my surprise, Mr. Fallon jumped ahead to address Mr. Hitchcock’s comments directly. And to more or less defend them. He argued that a key constituency for Pearson is what he called the “Spotify generation,” or “Generation Z,” to which he attributed the following characteristics:
They would rather rent or subscribe than own
They have a higher expectation of the use of video
They probably expect to be able to engage with courses through shorter and more frequent modules
They have higher expectations of user experience
The first characteristic is arguable. Within the world of curricular materials, students are strongly price-sensitive. The primary mechanism that the industry has offered students to get a lower price is rental/subscription. Students have tended to rent or subscribe to curricular materials. Does this mean that they prefer rental or subscription, or just that they are picking the least expensive option? The publishers have a financial interest in promoting rental and subscription, but the evidence of a Spotify-style preference among students seems limited.
The other three characteristics are probably true and definitely not new. All three are trends that the sector has known about and anticipated for literally decades. Before there was the Spotify or Netflix of education, there was the iTunes playlist of education. And before that, there were re-usable learning objects. All of these points are sufficiently simple, accessible, and well understood already that an analogy to Spotify or Netflix adds nothing useful.
Mr. Fallon was intent on getting me to admit that there is a reasonable interpretation of the analogy. Which I did. But his defense missed the point entirely. Sure, in principle, with some thought, one can find a non-offensive explanation for an analogy to Spotify or Netflix. But why would you bother to make such an analogy in the first place? And for heaven’s sake, why would you do so in public when you know that your customers are likely to find it offensive by default?
When I pressed, Mr. Fallon said, “You have never heard or read me say that Pearson wants to be the Netflix or Spotify of education.”
That is a carefully parsed sentence.
In fairness, Mr. Fallon then went on to demonstrate to me that he knows exactly what the company’s official and nuanced 2018 version of efficacy is. He can cite it chapter and verse. He can refer to prior public statements that align with that vision, such as his recommendation of World Without Mind, a book about the dangers of romanticizing the power of artificial intelligence. He can speak at length and in detail about the challenges of the lack of agreed-upon frameworks for student data privacy in educational research.
Which is not surprising. Mr. Fallon is a smart guy. Furthermore, he was the one who bet Pearson’s future on efficacy. It’s probably not an exaggeration to say that he bet his career on it as well. It would be surprising if he didn’t know this stuff inside and out. I’m not sure why he didn’t just say the analogies were bad and do not represent Pearson’s position. Maybe he was just trying to convince me that this was a non-story and not worth writing about. Had he simply led with the points about the challenges of doing efficacy right and refrained from vigorously defending the indefensible, he might have succeeded. Instead, after all of the work involved in making the interviews happen, and all of Pearson’s executive time and PR time spent dealing with me, we’re right back where we started.
I don’t know Pearson’s official position on the analogy to Netflix or Spotify because Mr. Fallon—the only person at the company who is fully empowered to definitively lay the question to rest—chose to parse words and defend a bad interview rather than just saying, “Yeah, we don’t think about our goals that way and shouldn’t say things like that.” In a few sentences, Mr. Fallon could have closed the door on the one aspect of the Forbes article that I feel compelled to write about. But he didn’t.
To be clear, the aspect in question isn’t that Mr. Hitchcock said something dumb in public. And it’s not that Mr. Fallon was unwilling to admit on the record that Mr. Hitchcock said something dumb. It’s that Pearson, as represented by the behavior of its CEO, does not appear to fully recognize, or at least fully acknowledge, how delicate and existential a messaging challenge they’ve created by betting their future on efficacy.
Pearson still communicates like a roll-up, and it can’t afford to do so anymore
So here’s the real question: Why? Why has Pearson made these unforced errors and then, having made them, compounded them?
Once again, I can’t know the answer for certain. There often are some organizational politics behind stories like this one. That sort of thing is largely outside of our purview at e-Literate. But the company’s behavior may also be symptomatic of a more systemic problem. As I said earlier, Mr. Hitchcock was absolutely correct in saying that Pearson had a huge technology mess resulting from the fact that it is a “roll-up”—a big company that was created by buying up many small companies. He is also correct in saying that Pearson cannot compete unless they fix that mess by creating one seamless platform. (This is exactly where it is tempting to make analogies to giant internet companies and exactly where EdTech executives should resist that temptation.)
Pearson has had an analogous problem with their organizational culture (as have all the major textbook publishers). It used to be that every editor had his or her own fiefdom, run relatively independently from the others. This doesn’t work when you all have to deliver content via a common and complex technology platform and when you have to adhere to common standards of learning design. The company has been working on changing that culture for a while, has made some progress, and probably isn’t finished yet.
But there’s a third leg to this stool which appears to be the one that Pearson is falling over: message discipline (which is icky marketing-speak for “stick to talking about the things you think are important to communicate and don’t go off on tangents about things that are less important”). Even just a few years ago, it would have been impossible to describe the company’s official positions or priorities on much because they were a conglomerate. What priorities do textbook editors share with the publishers of the Financial Times and the people who run Penguin Books? There has always been a certain amount of laissez faire chaos in Pearson’s communications, for understandable reasons. But the company has been selling off all the parts of the business that don’t fit under the common theme of educational impact. I don’t think it would be an exaggeration to say that Pearson aspires to sell only one thing, and that thing is efficacy.
And yet, they don’t communicate that way. They still communicate like a company with lots of products and lots of messages for lots of different audiences. They have not made this part of their transformation and, based on what what I have seen during the course of writing this story, I’m not confident that they even know that they need to make this change. I have no reason to believe this is the fault of Pearson’s marketing and communications people. It’s much more likely that the problem is the prioritization that is being given to them, or not being given to them, from the top.
Nobody yet knows if a large education vendor can succeed by making efficacy its main product. But I can say one thing with confidence: They won’t succeed if they don’t sell it. And right now, they are not selling it. They say some things about efficacy, but they say some things about a lot of things, like Netflix, and Spotify, and Microsoft Hololens, and jobs of the future, and probably lots of other stuff that Gmail mercifully filters out for me. If they want their monumental bet to have a real chance of paying off, then they need to be talking about efficacy. Only efficacy. Always efficacy. Everything else needs to be subordinated to a very specific, meticulously crafted, always-being-tuned message about efficacy. Anything that outright clashes or distracts from that message should be mercilessly cut. Any missteps should be immediately and definitively corrected. Saying something stupid in public is a recoverable sin for a company. Failing to relentlessly focus on communicating the complex and somewhat controversial value proposition, upon which you have bet the entire future of your company, to an academic audience that is not terribly inclined to listen to or trust your messages in the first place? That may not be a recoverable mistake.
