Can Pearson Sell Efficacy?

Almost five years ago, when Pearson announced that the company would reorganize itself around efficacy, I was impressed by the degree to which the company was going “all in” on a very much unproven strategy. It was incredibly bold. I wrote,

In all my years of covering the ed tech industry, I have never seen a company be so explicit and detailed about their strategy as Pearson is being now with their efficacy publications. Yes, there is plenty of marketing speak here. But there is also quite a bit about what they are actually doing as a company internally—details about pilots and quality reviews and hiring processes and M&A criteria. These are the gears that make a company go. The changes that Pearson is making in these areas are the best clues we can possibly have as to what the company really means when they say that they want efficacy to be at the core of their business going forward. And they have published this information for all the world to see.

These now-public details suggest a hugely ambitious change effort within the company. Phil and I have consulted for a few textbook publishers, including Pearson, and I worked for Cengage for a year and a half. We have a pretty good idea of the magnitude of the change management challenges these companies face right now and the strategies that various publishers are bringing to bear in an effort to meet them. I can say with absolute conviction that what Pearson has announced is no half-hearted attempt or PR window dressing, and I can say with equal conviction that what they are attempting will be enormously difficult to pull off. They are not screwing around. Whatever happens going forward, Pearson is likely to be a business school case study for the ages.

Pearson bet the farm on efficacy. As far as I can tell, they are still betting the farm on efficacy. If anything, they have doubled down in the five years since I wrote those words.

There are a number of reasons why this bet is remarkable, not least of which is that we still don’t know if a curricular materials company can be successful in the long run by promoting efficacy as a primary value proposition. In addition to all the hard work that Pearson needs to do to credibly claim that their products support some sensible definition of efficacy, they also have to tackle the equally hard challenge of convincing faculty that the company’s vision of efficacy, as delivered in their products, is a good reason to pick their product instead of the (many) alternatives. They have an enormous customer communications challenge. That was one of the main points of my original 7,000-word post on the company’s strategy.

Which is why I find it mystifying to see an article in Forbes which seems almost designed to distract from or even directly undermine the efficacy message that the company has been carefully honing over the last half a decade. The article, “How 174 Year Old Pearson Is Developing The Netflix Of Education” is a longish interview with Albert Hitchcock, Pearson’s Chief Operating Officer and Chief Technology Officer. The jarring title refers back to Mr. Hitchcock’s previous use of the analogy in an interview entitled ‘Pearson aims to become the ‘Netflix’ of education.‘ In responding to that first piece two years ago, I wrote,

Given his profile on LinkedIn, Mr. Hitchcock appears to be new to education except for whatever memories he has of his own days as a student. Let me offer a couple of suggestions on how to get along in education for him and the many vendor employees who are in a similar situation:

1. Never, ever, say that you want your company to be the Uber of education, the Airbnb of education, the Pokemon GO of education, or the [insert name of tech darling] of education unless you really enjoy being a recluse (or you are secretly a double agent for your employer’s direct competitor).
2. If you absolutely must say something like the above, then do not say you are the Netflix of education. Honestly, Netflix isn’t even great at being the Netflix of movies. The last time they recommended a movie that I actually wanted to watch was…uh…never.

There is a recurring cultural fantasy that “solving” the education “problem” consists of creating a customized playlist of little content bits. So really, more like the Spotify of education, if you want to play that game. This idea enrages educators because it trivializes what they do. Nobody who has taught believes that proper sequencing of content chunks is the hard part. (For a fully fleshed out prior example—or a “worked example” in teaching parlance—of how the sorts of comments Mr. Hitchcock have made typically play out in educational corporate branding over time, see my post-mortem on Udacity’s pivot away from higher education.)

In the more recent Forbes piece, the interviewer picks up on the Netflix analogy and asks Mr. Hitchcock about it. Not only does Mr. Hitchcock choose not to disavow the analogy; he expands on it by adding Spotify and Amazon. I’ll parse his exact language later in this post in the interest of fairness, but my point from the previous post stands. Regardless of the intention behind the analogies, they are toxic and should be avoided at all costs. This is doubly true for Pearson because, as I will explain in the next section of this post, they also run directly counter to the more credible, more refined articulation of the efficacy strategy that the company is promoting in 2018.

The fact that the analogies did make it into an interview with a mainstream publication like Forbes raises some larger questions. Is Pearson less committed to efficacy than I have believed them to be? Is there poor alignment in the company around what efficacy means? Are they just really bad at messaging? Did Mr. Hitchcock’s intent somehow get represented unfairly by a few poorly chosen words that were then taken out of context by an editor?

To find out the answers to these questions, I spoke with Pearson’s President of Global Product Tim Bozik, SVP of Efficacy and Research Kate Edwards, and CEO John Fallon. Here’s the short version of what I have concluded, based on those interviews and other evidence:

1. I am mostly still convinced that Pearson is fully committed to efficacy as a primary value proposition for all of their products and services going forward.
2. I do believe that the Forbes interviewer was bad, as were some of the editorial choices made by the publication. (Particularly the headline.)
3.  Even granting the benefit of the doubt on the previous point, Mr. Hitchcock made a number of very bad choices that no executive in his position should make and that cannot be explained away by blaming the editor or the interviewer.
4. Mr. Bozik and Ms. Edwards seemed fully aligned on what Pearson means by efficacy, how to talk about it, and how central it is to the company. Within the rules that they were bound to follow in an on-the-record interview through formal PR channels, they did the best job they could to articulate the most attractive and compelling version of Pearson’s efficacy strategy while diplomatically establishing some distance from Mr. Hitchcock’s questionable analogies.
5. Mr. Fallon chose a different path. He went to some lengths to justify or explain away the analogies. In doing so, he re-opened questions that Ms. Edwards and Mr. Bozik had all but closed for me regarding whether Pearson has the sensitivity and message discipline they will need to earn their customers’ trust regarding their commitment to efficacy.

For the long version, read on.

But before you do, you should be aware of our conflicts of interest, which I’m going to describe in more than the usual detail. Pearson is a current sponsor of the Empirical Educator Project. In addition, they have periodically engaged us in consulting projects over the years (although we do not have any current engagements with them.) Some of these projects have involved their efficacy work, either directly or indirectly. We received prior permission from the company to blog publicly about the first such engagement. Most recently, we were hired to review the company’s public  efficacy reports, provide the kind of feedback that we might publish in a blog post, and offer suggestions for improving the work. You can judge for yourself whether these engagements make us more or less trustworthy in our assessments.

Netflix and Spotify analogies are fundamentally incompatible with Pearson’s view of efficacy

In order to understand why these analogies are particularly bad for Pearson’s efficacy effort, it’s important to understand how the company’s position on efficacy has evolved. In that original post five years ago, I wrote,

Let’s think some more about the analogy to efficacy in health care. Suppose Pfizer declared that they were going to define the standards by which efficacy in medicine would be measured. They would conduct internal research, cross-reference it with external research, come up with a rating system for the research, and define what it means for medicines to be effective. They would then apply those standards to their own medicines. And, after all is said and done, they would share their system with physicians and university researchers in the hopes that the medical community might be reassured about the quality of Pfizer’s products and maybe even contribute some ideas to the framework around the edges. How confident would we be that what Pfizer delivers would consistently be in the objective best interest of improving health? This is not entirely hypothetical; much of the drug research that happens today is sponsored by drug companies. Unsurprisingly, this state of affairs is viewed by many as deeply problematic, to say the least. It certainly doesn’t help the brand value of Pfizer. But at least much of that medical research is conducted by physicians and academic researchers and is subject to the scientific peer review process. Pearson is creating their framework largely on their own, selectively inviting in external participants here and there.

I get why they had to do this. The company is bleeding money, it will take some time to stop the flow of blood, and they couldn’t wait to build consensus before they tied the tourniquet. But it is not going to get them where they want to go. While there are obvious concerns about ethics and about whether such a company-driven approach is fundamentally compatible with progress on complex questions such as defining what an education is good for and how we know when we have achieved these ends, I want to focus on the business aspects of the problem. I want to focus on why continuing down this path is bad for Pearson. Or rather, why driving hard toward becoming facilitators rather than owners of efficacy research is good for Pearson.

I could give a number of examples, but one should hopefully suffice. In preparing to write this post, I asked Annie Cellini, Pearson’s Senior Vice President of Marketing and Strategy, whether Pearson intends to share the completed rubrics for their products with customers and prospects. This was her reply:

Though we don’t plan to share product efficacy scoring as part of our sales and marketing materials per se, where a product has a strong research and evidence base, we will communicate that to customers. It’s also worth saying that the most important output of an efficacy review isn’t a rubric score. We believe that much of a review’s value comes from the conversations that it prompts teams to have, which focus on the path forward, and on how to improve the product or service from a learner perspective. A poor score does not mean the product doesn’t work well. It often means that teams are not collecting the type of data needed in order to get a sufficiently robust view of the product’s efficacy, or they may not have a sufficiently practical plan to continuously enhance the product based on data. Their improvement plan will encourage them to start gathering new information, to start working in new ways, and to make sure that their customers are aligned with the outcomes they plan to achieve and understand their role in the product’s path to efficacy.

This is a perfectly sensible and responsible reply if you believe that the main value of the Efficacy Framework to customers is in the data that results from the work a product team does after an efficacy review. But remember, the magic of the rubric is in the norming conversations. Annie’s reply suggests that Pearson understands this in terms of the Pearson-internal processes but not yet in terms of their relationships with their customers. If Pearson were to say to faculty, “Here’s what we think we know about the efficacy of this product, here’s what we don’t know yet, and here is how we are thinking about the question,” they might get a number of responses. Maybe they would get, “Oh, well here’s how I know that it’s effective with my class.” Or “The reason that you don’t have a good answer on effectiveness yet is that your rubric doesn’t provide a way to capture the educational value that your product delivers for my students.” Or “I don’t use this product because it has direct educational effectiveness. It frees me up from some grunt work so that I can conduct activities with the class that have educational impact.” Most of all, if you’re John Fallon, you really want faculty to say to their sales reps, “Huh. I never thought about the product in quite those terms, and it makes me think a little differently about how I might use it going forward. What can you tell me about the effectiveness of this other product that I’m thinking about using, at least as Pearson sees it?” And you really want your sales reps to run back to the product teams, hair on fire, saying “Quick! Tell me everything you know about the effectiveness of this product!”

Pearson won’t get that conversation by just publishing end results of their internal analysis when they have them, which means that they have a high risk of failing to align their products with the needs and desires of their market if they think about the relationship between their framework and their customers in that way. I don’t think Pearson fully gets that yet. While the authors of The Incomplete Guide frequently invoke terms like “community” and “leaders” in the document, they generally seem to mean the community and leaders within Pearson. The company’s efforts to reach out to the academic community for feedback and participation are generally framed as an extension of their efforts rather than the very heart of them. And yet, Pearson’s brightest possible future is not as a company that designs educationally effective products, but as one that facilitates conversation and research about efficacy within the broader academic community (and in so doing is able to design products that their customers agree are effective for important educational goals as determined by meaningful measures).

The Netflix, Spotify and, to a lesser extent, Amazon analogies all speak directly to the question of whether Pearson intends to define efficacy for educators or with educators. Netflix famously just deleted—not just removed, but deleted—all user reviews and switched from a five-star review system to a simple “thumbs-up/thumbs-down.” The subhead on the Vanity Fair article I linked to in the previous sentence is “A step closer to a wholly ‘because you watched’ world.” The implication is that Netflix trusts the algorithm more than the humans to evaluate quality. Equally famously, Apple CEO Tim Cook has highlighted the company’s decision to use human curators of music, in contrast to Spotify’s decision to rely completely on its algorithms:

We worry about the humanity being drained out of music, about it becoming a bits-and-bytes kind of world instead of the art and craft.

Whenever Pearson makes any mention of Netflix or Spotify—or Amazon, which is known for its recommendation engine as well—the company risks appearing to come down on the side of the algorithm over the human. Educators worry about the humanity being drained out of teaching, about it becoming a bits-and-bytes kind of world instead of the art and craft. In 2018, is that where Pearson has come down on their definition of efficacy?

Not as far as I can tell.

Pearson, in fact, recently made a high-profile hire away from Intel of artificial intelligence expert Milena Marinova. Here’s how she described her work in an interview with The Bookseller:

Right now, I am working on developing human-centric AI – this means making the learning experience better for students and teachers; enabling lifelong learning through more accessible and affordable products; and building better products and solutions using new technology.

What does “human-centric AI” mean in this context? Mr. Bozik commented on this directly in our interview. He described it as an effort to help teachers and students get a better view into their own learning. He said the company is placing a strong emphasis on early intervention and formative assessment, and providing “feedback and insights.” He also said,

Pearson strives for respect for both the ambition and the humility of [its efficacy strategy]. The ambition is that efficacy means outcomes. Full stop. It’s the potential to help people live better lives. That is our purpose. The humility part is that it’s hard. We don’t underestimate the difficulty of it. It starts with having an empathy for teachers and learners, rather than just teaching and learning.

For Ms. Edwards’ perspective, while I could quote her from my interview, I think you’ll get a better sense of her from her lightning talk at the Empirical Educator Project summit in February:

Here’s the part of her talk that jumps out as relevant to the question at hand:

How do we maximize the uniquely human attributes that educators—faculty—bring to the table, and combine them with all the productivity enhancements that come from things like data science, AI, and the improvements we’re seeing as the result of technology. All with the emphasis on helping students achieve outcomes that really matter to them.

Ms. Edwards then goes on to talk about the resources that Pearson either has already contributed or will contribute under a Creative Commons license. These include a set of rubrics for evaluating curricular materials (or course designs) based on a set of academically accepted and empirically verified learning science principles and a set of tools for instructors that want to conduct action research. It’s these contributions, these public actions, that speak the most persuasively to Phil and me when we evaluate a company’s intent. (For more on Ms. Edwards’ views and her characterization of Pearson’s efficacy work, you can read her recent posts on LinkedIn.)

Taken together with Pearson’s other work, such as their first publicly released efficacy reports on their products, the message conveyed by the actual work that I have seen from the company is clear: They aspire to define efficacy with educators and students. As I wrote earlier in this post, I have had opportunities to view this work from inside and out over the past five years. I honestly believe that the people working on the efficacy and product teams—Ms. Edwards’ and Mr. Bozik’s teams, respectively—buy into that goal and are actively working toward it.

Pearson’s intent doesn’t matter if nobody believes them

The problem is that intent of the people working at the company, level of commitment to that intent by the CEO and the small circle of people that make decisions within a company, and customer perception of intent are three different things. When I published my post the first time Mr. Hitchcock was quoted talking about Netflix, some of the people who worked on product design and efficacy at Pearson were quite angry with me. They felt that I had misrepresented Pearson’s position, because—and this is key—they also felt that Mr. Hitchcock’s comments were not representative of Pearson’s position. I replied that I wasn’t sure there was any way to determine what Pearson’s position is. Mr. Hitchcock is quite senior. He reports directly to the CEO. I could ask other executives for their own responses, but they wouldn’t be any more authoritative than Mr. Hitchcock’s. Unless I could speak directly to Mr. Fallon—which I didn’t believe would be possible at the time—I had no way of definitively determining which beliefs the elusive entity known as “Pearson” holds.

What I did know is how academia would receive the phrase “the Netflix of Education.” I knew this because Mr. Hitchcock was not the first person to use it, as was acknowledged by a Pearson employee in a 2016 EdSurge story entitled “Why We Don’t Need a ‘Netflix for Education.’” Its toxicity was widely known inside Pearson, at least within the product and efficacy groups.

Last month, five years into the massive effort to align the company around efficacy, the analogy shows up again, expanded upon, by the same senior executive. And this time, it’s not published in a specialty IT outlet. It’s in Forbes. Interviews at that level do not happen at big companies without being vetted. If you’re a senior executive at a company like Pearson, you go through formal media training, and you are often accompanied on interviews by a PR professional. These tend to be usually serious, orchestrated affairs. There are rules. How was it possible that this interview was approved, went through the proper channels, and went off the rails? Do Pearson executives see it as being as far off-message as I do? Is there conflict or confusion about the efficacy strategy at the highest levels of the organization that isn’t obvious to the casual observer?

Let’s take a look at what Mr. Hitchcock actually said and then turn to the executive on-the-record reactions to it.

Mr. Hitchcock’s comments were bad and cannot be explained away by bad editing

Here’s the part of the article where Netflix came up:

High: You described the vision of delivering Pearson’s education, content, and services through a single platform as creating the “Netflix of Education.” Could you talk about this long-term vision?

Hitchcock: The intention of the message was to have the viewpoint that we needed to move to a platform type of model where we have all our products, services, and capabilities that we deliver to our customers in a single ecosystem. A great deal has been written around this model at Pearson, and that is especially relevant as our company grew through acquisitions. Ours is a diverse business that is over 170 years old and has had many different types of companies under the umbrella of Pearson for many years. Recently, education has become the primary focus, but that complexity that was acquired over those decades was inherent within the company. The way we serve our customers had been across many different brands and many different types of digital products. We sell millions of books and material on the digital side. We had to figure out how to transform that to be a consistent, high-quality branding experience and one that is comparable to the best customer experiences out there.

Silicon Valley companies create the benchmark for the digital experience by being platform businesses. Our vision is to leverage the opportunity to transform along similar lines in terms of having a single platform globally that could deliver all our educational content and courseware. Furthermore, this would allow us to move into a more personalized experience that delivers high-quality education outcomes. It would be game-changing for not only Pearson, but for the entire industry if we could create that single platform, similar to Netflix, Spotify, and Amazon. [Emphasis added.] This platform would be highly scalable, global in nature, high-quality, and a platform that could deliver all our experiences around the world to millions of learners. We are currently working to deliver high-quality courses to students that are proven to help them learn and progress their lives through education and ultimately to their professional lives.

In the interest of fairness, I’ll enumerate the extenuating circumstances:

• The interviewer, not Mr. Hitchcock, brought up Netflix. And a Forbes editor, not Mr. Hitchcock, chose to feature Netflix in the headline.
• Mr. Hitchcock seems to be making a point about the need to support a seamless, end-to-end, born-digital customer experience for students. Further, he is absolutely correct that Pearson, like all of the larger textbook publishers, grew through acquisition and have had a patchwork of many different systems that were never designed to work together. These are entirely valid and important issues for Pearson’s CTO and COO to discuss.
• He does not take the last, most toxic step (this time) of describing Pearson as the “Netflix of education.” In fact, he seems to make a modest effort to narrow the scope of the analogy in response to the question: “The intention of the message was to have the viewpoint….”

Is it possible that Mr. Hitchcock meant something innocuous and unobjectionable by the analogy? Yes, it is. Is it possible that he honestly tried, in his own way, to communicate the narrowness of his intent with the analogy? Absolutely. Is this a relatively minor and understandable slip that would have been fine if it had just been worded a little more clearly?

Absolutely not.

I reiterate: There are no circumstances under which any EdTech executive should ever make an analogy between their company or offering and Netflix or Spotify. Period. Full Stop. It is not a mistake that anyone in Mr. Hitchcock’s position should ever make. In fact, if the analogy is brought up by an interviewer, as it was in this case, the first words out of the executive’s response should be “To be clear, Pearson does not aspire to be the Netflix of education.” That goes doubly for Mr. Hitchcock because he actually said that once before and should have taken the opportunity to correct the record, and trebly because the analogy can easily be interpreted as being in direct contradiction to the company’s bet-the-farm strategy, around which it has taken them five years to arrive at their current calibration in message and in action.

Nor can this easily be chocked up to bad editing. In fact, I question whether there was any significant editing whatsoever. This isn’t one of those tightly written articles in the reporter’s own voice with a one- or two-sentence quote sprinkled in here or there. This is an interview, with short questions followed by long answers that do not appear to be heavily edited for clarity or brevity.

Even worse, there is a pattern to Mr. Hitchcock’s answers which leave the distinct impression that he, the CTO and COO, is directly responsible for or taking credit for the company’s strategy on teaching and learning. He deftly pairs “I believe” statements with “Pearson has done” statements. It’s very easy to read these statements as Mr. Hitchcock making a causal connection between his beliefs and Pearson’s strategy decisions, which is an impression that is only reinforced by the sycophantic fawning of the interviewer.

Here’s an example:

High: How is the process of experimentation around new technologies done? Specifically, as new technologies emerge, how do you encourage your team to begin to experiment and to decipher more of the specifics of how that might be leveraged?

Hitchcock: […] Overall, I believe it is a combination of aspects. I believe it is about us looking at other ideas that other companies are implementing and seeing where we can bring these concepts together to deliver with the lasting initiatives that we are trying to put in place at Pearson. Then, as we create the single platform, it is about how we start thinking about future R&D and looking forward towards a five-year horizon. It is about seeing the ideas that are coming that we can then bolt into the platform to take us to the next level. More specifically, with AI, we recently hired a senior leader from Intel to help us to bring the entire AI area to focus in terms of what it is going to do for our business. We are building AI and machine learning capabilities and technology in other areas of our company to look at how we transform all aspects of our business using AI and machine learning.

As I mentioned earlier, it is true that Pearson has hired a senior AI leader from intel. She reports to Mr. Bozik, not Mr. Hitchcock. As President of Global Product, Mr. Bozik’s part of the company makes product design decisions. Mr. Hitchcock’s group is in charge of the software development required to build the product. Both men report directly to Mr. Fallon. Ms. Edwards reports to Mr. Bozik. I don’t know what role Mr. Hitchcock may or may not have played in the new hire, but she definitely does not work for him. That is not at all clear from the interview.

Here’s another example:

High: With new technology and innovation, you underscored how the pace of change is faster than ever. Additionally, fostering learning agility for individuals, enterprises, and the people who work within companies has become an important topic especially given the fact that a skill that one has today may be rendered obsolete quickly. How do you work to stay agile and ensure that your organization is doing the same thing?

Hitchcock: We recently released a report with the Oxford Martin School on the future skills and learning that will be required in the workplace. One of the things we focused on is the aspect of employability and equipping students with the skills that are necessary for future employment and progression within employment. One of the areas I believe is a great opportunity for Pearson is to increasingly serve education, not just through school and formal education, but with employers to transform their workforce. When I connect to our suppliers in the technology industry or my fellow CIOs, one thing that is evident is that nearly every company is faced with the challenge of re-skilling their workforce. This must be done to cope with the demands of the digital revolution that we are going through and the impact of AI, robotics, and other emerging technologies. I believe Pearson is exceptionally well positioned to assist companies in transforming the skills they need to acquire and develop.

I started out as an engineer, and I became incorporated and then chartered through the Institute of Engineering and Technology. An engineer is no different than a surgeon or an accountant in the sense that the profession demands ongoing development certifications. I believe people want to continue the learning part. Because of this, the opportunity is there for us to become the delivery of lifelong learning for both individuals, education institutions, and government to achieve that. In terms of how we do that internally, at Pearson, there is a big focus on internal employee development.

We started the Technology Academy shortly after I joined to try and take the technology workforce to the next level in terms of their abilities to equip them with the skills needed for our own journey around the digital change. That is something we will continue to work on and sponsor. We work very closely with institutions and universities to do that as well as our own people. It is a fascinating journey and an incredibly important one for society over the next few years. As society changes, we need to evolve to support the future digital requirements.

One could be forgiven for assuming, based on this answer, that Pearson’s focus on workplace skills and continuing education was Mr. Hitchcock’s idea, which he arrived at on his own based on his personal biographical experience. But again, Mr. Bozik and Ms. Edwards have far more authority to set this kind of product strategy than Mr. Hitchcock does.

The net effect of Mr. Hitchcock’s answers is a real problem for Pearson’s brand. What is a typical academic’s nightmare vision of Pearson’s worst possible future? A company whose ideas of education come from its CTO; a man whose previous job was being CIO for a phone company, who thinks that Spotify and Netflix are positive examples of the company that Pearson wants to become and isn’t afraid to say so, repeatedly, in public, despite previous criticism for having done so.

Also? No mention of efficacy in the article. Anywhere. Not once. Not a word. Not a whisper. In an wide-ranging, open-ended interview about Pearson’s future.

Pearson’s senior executives tried their best to clean up the mess

As I mentioned earlier, it’s hard to figure out Pearson’s official position on…well…anything. But Pearson’s official position, or what the organization collectively “thinks,” is a critical question in this case. Given that the interview of somebody who directly reports to the CEO, and who repeated (and arguably expanded upon) an analogy for which the company had taken flak, the question is whether the root of the problem is a failure of corporate communication or a failure of alignment of senior management. This is a high-stakes question. And it’s one that can only be answered definitively by Pearson’s most senior executives.

Given that, I wanted to get as close to a definitive answer as possible. So I did something that we don’t typically do at e-Literate. I went through proper PR channels. There are a number of reasons we don’t like gathering information this way. One is that the people we are talking to are chaperoned and constrained to hew as closely as possible to the company line. When we approach people informally, they are freer to give us information off the record that provides further validation or nuance to their on-the-record comments. We also generally try to cultivate an approach to our analysis that doesn’t depend on executives continuing to grant us access because we believe that doing so minimizes one kind of pressure on our objectivity.

But for this particular interview, the constraints of the formal channel were helpful. I could already make pretty good inferences about what Ms. Edwards and Mr. Bozik likely think as individuals. I have gotten to know Ms. Edwards through the Empirical Educator Project and, while I haven’t had much direct exposure to Mr. Bozik, I know quite a bit about the work that has come out of his shop. Further, neither Ms. Edwards nor Mr. Bozik were responsible for—or could be individually responsible for—Mr. Hitchcock’s comments. As his peers, they were not in a position to make statements that were more definitive than his.

By going through the formal PR channel, I could speak not just to them but through them to the aggregate entity known as “Pearson.” I wanted Mr. Bozik and Ms. Edwards to have the kind of pre-interview huddle orchestrated by PR that occurs when these media requests are made, and I wanted them to be constrained by whatever had been decided in that huddle. By doing so, they could provide answers that are official in a way that they wouldn’t be through less formal channels. Their job in this kind of an interview is to give their best, most honest version of the official company line.

(By the way, I didn’t ask specifically for these two executives. Part of the experiment was to see who Pearson decided to send. They did not decide to send Mr. Hitchcock. I don’t know if that was purely due to scheduling conflicts or if there were other considerations.)

Let’s be clear: 90% of my interview with Mr. Bozik and Ms. Edwards was kabuki theater. It was highly ritualized and scripted, with moments of drama that were entirely predictable. That was by design and in no way their fault. They were constrained by the script. I even gave them my basic line of questioning in advance so that the “company” could formulate “its” answers in advance. My questions consisted of four parts:

1. Describe Pearson’s current view of what “efficacy” means, preferably covering specific examples of efficacy work
2. Clarify Mr. Hitchcock’s role in the company specifically as it pertains to decision-making around the efficacy initiative and its integration with product design
3. Review Mr. Hitchcock’s comments in Forbes, in the context of the first two parts of our discussion
4. Describe Pearson’s official position on whether and in what sense it aspires to be the Netflix, Spotify, or Amazon of education

The answers to the first two sets of questions were entirely predictable. I wanted to make sure that the official characterizations in the interview were consistent with what I thought I already knew. They were. Mr. Bozik and Ms. Edwards did their job. They faithfully described what I had come to understand as the company’s official positions on those questions. The third and fourth sets of questions were the test. In the third, Mr. Bozik did his best to provide the most charitable honest reading of Mr. Hitchcock’s comments that he could. Again, he did his job.

When I asked him the fourth question, using more or less the same words as above, he gave the the only correct answer: “Let’s be clear: Pearson does not aspire to be the Netflix or Spotify of education.” Those were the very first words out of his mouth. Yet again, he did his job.

At that point, I went off-script. I asked him how, given that statement, Mr. Hitchcock could use that analogy for a second time. In the entire ninety-minute interview, this was the only moment when Mr. Bozik seemed to struggle for words. As he should have. Because there is no good answer to that question.

I had put him in an impossible situation. He was bound by the rules of the kabuki play we were in to give answers that support the company line (and his colleague). He didn’t have the option to say, “Off the record, it was a dumb thing to say.” He had only two options he could take without breaking the rules of engagement: rationalize that which I believe he knew was wrong, or flail around for the best honest answer he could give.

Mr. Bozik chose to flail. I think better of him for it.

But their CEO blew it

At the end of the interview, I found out that Pearson was going to grant my request to speak directly to the company’s CEO. I honestly didn’t expect that request to be met. Pearson is at least several times as large as the next largest ed tech company that we cover. By some measures, they are an order of magnitude larger. So I was surprised and gratified that Mr. Fallon was willing to make time for the conversation.

And it was an important step for the specific purpose at hand. Mr. Fallon is the only person at Pearson who is empowered to fully repudiate statements by the man who reports directly to him. (This isn’t true at all companies, but it tends to be true at companies as large as Pearson.) Once again, I telegraphed my line of questioning in advance, which was essentially an abbreviated version of the same approach I had used with the previous interview.

To my surprise, Mr. Fallon jumped ahead to address Mr. Hitchcock’s comments directly. And to more or less defend them. He argued that a key constituency for Pearson is what he called the “Spotify generation,” or “Generation Z,” to which he attributed the following characteristics:

• They would rather rent or subscribe than own
• They have a higher expectation of the use of video
• They probably expect to be able to engage with courses through shorter and more frequent modules
• They have higher expectations of user experience

The first characteristic is arguable. Within the world of curricular materials, students are strongly price-sensitive. The primary mechanism that the industry has offered students to get a lower price is rental/subscription. Students have tended to rent or subscribe to curricular materials. Does this mean that they prefer rental or subscription, or just that they are picking the least expensive option? The publishers have a financial interest in promoting rental and subscription, but the evidence of a Spotify-style preference among students seems limited.

The other three characteristics are probably true and definitely not new. All three are trends that the sector has known about and anticipated for literally decades. Before there was the Spotify or Netflix of education, there was the iTunes playlist of education. And before that, there were re-usable learning objects. All of these points are sufficiently simple, accessible, and well understood already that an analogy to Spotify or Netflix adds nothing useful.

Mr. Fallon was intent on getting me to admit that there is a reasonable interpretation of the analogy. Which I did. But his defense missed the point entirely. Sure, in principle, with some thought, one can find a non-offensive explanation for an analogy to Spotify or Netflix. But why would you bother to make such an analogy in the first place? And for heaven’s sake, why would you do so in public when you know that your customers are likely to find it offensive by default?

When I pressed, Mr. Fallon said, “You have never heard or read me say that Pearson wants to be the Netflix or Spotify of education.”

That is a carefully parsed sentence.

In fairness, Mr. Fallon then went on to demonstrate to me that he knows exactly what the company’s official and nuanced 2018 version of efficacy is. He can cite it chapter and verse. He can refer to prior public statements that align with that vision, such as his recommendation of World Without Mind, a book about the dangers of romanticizing the power of artificial intelligence. He can speak at length and in detail about the challenges of the lack of agreed-upon frameworks for student data privacy in educational research.

Which is not surprising. Mr. Fallon is a smart guy. Furthermore, he was the one who bet Pearson’s future on efficacy. It’s probably not an exaggeration to say that he bet his career on it as well. It would be surprising if he didn’t know this stuff inside and out. I’m not sure why he didn’t just say the analogies were bad and do not represent Pearson’s position. Maybe he was just trying to convince me that this was a non-story and not worth writing about. Had he simply led with the points about the challenges of doing efficacy right and refrained from vigorously defending the indefensible, he might have succeeded. Instead, after all of the work involved in making the interviews happen, and all of Pearson’s executive time and PR time spent dealing with me, we’re right back where we started.

I don’t know Pearson’s official position on the analogy to Netflix or Spotify because Mr. Fallon—the only person at the company who is fully empowered to definitively lay the question to rest—chose to parse words and defend a bad interview rather than just saying, “Yeah, we don’t think about our goals that way and shouldn’t say things like that.” In a few sentences, Mr. Fallon could have closed the door on the one aspect of the Forbes article that I feel compelled to write about. But he didn’t.

To be clear, the aspect in question isn’t that Mr. Hitchcock said something dumb in public. And it’s not that Mr. Fallon was unwilling to admit on the record that Mr. Hitchcock said something dumb. It’s that Pearson, as represented by the behavior of its CEO, does not appear to fully recognize, or at least fully acknowledge, how delicate and existential a messaging challenge they’ve created by betting their future on efficacy.

Pearson still communicates like a roll-up, and it can’t afford to do so anymore

So here’s the real question: Why? Why has Pearson made these unforced errors and then, having made them, compounded them?

Once again, I can’t know the answer for certain. There often are some organizational politics behind stories like this one. That sort of thing is largely outside of our purview at e-Literate. But the company’s behavior may also be symptomatic of a more systemic problem. As I said earlier, Mr. Hitchcock was absolutely correct in saying that Pearson had a huge technology mess resulting from the fact that it is a “roll-up”—a big company that was created by buying up many small companies. He is also correct in saying that Pearson cannot compete unless they fix that mess by creating one seamless platform. (This is exactly where it is tempting to make analogies to giant internet companies and exactly where EdTech executives should resist that temptation.)

Pearson has had an analogous problem with their organizational culture (as have all the major textbook publishers). It used to be that every editor had his or her own fiefdom, run relatively independently from the others. This doesn’t work when you all have to deliver content via a common and complex technology platform and when you have to adhere to common standards of learning design. The company has been working on changing that culture for a while, has made some progress, and probably isn’t finished yet.

But there’s a third leg to this stool which appears to be the one that Pearson is falling over: message discipline (which is icky marketing-speak for “stick to talking about the things you think are important to communicate and don’t go off on tangents about things that are less important”). Even just a few years ago, it would have been impossible to describe the company’s official positions or priorities on much because they were a conglomerate. What priorities do textbook editors share with the publishers of the Financial Times and the people who run Penguin Books? There has always been a certain amount of laissez faire chaos in Pearson’s communications, for understandable reasons. But the company has been selling off all the parts of the business that don’t fit under the common theme of educational impact. I don’t think it would be an exaggeration to say that Pearson aspires to sell only one thing, and that thing is efficacy.

And yet, they don’t communicate that way. They still communicate like a company with lots of products and lots of messages for lots of different audiences. They have not made this part of their transformation and, based on what what I have seen during the course of writing this story, I’m not confident that they even know that they need to make this change. I have no reason to believe this is the fault of Pearson’s marketing and communications people. It’s much more likely that the problem is the prioritization that is being given to them, or not being given to them, from the top.

Nobody yet knows if a large education vendor can succeed by making efficacy its main product. But I can say one thing with confidence: They won’t succeed if they don’t sell it. And right now, they are not selling it. They say some things about efficacy, but they say some things about a lot of things, like Netflix, and Spotify, and Microsoft Hololens, and jobs of the future, and probably lots of other stuff that Gmail mercifully filters out for me. If they want their monumental bet to have a real chance of paying off, then they need to be talking about efficacy. Only efficacy. Always efficacy. Everything else needs to be subordinated to a very specific, meticulously crafted, always-being-tuned message about efficacy. Anything that outright clashes or distracts from that message should be mercilessly cut. Any missteps should be immediately and definitively corrected. Saying something stupid in public is a recoverable sin for a company. Failing to relentlessly focus on communicating the complex and somewhat controversial value proposition, upon which you have bet the entire future of your company, to an academic audience that is not terribly inclined to listen to or trust your messages in the first place? That may not be a recoverable mistake.