Category: Academics & Academia

The “Academics and Academia” category covers topics related the ways in which colleges and universities function that are relevant to technology-supported education. One key aspect covered here is pedagogy—how people teach—and how technology impacts teaching and learning.

But this category also includes more institutional aspects that are relevant to technology-supported education, such as how campus leadership supports (or doesn’t support) new initiatives, politics and bureaucracy that impact these efforts, and so on.

Finally, “Academics and Academia” covers commercial and non-profit services that provide support for technology-supported education initiatives, such as Online Program Management (OPM) companies.

Toward Operational Excellence at Student Success: Double-Loop Learning
Before I move on to my next case study in academic institutions moving toward operational excellence at supporting student success, I want to revisit a section toward the end of my last post on the California Community Colleges Online Education Initiative (OEI). I was looking at the alignment that has to be achieved at various levels in the academic organization in order to encourage all the stakeholders to embrace this collective mission, with all the changes to their day-to-day work and even professional identities it would entail. Much of the piece is about the work that had been done so far to get alignment at various levels within the administration. But toward the end of the piece, I speculated a bit on potential opportunities for fostering faculty alignment through a course peer review process using a common rubric:

[T]o me, one of the most interesting vectors for culture-building is the course exchange course quality rubric. Every course on the exchange has to be evaluated against a rubric of evidence-backed effective online teaching practices. As the pace at which exchange courses are developed increases, OEI will not be able to keep up with demand to evaluate these courses using central staff. So they are creating a peer reviewer mechanism in which faculty on the campuses are trained on the rubric and presumably compensated to review courses that are candidates for the exchange.

This opportunity fascinates me. We know that faculty who go through an expert-supported course redesign process often experience intellectually deep and emotionally moving shifts in their teaching strategies. Is the same true when faculty are trained reviewers of their colleagues’ redesigned courses? What effect will simply exposing faculty to more and different course designs have? How will their role as reviewers and critiquers shape or enhance that effect? Can a continuously improved and updated rubric become a vector for sharing new research-supported processes across the system on an ongoing basis? Will the impact be broad and deep enough to foster new kinds of intra- and inter-campus faculty dialogs about the scholarship of teaching and learning (SoTL)? Will these cultural changes help to foster alignment around continuous operational improvement for enabling student success? This is the last mile problem of higher education. Operational excellence at student success cannot be achieved unless it is infused in the daily operations in individual classrooms. That requires affirmative faculty buy-in, support, training, and embedding in a culture that invites them into the larger conversation.

Unpacking this a bit, what does it really mean to build a culture of operational excellence in supporting student success? What kind of change would be necessary at the individual level to achieve change at the organizational level?

Organizational learning

There is a useful concept in organizational psychology called “double-loop learning.” I’ll give a simple example of a non-academic organization first to make the concept clear. Suppose your company manufactures smartphones. You want supply to match demand almost exactly as possible. If you manufacture too many phones, then you will sink expense into building units that will sit on the shelves and fairly quickly become obsolete. But if you manufacture too few, then you won’t have phones to sell at the moments that people need to buy them, thus encouraging them to buy a different (more available) phone instead. In a single-loop model, you have one lever to pull, which is how many phones you produce at a given time. It’s a like a thermostat: If the room is too cold, then turn on the furnace. If the room is warm enough, then turn off the furnace. If there are not enough phones on the shelves, then turn up production. If there are too many phones on the shelves, then turn down the production line.

The problem is that there’s a significant lag between when the order is given to produce more phones and when they arrive on the shelves. During that time period, demand can change. Maybe by the time the new phones the company produces during a period of high demand actually land on the shelves during the beginning of a recession, or right after a competitor releases their hot new model. The single-loop, thermostat-like model doesn’t work very well.

Of course, the people who run the company are smart enough to know this, so they come up with all sorts of work-arounds. They build warehouses to hold excess phones near where they are built, since holding onto the phones that way is cheaper than shipping them halfway across the world and negotiating with the retail stores that are selling them and may want to ship excess inventory back. They build sophisticated forecasting models that account for factors such as the economy and competitor behavior, so the chances of them being badly wrong are reduced. These are all work-arounds to a fundamental problem regarding the costliness of being wrong in your demand forecasts. And this is exactly the way manufacturers of all kinds of complex items, including smartphones, used to operate in the old days.

But then somebody somewhere questioned a fundamental premise that drove so much effort and activity: Does it have to take so long from the time the company order new products to be manufactured until those products reach the retail shelves? Maybe there’s some part that often holds up the whole product; if we could only use a different part, or make the part ourselves, then we could get rid of a lot of the delays. Maybe the places where those component parts come from are farther away from our factory than they need to be; if we could just get them to move closer, then we could cut down on the lags. Maybe we have extra steps in our manufacturing process, or use outdated equipment; if we could only make some updates, then we can shorten the lag. And maybe if we do all of these things, as well as more generally finding and making changes anyplace where the process bogs down, then maybe we don’t have to put products on shelves at all. Maybe we can get the delay between order and manufacture short enough that we could start manufacturing the device when the consumer orders it and get it assembled and shipped fast enough that the consumer would tolerate the delay.

This is double-loop learning. Organizations not only use processes that allow them to make adjustments but also regularly examine the assumptions behind those processes that may be unnecessarily getting in the way of achieving organizational goals. We assume that we have to develop processes to mitigate bad product demand forecasts because we assume that those costs will be high because, in turn, we assume that manufacturing the product will take a long time once we decide to do it. But what if we’re wrong?

Double-loop thinking is a reasonably simple concept to understand but very hard to execute well and consistently. In the smartphone manufacturer example, think about all the many kinds of assumptions in the way things had always been done that would have to be identified, questioned, and replaced with a better-designed alternative. Particularly in the early days, when there weren’t models to copy or lessons learned elsewhere, no one person who could see all the changes that would have to be made. There would be many people across the organization—in manufacturing, product design, contract negotiation, shipping, retail relations, and so on—who would each be able to spot an individual sub-optimization in her daily work experience. And then more people would have to be involved in designing a solution to each sub-optimization, including accounting for all the ripple effects across other aspects of the organization. It would be an all-hands-on-deck sort of affair. Everyone would be needed to find problems, identify potential solutions, check those solutions for side-effects, and then implement them well.

Double-loop learning in academia

Now think about a few of the many questions that are starting to be asked about the operating assumptions about the education-related processes of colleges and universities:
- Why must students stay in a course for a set number of weeks, regardless of how quickly or slowly they are capable of learning the material?
- Why are students only able to register for and start a course at most a couple of set times in the year?
- Why some very common teaching modalities based on the default assumption that all students learn roughly the same way and encounter roughly the same rough spots?
- Why do we define the minimum math literacy for a college degree as basic algebra rather than, say, statistics?
- We do we believe that professorial training requires at least five years of deep disciplinary education and at most one course in pedagogical education?
- Why do faculty gain job security through research excellence far more than through teaching excellence?
- Why do we assume that students know and understand everything they need to do from the moment they receive their college acceptance to the moment they arrive on campus for the start of their first semester of class?
- Why do we assume we can know when individual students are in trouble and need help from the academic institutions when no employee of that institution sees the student for more than a few hours in the week—at most—and there is no good mechanism for sharing concerns and observations among the people who have contact with that student?
Think about the people who were in a position to spot each of these assumptions. Think about all the people required to design, troubleshoot, and implement alternatives that arise out of questioning the assumptions. If we want to reliably create student-ready colleges, then we need to be able to identify many unwarranted assumptions and design many alternative ways of doing things in ways that will deeply affect the ways in which academic institutions—and the people employed by them—work. To change everything, you need everyone. That specifically includes faculty.

A rubric as a vector for change

Way back in late 2013, I wrote about Pearson using a rubric to try to catalyze this sort of broad-based organizational shift in (critical) thinking: ((Pearson is a sponsor of e-Literate’s Empirical Educator Project.))

So if you’re the CEO of major textbook publisher and you want to unite the entire 45,000-employee company around a plan to transform the way the company does business, what do you do? Surprisingly, Pearson’s CEO John Fallon’s answer was, “I’ll create a rubric.”

I’m not going to analyze Pearson’s rubric in detail here…. I’ll say this much about it: It’s nothing special. It’s not bad, but it’s not genius either. There are plenty of flaws and limitations you could find if you worked at it and applied it broadly enough. There is no magic in it.

But here’s the thing: There is neverany magic in a rubric. The magic, when there is any, happens from the norming conversations that the rubric engenders. It happens when one colleague says to another, “What do you mean by ‘quality of evidence’?” Or “I scored that course a 2 on effectiveness. Why did you think it was a 4?” To the degree that the Effectiveness Framework proves to have any magic for Pearson, it will be in the norming conversations that it engenders across the company. Like our hypothetical Berkeley president, Fallon is working with diverse groups within an institution that has a culture of independence and Balkanization. Some of this is for good reason; conversations about effectiveness in chemistry education should look very different from conversations about effectiveness in fine arts education. Some of the fractiousness is about lack of a common culture and language necessary to discuss what otherwise arecommon challenges. And some of it is just human territoriality and self-interest. The first two challenges might be addressed by having a deep and wide ongoing norming conversation about a rubric that is general enough to cover a wide range disciplines and products but focused enough to provoke important discussions. The goal is for that conversation to become the basis for a new culture. The third challenge might be addressed by reinforcing that culture through your HR and other business practices.

Since I wrote that post, Pearson has developed a set of rubrics for evaluating whether a given product supports research-backed learning design principles. They have rolled those rubrics out to every product team and trained their product teams on how to use them. They have released them under a Creative Commons license and are. (For more on both the resource itself and Pearson’s interest in working with academics to make them more useful to academia, see the talk given by Pearson’s Global Head of Efficacy and Reach at last year’s Empirical Educator Project summit.) So Pearson continues to use what is essentially an academic strategy, not that different from the one being rolled out by California OEI, to build a double-loop culture around designing educational content and software functionality that are more effective at impacting student outcomes.

The rubric development, training, and norming processes are necessary but not sufficient. As I suggest in that last sentence of the Pearson post quote, other organizational processes need to be put in place as well in order to get the desired effect. It would be easy to get faculty thinking that the new practices are baked into the rubric, and as long as everybody is aligned with them, you’re good. The organization goes through the double-loop, but only until the norming process is complete. This is, in fact, what happens in many colleges and universities that adopt course quality rubrics. The institution has to mindfully employ the rubric updating, retraining, and renorming processes as methods for collaborative innovation. The rubric needs to be designed at a high enough level that it invites discussion and thought rather than rote implementation. The processes around it need to be collaborative rather than broadcast-only. And many other processes—like compensation for time invested or rewards for innovation, to take a couple of obvious examples—need to be created or modified to support and dovetail with the rubric processes.

There are lots of organizations that implement course quality rubrics. Enough that we should be able to start gathering stories and effective practices for using them to foster continuous organizational improvement. If anyone has a good example, please let me know.
December 10, 2018
Coursera CEO Interview: Betting on OPM market and shift to low-cost masters degrees

In mid 2012 during the midst of MOOC mania, I wrote a post noting that we should pay attention to future generations of the concept and that there were four barriers that the MOOC vendors would have to overcome to have any long-lasting impact.

Given this short timeline and the nature of investment-backed educational experiments, I think the real focus should be on whether and how MOOCs or successor models build on current scalability and openness while overcoming these four barriers.

Six years later, it is becoming increasingly clear that the next-generation model for MOOCs in higher education is to become a form of Online Program Management (OPM) providers, including the near-term focus on master’s degrees. The OPM market has demonstrated revenue models (tuition revenue sharing mixed with fee-for-service), the end credential is the already-accepted degree, course completion rates are higher for paying and matriculated students, and degree programs have methods for student authentication. In other words, the MOOC-based OPM model is the next-generation designed to address these challenges.

The shift into the OPM market has been documented in a series of posts in July of 2017, March, April, and May of 2018; and from Dhawal Shah from Class Central . In the May e-Literate post:

The picture one gets is of a chaotic market that is not for the faint of heart, and one that will likely see further consolidations and category changes. 2U, for its part, has been successful partially due to a niche strategy where they go after elite master’s programs and mostly avoid direct competition or engagement with the rest of the market. And recently we have started to see the MOOC providers become OPM providers – where the primary revenue for Coursera and FutureLearn are based on revenue sharing with online programs, albeit with lower sharing rates and with very different marketing approaches. In other words, there seems to be several efforts to enter into the same OPM race, but if possible to avoid being in the mainline rev-share OPM market.

Last week Julia Stiglitz from GSV Advisors, in their first podcast episode, interviewed Coursera CEO Jeff Maggioncalda who joined the company summer 2017. This interview gives the clearest view yet of Coursera’s emerging business model, and by extension it helps explain the new subset of MOOC-based OPM that includes FutureLearn, edX, and Udacity as vendors. ((Outside of Georgia Tech legacy contract, Udacity has moved to corporate professional development market, which is a different approach to same problem.)) I think that the media narrative of tuition revenue-sharing vs. fee-for-service OPM models is overblown, especially since there is a spectrum in that respect more than a binary choice of OPM vendor types. What the MOOC-based OPM entry introduces is a more fundamental characteristic of how traditional institutions develop online programs – namely low-cost vs. full-cost online degrees.

The first note from the interview is that the Coursera of 2018 is not the Coursera of 2012. While Maggioncalda still shows aspects of that old-time MOOC belief system, his approaches are very much rooted in focusing Coursera on a solid business model. And the difference shows. The second note is that 2U’s success in the OPM market and a successful IPO had a big influence on Coursera’s shift. [Emphasis added in transcript]

Julia: You know when you first joined. You spent some time looking at Coursera’s strategy, and really digging in and looking at the different businesses that Coursera had, and one of them that you were particularly attracted to, and you have put increased attention on here at Coursera is the online degrees business. What was it about online degrees that excited you?

Jeff: Yeah. This is sort of I think another good example of what good entrepreneurs have to do, is you have to have feedback loops; you need to get information from multiple sources to understand the nature of a problem so that you can come up with solution. The nature of an opportunity so you can develop a strategy to go after it. It’s really actually pretty simple. I came in – you were on the team, too, we did a lot. We call them deep dives. We went all through the business model, and there’s a great book Business Model Generation that really, to me, gives a nice framework for saying this is what a business model is. It is a target customer. It is a value proposition and offering that solves their needs. It’s a set of channels of how you acquire those customers. It’s a servicing models of how you service the models. Internally it’s the key activities and resources you bring to bear on that. It’s the partners that you work with. It’s the financial revenues and costs, and is your competition. So and that’s the framework. And we stepped through every one of those. I wrote 250 questions across that business model that we as an executive team went through. You know step by step by step, so that everybody learned the nature of our business. And what became very obvious is we had a few things that nobody else really has.

We had 36 million learners, at the time it was 25 million. 25 million learners from all around the world. That’s a pretty big asset. We had university partners. Now there are competitors out there like LinkedIn Learning, previously Lynda, like PluralSight, like SkillSoft. You know there’s YouTube, there’s Khan Academy – there’s a lot of content out there. You were one of the ones who told me in one of those early meetings, “Hey we’re worried that content, generic content, might become a commodity.” Well, we don’t want to play a commodity game. So what is it about my partners that’s super distinctive? Well our partners are universities, and they’re not just the universities, they’re the best universities in the world, and they’re spread around the world. So you say, well I’ve got a resource that almost no one has, which is this network of universities. Right now they’re publishing MOOCs, and there’s something special about MOOCS, but MOOCs are a little more susceptible to that commoditization just as MOOCs. But what was not very susceptible to commoditization are degrees. So that’s OK. We have an asset nobody else has, and what they do really well is degrees, and they still have market sizing. How big is the market for degrees? 1.5 trillion dollars. Okay, well that’s a pretty big opportunity. And then you say, what’s the likelihood that that industry could be transformed due to technology . . . You know, some industries it’s easier to transform, others, it’s harder. The provision of education is absolutely set up nicely to be enhanced, transformed by technology.

I think Uber and Lyft were really smart when they said “you know on-demand transportation, called a taxi, it’s a big market, but it’s a broken product. And if we just do some sort of digital view of this kind of redesign what on-demand transportation looks like, it’ll be a much bigger market.” I’m looking at degrees, I’m not saying it’s broken altogether, but if you look at the student debt out there, you look at the the lack of access, and you look at how inconvenient it is for people to have to stop their lives – especially for master’s degrees – quit their job, move their family, pay hundreds of thousands of dollars, forfeit their income. That’s a broken product. So I thought we got partners who are really good, and a massive economic opportunity, and a product that is just ready to be dramatically improved by technology, and so I thought this is pretty good. We should go after this. By the way we also had 2U trading at like a 12 times forward multiple. So clearly Wall Street loved the idea of online degrees, and 2U’s been doing great. They’re growing really rapidly, so there’s a data point out that says, hey this company is doing really, really well; We should be able to do pretty well here, too.

Julia: Could you share a little bit about what this redesign looks like? Because the online degrees aren’t new. You know 2U is doing them. And before 2U there were a whole set of online degree providers that were out there, so why is what Coursera is doing different?

Jeff: Yeah I think it’s a few things. The number one, I would say, is quality. When I say quality, I mean the quality of the credential. So a lot of people have spent a lot of money on for-profit college degrees that just don’t have very good credential value, they’re not recognized in the job market. You pay a lot of money, you don’t get much back for it. One of the reasons that people pay so much for the top universities is that those types of degrees means something in the job market. There have been a lot of online degrees out there, from universities, that charge a lot and don’t get you very far. Our partners happen to be the best universities in the world, with the highest credential value in the world. When these degrees come online, and these degrees online are the same degrees as on campus, you’re getting something as a credential that’s extremely valuable. That should have a very high ROI. Because we’re doing it online the cost is often less than half. So it’s a top quality credential at half the price. Same credential you get on campus.

Different Assumptions on Tuition

There’s a lot of useful insight in the full interview, but I’d like to call out the fundamental question that gets raised about online education with this market view. Should online degrees from traditional universities cost the same as face-to-face offerings, or should they cost significantly less?

For full-service revenue-sharing segment of the OPM market, some core assumptions are built on the assumption of high revenue share percentages and full-priced online degrees. 2U is probably the best-known and arguably the most successful OPM company, and like Coursera they target elite institutions as partners. In 2U’s website under “Our Approach” they describe how their online programs typically charge the same as on-campus programs.

Most of the full-service revenue-sharing segment of the OPM market is similar in its view, whether from Pearson, Wiley, Academic Partnerships, or others – relying on consistent tuition as for online programs, and if there are lower prices they tend to be marginally lower. ((Disclosure: 2U and Pearson are sponsoring participants in our Empirical Educator Project.)) The Coursera approach is in direct contrast with this view, based on the interview as well as several of their online degree programs. There are arguments for either approach. With full-cost tuition, the idea is that the online degree gives at least as much value to students as the face-to-face, or on-campus, degree, and therefore students will be willing to pay the same. With low-cost tuition, the idea is that while students get the same value, “because we’re doing it online” the costs should necessarily be lower. Online infrastructure and marginal costs are much lower than investment in physical facilities. The point here is that this is a fundamentally different set of assumptions.

Writing about the Illinois $22,000 iMBA program, Marc Ethier described this different approach to pricing:

For most, the initial appeal of the program was certainly the price tag. Illinois’ iMBA costs a fraction of a degree from an elite school, where the median cost is roughly $171,000 and can break the $200,000 mark at the far end of the scale. Illinois’ own residential two-year MBA costs more than $100,000. Arshad Saiyed, executive director of online programs at the Gies College, acknowledges that the low cost brought the program to many prospective students’ attention — but says the iMBA has kept students around through a combination of high-quality instruction and successful community building.

Different Assumptions on Student Recruiting

For OPM full-service vendors, the largest expense is typically marketing and sales – i.e. recruiting potential qualified students. The predominant approach to OPM student recruitment has been based on digital marketing – advertising and outreach on social media platforms, search engine placement, digital advertisements in articles. With the MOOC-based OPM subset of the market, there is now an alternative approach based on having a multi-sided platform model. Coursera views their 36 million registered learners as an asset – a natural base of potential students for online degrees that can be reached without external advertising. In addition, the original aim and design of large-scale MOOCs is based on ability to easily sign up new learners for low- or no-cost, with the opportunity to move these students into higher-cost credentials and degrees over time, not requiring full financial commitments from students up front. While a Coursera or FutureLearn might use digital marketing for recruitment, that is not their primary method.

Different Assumptions on Course Size

Related to the above assumptions, in 2U’s case the class size is small – typically 10 – 20 students leveraging the platform’s design around small discussion groups, using both synchronous and asynchronous learning. This 2018 article about Washington University’s two programs partnering with 2U partially describes this approach.

But what is it like for student to pursue a graduate degree in law fully online? How could a pre-recorded lecture support the active teaching that’s integral to discipline? After all, watching a video isn’t the same as participating in a conversation. To support such engagement, 2U created a new tool.

“Through building an online LLM [master’s of law] program with Washington University in St. Louis, we learned how to design one of the most important tools we provide today: the bidirectional learning tool, or BLT,” said Chip Paucek, co-founder and CEO of 2U. “Socratic-style teaching is fundamental to all law curriculum and coursework. As such, it was imperative for us to design a way to conduct Socratic-style group discussions for Wash U once we signed their online LLM program.

“What we didn’t realize is that while we were developing a software tool to help solve the challenge of teaching the Socratic method online, we were simultaneously creating a way to facilitate discussion-based learning in an asynchronous environment that would eventually be used in all of our future partner programs.”

The approach that 2U and Wash U Law conceived relies upon the ingenious integration of asynchronous and synchronous course components. Instead of lecturing from a podium, faculty address small groups of student actors. At key points, the instructor breaks the fourth wall and addresses the online student, who is prompted to answer without the benefit of knowing how his or her peers have responded. In other words, students can’t piggyback like they might in an in-person class.

After responding, online students can review one another’s answers. They might be prompted to answer follow-up questions, or they might be asked to come to the next live class prepared to defend whatever position they’ve chosen. The preparatory work that might otherwise happen during an in-person class is accomplished in advance through the pre-recorded sessions, enabling faculty to make better use of live, synchronous time.

In contrast, consider a Class Central interview with Maggioncalda when talking about scaling and its challenges.

I think about systems. As the system gets bigger, where would the bottlenecks emerge? My sense is that the bottlenecks will emerge in live sessions and in grading. That’s my guess. The grading, I’m actually not so concerned about because I think the ability to automate grading at scale will become pretty good. The live sessions get tricky. From a technology perspective, I’m not that worried about it. It’s the professor’s time and attention. My thought is it’s going to be a little bit like pyramid, where the number of hours that the main professor puts in won’t really change. If you think about how medical systems have worked, a doctor is in the system, but the number of minutes and hours that a doctor spends [with each patient] becomes an increasingly smaller portion of the total time [during which medical treatment is being delivered]. I think it will probably be somewhat similar for education. The size of the classes could be big, let’s say 10,000. But that will be broken into sections of say 50. And each of those sections has an expert who’s probably not the professor. Also, there will be a lot more collaborative learning among the peers in the class. If you think about it, a lot of learning does actually happen among the folks in a class. The expert just dispensing wisdom is not the way most learning happens. I call it “high engagement learning at scale.” A major piece of high engagement learning at scale is utilizing your classmates to provide a highly valuable learning experience.

Coursera is pursuing a path to enable high enrollments in low-cost programs, and they view their challenge to balance scale and student engagement, with class sections of ~50 students.

Good Enough vs. Better Enough

In two posts recently, Michael described a battle in the digital curricular materials market. Focusing on Cengage Unlimited in the first one, he described this dynamic.

Make no mistake; this is a potential inflection point in the curricular materials market. There is a war raging between curricular materials that are “good enough,” meaning that the lower price has a bigger impact on student outcomes than any differences in the quality of more expensive alternatives, versus “better enough,” meaning both instructors and students believe the product makes a sufficient difference in student outcomes that the more expensive product is worth the premium. Cengage is betting the farm on “good enough” beating out “better enough” and, win or lose, their bet could cause tectonic shifts in how curricular materials are developed, purchased, and used. It will have implications for inclusive access, adaptive courseware, textbook companies, textbook authors, and the landscape of options available to students and teachers.

Elaborating in the second post:

The distinction I’m trying to make between two strategies is a little tricky. I’m not arguing that Cengage, for example, thinks that their products aren’t great or that they think all anybody needs is the cheapest PDF possible. And on the other hand, “better enough” no longer means better editing or better production values, which is the way that textbook publishers used to position themselves against OER (and still do sometimes, although that reflex is beginning to fade). Rather, it’s about improving student outcomes.

What we are seeing in the OPM market, with the introduction of MOOC-based degrees, is a new battle. MOOC providers and its partner institutions, represented by Coursera, betting on “good enough”; and 2U and its partners betting on “better enough”. Like the curricular materials market, the product is based on student outcomes, which wraps in the value of the credential coming from the university along with the academic and administrative experience enabled by the company. Coursera obviously believes in the quality of their experience, and their partners have some programs that are not deeply discounted, but their market position is based on the program price being the compelling feature for students, including free or low-cost on-ramps. 2U understands that students are seeking more cost effective options, which was one driver behind creating the short-course segment with the acquisition of GetSmarter, but their market position is based on quality of experience and value of credential being the compelling feature for students. But the difference in approaches is stark and significant.

While there is likely room in the market for both approaches, the Coursera of 2018 (and not the Coursera of 2012) deserves careful observation to understand future trends with online degrees. Win or lose, their bet on low-cost online degrees will have big implications in the market.

December 6, 2018
Experience Economy: Enterprise software view into persistence and future of LMS market

Earlier this month Ben Thompson from Stratechery wrote a post, analyzing SAP’s $8 billion acquisition of Qualtrics, that provides insight into the shift in value proposition of the academic LMS. The SAP explanation along enterprise software lines shows the broader shift of enterprise software extending the view of the internal operations of an organization to also include a deeper view of the end users of an organizations offerings – students in the case of the LMS.

Thompson describes how SAP was founded in the 1970s and has a dominant position in Enterprise Resource Planning (ERP) systems that use central databases to provide customers with “a ‘real-time’ view of the state of their company” – essentially showing what the company is doing from an internal view. Customer Relationship Management (CRM) products emerged in the 1990s with the rise of ubiquitous PCs and the emerging Internet, tracking interactions with a company’s customers across time and across multiple locations – essentially showing a view of who the customers are and their interactions. Thompson then describes the challenge that modern companies face.

Fast forward another 20 years and the world has dramatically shifted yet again: not only are computing devices and Internet access ubiquitous, but critically, that ubiquity is not confined to businesses: customers, the ultimate endpoint of any business, are today just as connected as the employees of any large enterprise.

This can be a rather frightening proposition for large businesses: look no further than social media, where seemingly every week some terrible story about a company with poor customer service goes viral; there are an untold number of similar sob stories shared instantly with friends and family.

This same trend applies in education, with students being just as connected as faculty and staff of a college or university.

Thompson then quotes SAP’s CEO from a recent investor call [emphasis added]:

There are millions of complaints every day about disappointing customer experiences. This is called the experience gap. Businesses used to have time to sort this out, but in today’s unforgiving world, the damage is immediate, disruption is imminent. This has shifted the challenge from a running a business to guaranteeing great experiences for every single person.

Qualtrics provides a survey tool along with a sophisticated set of analytics and reporting tools based on this data – the key for SAP to understand consumer experiences. What is crucial, however, is not the standalone capabilities of Qualtrics, but the combination, again described by SAP’s CEO [emphasis added]:

To win in the experience economy there are two pieces to the puzzle. SAP has the first one: operational data, or what we call O-data, from the systems that run companies. Our applications portfolio is end-to-end, from demand chain to supply chain. The second piece of the puzzle is owned by Qualtrics. Experience data, or, X-data. This is actual feedback in real-time from actual people. How they’re engaging with a company’s brand. Are they satisfied with the customer experience that was offered. Is the product doing what they expected? What do they feel about the direction of their employer?

Think of it this way: the O-data tells you what happened, the X-data tells you why it happened.

This view of enterprise software navigating the larger trends of ubiquitous technology and connectivity, leading from the what to who to why, provides clarity on many of the trends we see in the ed tech world.

In education, the Learning Management System (LMS) was originally and more accurately called a Course Management System, and it has historically been focused on the management of courses, primarily through announcements to class, rosters, grade book, distribution of syllabus and course content, and submission of student work. Consider this figure from the ECAR Study of Faculty and Information Technology, 2017 that mirrors several other studies in its results:

While the modern LMS has advanced in many ways – particularly around usability, interoperability, and system reliability – the common usage of the this ERP-of-the-classroom has remained fairly steady. The dominant usage is managing the what of courses.

The LMS provides tools to manage communications – a view of the who of courses – through inbox, discussion boards, announcements, and various conferencing apps, but of these the dominant usage is through announcements. One way communication from faculty to students. The tools are there but not the reality of holistic views of interactions with students.

The shift in education from running a course to guaranteeing great experiences for students, to bastardize the SAP explanation, is much like the move towards experience management referred to in the Stratechery article. The movement is in its infancy, and it is likely to be measured in terms of decades, not years. Michael referred to this move in his most recent post.

If you’re a regular e-Literate reader, you know we have a macro thesis that the higher education sector is in the early stages of an evolution from having a philosophical commitment to student success toward having an operational commitment to student success. In other words, colleges and universities are starting to approach student success systematically, not as the natural by-product of hiring good faculty but as something that every student-facing aspect of the institution needs to be optimized for.

When you talk about student success, and great experiences, you have to go well beyond the official production of course content and grades and rosters. It doesn’t just matter what grades students get, it matters whether each student is learning, whether and when they get frustrated, and how often they’re engaging in the class. This gets to learning analytics and formative assessments and opportunities for students to quickly get help.

None of this is new, per se, and we’ve even seen attempts at alternative learning platforms to address this richer ecosystem. Consider the learning platforms designed initially to support competency-based education (CBE) such as Motivis Learning (spun out of Southern New Hampshire University’s College for America) or Sagence Learning (formerly FlatWorld Knowledge). These systems ((Disclosure: SNHU and Motivis were past subscribers to our LMS Market Analysis service.)), often called Learning Resource Management (LRM) systems, are designed to “see a holistic view” of students and “track student engagement”. They are designed to achieve the stated goals of SAP to combine operational data and tools along with experience data and tools.

We’ll get into more detail in future posts, but the category often labeled as adaptive courseware platforms are another example of next-generation systems that are designed to capture both operational data and experience data. These systems blur the boundaries between content and platforms and have the advantage of combining the two into a common design, which should allow deeper instrumentation of student activity during the learning process.

These examples get to the common question of whether the LMS will survive and exist in 10 years. The original LMS concept was designed around a course, not the learner, and most usage is administrative in nature, not learning activities. Shouldn’t next-generation systems like LRMs overtake the LMS market, as these companies expand beyond just CBE programs (see this post for context)? Well, the data do not show signs of this movement, and in fact the LMS market has been consolidating around just four solutions for institutional adoption – Canvas, D2L, Blackboard, and Moodle.

In the meantime, most of the LMS vendors have been adding functionality, whether through extension of their platforms or strategic integrations with third party tools, that seeks to provide views of the student experience. Learning analytics and reporting capabilities, mastery learning additions, federated sharing of student activity data.

One reason for the persistence of the primary LMS is that the LRM and courseware markets are not the ERP market. There are no SAPs in these worlds that already have ubiquitous usage. According to the Stratechery article “SAP is at the center of 77% of transactions worldwide”. The LRM typically starts out in a new CBE program with dozens, or maybe hundreds of students.

What is dominant in higher education circles? The LMS. It is one of the few ed tech solutions used in a majority of courses across online, blended, and face-to-face modalities. What the market appears to be doing is waiting for solutions that build on top of the LMS, or even extend the LMS itself, rather than replacing the LMS. And one of the main reasons is that the LMS has already been accepted as the enterprise system for academic usage, with operational data and tools managing the what of courses. It may be that over time alternative learning platform models will build up enough market share to become a credible threat to change the broader LMS market, but the signs so far are not encouraging for those vendors.

Qualtrics proved to be so valuable ($8 billion) because it could augment the ubiquitous SAP. SurveyMonkey, by contrast, went public as a standalone company and is worth far less ($1.8 billion, still a respectable number).

Looking into the future, the LMS will have to provide useful analytics on student outcomes, learning, and experiences along the way. Shifting from mostly running a course to guaranteeing great experiences for students. Whether this happens within the LMS of the future or as third-party augmentations of the LMS, and whether this happens with the current top four vendors or a different set, is not known. But the move to combine operational and experience data and tools is a trend we should expect to see over the next decade, both in ERP systems like SAP and in the academic LMS market.

December 2, 2018
Toward Operational Excellence at Student Success: California Community Colleges

If you’re a regular e-Literate reader, you know we have a macro thesis that the higher education sector is in the early stages of an evolution from having a philosophical commitment to student success toward having an operational commitment to student success. In other words, colleges and universities are starting to approach student success systematically, not as the natural by-product of hiring good faculty but as something that every student-facing aspect of the institution needs to be optimized for.

There is no road map for making this transformation and a number of formidable obstacles to it. First, academia was simply never designed for this purpose. The civilizational goal of empowering every human to live up to her or his potential via access to higher education is very new. Much newer than higher education system itself. In fact, it’s almost a thousand years newer. The University of Bologna in Italy, which is the world’s oldest university in continuous operation, was founded in 1088. The Morrill Land Grant Act, which created the first public universities in the United States, was passed in 1862. The G.I. Bill passed in 1944. Pell grants were created as part of the Higher Education Act in 1965. In 2018, achieving the as yet unrealized ambition of access to higher education regardless of income is very much a live political discussion. Just this month, the Sacramento Bee reported on a poll showing that 58% of Californians view college affordability as “a big problem,” with another 25% saying it is “somewhat of a problem.”

Even newer is the idea that we should not only be giving universal access to higher education but also taking responsibility to ensure that, once students have access, the institution is maximizing their chances of success (as opposed, for example, to the much older and still much more common idea of elitist “weeder” programs that filter for the “best” by failing out most). The deep structure of academia, from its governance to its professional training to its funding structure to its culture, is the evolutionary product of serving different missions than the one which we are now asking it to serve.

Second, even if we agree to embrace the mission of universal access and affirmative responsibility for student success in higher education writ large, how that plays out is very different at, say, Stanford, Loyola Marymount, UC Berkeley, Cal State Chico, and Los Angeles City College. The requirements for access are different. The definitions of and requirements for success are different.

And then their are the students, each of whom comes with her own definition of success, life goals, strengths, needs, and life context.

This is a hard problem. One that drives a lot of our work and our thinking. In a series of posts, I’m going to try to lay out what that shift looks like in a variety of academic contexts, how a successful shift across the sector would impact the future various ed tech product categories, and how the Empirical Educator Project (EEP) is intended to foster a methodology for empowering that shift.

In this first case study, I decided to start with the California Community Colleges Online Education Initiative (OEI). ((Disclosure: CCC OEI is a consulting client of ours.)) In fact, much of the structure of this post is drawn from an analysis we wrote on their behalf for the California State Legislature. I’m interested in extracting some generalizable lessons from OEI’s design. It’s important to be clear that the story I’m telling here is compatible with but not quite the same as OEI’s official position as represented in the report that they submitted to the legislature. OEI is also an interesting place to start this post series because, as we will see, it operates in an extreme environment that makes it particularly instructive.

This is a story about the whole being greater than the sum of its parts. OEI has put together a number of pieces that other institutions also have put in place, either individually or in various combinations. But they have done so with larger strategic vision for the future of California Community Colleges firmly and consistently in mind. It is also the story of a work in progress. California OEI has some impressive early successes under its belt. But it is also a hugely ambitious effort with much still to achieve. (Its in-process merger and rebranding with California Virtual Campus (CVC) is one example of forward-looking plans that I will touch on later in this post.)

Aligning the “business” drivers

When you go to the home page of the California Community Colleges web site, the first thing you will see, right at the top of the page, is the following:

The California Community Colleges is the largest system of higher education in the nation, with 2.1 million students attending 115 colleges. Our colleges provide students with the knowledge and background necessary to compete in today’s economy. With a wide range of educational offerings, the colleges provide workforce training, basic courses in English and math, certificate and degree programs and preparation for transfer to four-year institutions.

That’s a lot of students and a lot of colleges. One more college than last year, in fact. The legislature just approved the creation of a 115th campus (which will be virtual). California Community Colleges cover a lot of ground—literally as well as metaphorically. If you were to drive from College of the Siskiyous, which is about an hour south of the Oregon border, to Imperial Valley College which is about 20 minutes from the Mexico border, you would have to travel over 820 miles. They serve the top 100% of students. A lot of ground indeed.

There’s one word you won’t find in that rather dramatic description of California Community Colleges: “system.” Many state college and university systems are pretty big on local control, but California Community Colleges takes that principle to an extreme. For example, despite being a program intended to serve the entire system, OEI is run out of the Foothill-De Anza Community College District (after winning a competitive grant) because the Chancellor’s Office of California Community Colleges is more or less forbidden the legislature from running it centrally. California’s legislators are fiercely protective of the autonomy of their home districts. And as far as I know, OEI has negligible power to compel campuses to do anything.

In that environment, how do you help the entire 2.1-million-student, 115-campus, 820-mile-long “system” move together toward better operational excellence in enabling student success?

Academics tend to bristle at terms like “business drivers” and “business processes” when applied to academia, and there are good reasons to be cautious about using them. I’m applying the terms narrowly here because terms like “sustainability” are less effective at focusing people’s thinking about the machinery of balancing budgets. At the end of the day, colleges and universities need to take in as much money as they spend in order to keep fulfilling their mission. If you want to think clearly about how the sustainability machine works, then business is not a terrible metaphor. People have a basic, intuitive sense of what kind of machine a business is. The same sort of machinery is obscured the moment you start using words like “university” or even “institution” (or “sustainability”).

What do businesses (or sustainability machines) need? Money. There are a number of ways to have more money. One is to spend less of it. So one of OEI’s first moves was to offer to pay for the campus’ LMS, thus relieving each campus of the need to spend that money. LMS licensing may be an insignificant expense for an R1 university with a big endowment, but for a community college, it matters. There is no wiggle room in the budget. Hard choices have to be made—choices that impact student access and student success. We found anecdotal evidence that campuses have been using the money freed up by OEI’s LMS subsidy to invest in student success.

Our courses are much improved. In one year we have had 75% of current online instructors are fully certified. Almost 80 additional faculty are in process of being certified. We have approved 46 online course sections and reviewed or are currently review this semester another 35-40 courses. Without the resources from OEI and @one, we could not have made this happen.

– Faculty Senate Curriculum Chair, College of the Desert

Funding that would be used for [the common course management system] can be redirected to training for faculty who need extra help learning HOW to teach online.

– Dean, Business, Technology, and Career Technical Education, Ohlone College

Part of the people/resources that Coastline was able to shift, include our new Faculty Success Center, whose staff was able to create a new online course template in Canvas that helps faculty design a quality course. In addition, we were able to devote trainers to help faculty learn to use [the common course management system]. In this environment, our Academic Senate then felt comfortable mandating training for online instructors, something we never had before. I believe all this would not have happened if we had to pay for the [the common course management system] license….

So, continued state/OEI support for the…license will be critical for us to continue to train/support faculty and disseminate the use of these [OEI support] apps and support services….

One thing we were able to do, due to the free license, is pay all District faculty a stipend for the completion of [course management system] training.

– Associate Dean, Distance Learning, Coastline Community College

So just saving the campuses more money, by itself, led to actions by at least some campuses invested in improving their operational excellence at enabling student success. But that was really just the beginning. First, to get the subsidy, the campuses had to agree to do certain things. One of which was to adopt the same LMS. There were reasons for this, which I’ll get to shortly. For now, consider the likelihood of getting that many resource-strapped community colleges to migrate LMSs. How hard would it be? How long would it take?

All 114 campuses signed the contract agreeing to move to the common LMS, and many moved quickly to implement. In fact, the migration proceeded so far ahead of schedule that OEI had to go back to the legislature and request additional funding to cover the unanticipated extra subsidies. Saving these campuses money was a powerful motivator. And, as we’ll see, OEI accomplished a lot more than meets the eye with this one seemingly prosaic move of subsidizing an important but work-a-day piece of enterprise software.

How else can businesses make more money? By doing a good job of aligning their investments with their business opportunities. A grocery store doesn’t want to overstock with produce that will spoil on the shelves. But it also doesn’t want to run out of that produce when there’s high demand. And demand is variable. Demand for produce the week before Thanksgiving is likely to be different than the week after.

Colleges have an inventory management problem too. Sometimes courses are under-enrolled; other times they are over-enrolled. Both represent money problems to the campuses. One of the reasons that OEI wanted all the colleges on the same LMS—not just the same brand, but the same instance—was to create a course exchange. Balancing course “inventory” in a single community college is tough. Room availability, instructor availability, changes in the job market and economy, and the unpredictability of part-time student enrollments all work against you. But balancing “inventory” across 114 community colleges is less hard (once you can figure out how to get it to work in the first place). One campus may be over-enrolled in macroeconomics, but chances are pretty good that one of the 113 other campuses is under-enrolled in the same course. If you can get enough campuses to put enough courses on the online course exchange, then you can solve a “business” problem for all of the campuses. And the more courses there are on the exchange, the more valuable it becomes to the campuses. Thus, colleges have incentives to create courses for the exchange, and the more courses that are created, the more incentive the colleges have to utilize the exchange.

You could tell this same story from a student access perspective. Over-enrolled courses prevent students from taking them in a timely way. If the course is required, this could force them to delay graduation (and a full-time or better paying job), take on additional unneeded courses in order to qualify for financial aid, take extra financial aid from the state and federal governments, take up an enrollment space that might have gone to other students, and increase the risk that they will not graduate. Under-enrolled courses risk cancellation, with many of the same knock-on effects. I don’t mean to neglect or downplay this portion of the story.

But the focus on business incentives lets us think more clearly about the machinery of the institution itself. Which, in turn, helps us to think clearly about how that machine works and how it can be tuned. OEI designed a machine to drive operational excellence at enabling student success across the largest community college system in the country. And it runs on only positive incentives because. This design constraint immediately rules out copying some of the most frequently cited examples of innovative universities which, through one mechanism or another, can exert varying degrees of top-down control. At ASU, President Michael Crow has an unusually strong hand to play within a reasonably traditional structure of faculty shared governance. (Ithaka S+R has some interesting and revealing interviews of some of ASU’s top leaders that give some hints about how that governance works.) Western Governors University is more extreme; there is no faculty senate and no shared governance. SNHU’s Paul LeBlanc has tried a combination of strategies, working with with the faculty senate on governance of the traditional college while separating out their College of Online and Continuing Education (COCE) and running it in a way that is only loosely coupled to the shared governance of the rest of the university. Like many universities and systems, OEI cannot redesign the machine from the top down. so thinking about the live-or-die campus sustainability incentives that could be used to drive collective action has been a central principle that influenced the rest of OEI’s design.

Creating the infrastructure

The desire to move all campuses in the system to one LMS wasn’t just for the sake of contracting convenience. It accomplished a variety of goals. First, it became a foundational layer of software infrastructure for rolling out other system-wide capabilities and services, from plagiarism detection to online tutoring to faculty training and help resources. Having everybody on the same instance of the same platform—cloud-hosted Instructure Canvas—made it much easier to do this. In the old world, where campuses were on a hodgepodge of different self-hosted and vendor-hosted LMSs, the best the system could have accomplished would have been common contracting. It would still be up to each campus to integrate and support the tools and services. After all, the way a tool looks and works in Moodle can be different than in Brightspace. Centralized support would have been a nightmare. And remember, these campuses are very tight on resources. Supporting add-on tools and services is costly to them.

In addition, sharing one LMS made it easier for OEI to create faculty training that could be shared across the system, and for campuses to do the same. As with system-wide licensing for LMS-connected tools and services, it’s not impossible to do this in a system with different LMSs. But the added friction makes it less likely to happen. I’m going to use the “B” word again: academia needs to think about business processes. Once again, stripping away the culture- and mission-inflected language lets us see the machinery more clearly. A business process is the way in which a business accomplishes something that is important for the business. For example, how does a business make sure that all its employees have up-to-date software, including critical ones like system updates and the latest anti-virus software? Sure, they could leave that to the individual employees to do. We’ve all updated our software on our personal computers; it can be done. But how likely is it that everyone will do so in a fashion that is timely, reliable, and consistently correct? And what work are all those employees not getting done while they are wrestling with software updates? It’s better to develop a business process for pushing out those updates from a central IT group so that employees don’t have to worry about them. Likewise, there are effiency benefits to centrally rolling out and support services for 115 campuses than to have each campus IT support person duplicate the effort. From a perspective of strengthening the business drivers that hold the group together, all of these benefits can be boiled down to saving money by providing additional capabilities with reduced cost to on-campus resources (in direct licensing fees, support staff time, or both). As we have already seen, the campuses tend to invest the money they’ve saved in enhancements that are specific to their local needs and that benefit their students.

The common LMS also helps with the over- and under-enrollment problem. Having all course exchange courses on a single instance of a common LMS made it easier both to provide more data to the campuses that would help them with their planning and to reduce friction in expanding the course exchange. If everybody is using the same system, that’s one less thing for faculty and students to learn, less help desk support, and more productive support (for both course delivery and course design) because the OEI staff don’t have to try to accommodate multiple flavors of learning environments.

Of course, there are trade-offs, the biggest one being autonomy. In OEI’s case, for example, all the campuses had to agree to use the same LMS rather than choosing their own. Anyone who has run a campus LMS selection process knows it can be an exercise in delicate diplomacy. Imagine doing the same with 114 campuses. As we’ll see, OEI turned this challenge into an opportunity. I’ll have more to say about that in the next section.

Anyway, once you start seeing infrastructure as the structure “underneath” (i.e., “infra-“) that supports business processes, two things immediately start to happen. First, your definition of success changes. You can no longer declare victory just because you successfully installed the software and got people to start using it. You have to start looking at whether it successfully enabled or improved the business processes you were intending to support.

Our Professional Development Coordinator is encouraging the creation of Pro Dev workshops in [common course management system] Canvas, such as health and wellness (“dealing with difficult people”), how to create Open Educational Resources, how to use [Student Learning Outcomes] for better teaching, and a lecture on science and its assumptions. What is developed at Butte can be instantly shared with other schools, and vice versa. I see a renaissance of Pro Dev opportunities!

– Technology Mediated Instruction coordinator, Butte College

The second thing that happens when you start thinking in terms of business processes is you identify new problems as well as rethinking and reprioritizing old ones. For example, OEI is in the process of revamping (and merging brands with) California Virtual Campus (CVC). Why? CVC exists today. It’s a web-based catalog of online courses students can across the California Community Colleges and California State University System, which is many more than the handful of OEI exchange courses. As of 2017, CVC included 23,445 online courses and 1,376 degree programs. So it’s big. If you think of infrastructure as a thing to have, then you might think of CVC as a massive success.

But if you think about CVC as the structure underneath that supports the critical business process of students finding, (wisely) selecting, and registering for online courses across the many campuses represented in the CVC course catalog, then you start to develop different metrics for success. And if you also think about the back-end process of making sure the right institutions get the registration information, tuition, and transcript information (respectively), then CVC becomes both a critical priority and a tough piece of infrastructure to build well, particularly across so many different campuses that are not all migrated to a single instance of common Student Information System (SIS) software. If students fail to register for online courses that they need because the process is too cumbersome, or if they don’t get properly credited by their home institutions after taking an exchange course, that is bad for the long-term health of both the students and their institutions. This is what it really means to say that some infrastructure is “mission-critical.” CVC is a big catalog with lots of courses, but it does not yet do a great job of fulfilling its mission-critical role of helping students find, register for, get credit for, and pay for their courses. Each of those is a business process that CVC should support. And the number of courses in the catalog tells us very little about how well the software is supporting those processes for the students and the campuses.

If you think about infrastructure in this way, then your communications to your stakeholders will also change. Here’s an explainer video they had us create for them in order to help communicate that message to the various campus folks:

(Source video: https://youtu.be/1DdlaIZYiDI)

Why was this so important to communicate? OEI could have gone to their constituents with a message of “Here’s a bunch of great free stuff for you!” Instead, they chose a much more challenging message to communicate; one about the ripple effects of having shared infrastructure. That wasn’t an obvious choice.

If you’ve read any one of a million articles on how to succeed with any major campus-wide initiative, you will have read the cliché about how important it is to “get buy-in.” Most of the time, “getting buy-in” is interpreted as “handling objections” or “reducing resistance.” It is an obstacle to get past. But the video above shows that the OEI leadership has interpreted the term differently. You only communicate to your stakeholders in this way if you believe that buy-in is infrastructure.

Fostering a culture

In our consulting work, we facilitate LMS selection processes reasonably often. The more forward-thinking institutions view these processes as opportunities. How often do you get to gather a group of faculty and other academic stakeholders from across your institution in one room and have them talk to each other about how they teach and what they need to serve their students well? A search for a product like an LMS can become a rare opportunity for focused, intensive, purpose-driven community-building. Yes, it lowers resistance, enabling people who might not be happy with the final decision to at least feel like they were heard. But it also begins to foster familiarity and dialog that can help foster a broader and more lasting community of purpose. When you’re trying to build such a community across 114 campuses as part of an ambitious and completely voluntary system-wide effort, taking that view of LMS selection is even more important. Needless to say, it wasn’t easy. Or seamless. That said, both during the selection process and and afterward when communicating the results, OEI worked toward building affirmative buy-in—not just lowering resistance but increasing the sense of goodwill and common purpose. For example, the selection committee was near unanimous in its selection, with the sole dissenter acknowledging the importance of what the committee was doing together and supporting the final decision.

The program design elements I’ve described so far helped OEI to foster increased organizational alignment at two levels across campuses. The campus executives who are responsible for financial health and sustainability of their campuses are aligned through infrastructure subsidization and the course exchange. In 2018, the state legislature decided to augment the initiative with an additional $35 million funding. California Community Colleges has chosen to invest that extra money capacity-building. In particular, the money will go toward grant programs intended to enable the campuses to launch more online courses on the OEI-CVC infrastructure, thus further strengthening this alignment while aiming to serve more students effectively. The common infrastructure and the culture-building process around it has helped to build a culture among the academic and technical support staff across campuses. The hoped-for consequence is that improvements on one campus will travel more quickly and easily to others:

I now know that I can ring up any other [Distance Education] Coordinator and we’ll be speaking the same “language” regarding the use, training, and administration of the [course management system]. I’m also really looking forward to faculty being able to share ideas and resources via Commons.

– Director of Distance Education, Santa Rosa Junior College

Building a similar sort of sharing network among the faculty in the system is an even larger challenge. The culture-building work has been ongoing work for years now, but one can acknowledge all the hard work and progress to-date while still also recognizing that this is a huge project that has barely begun. Certainly, having common resources and common platform supported by OEI has provided a boost, as has the inclusion of faculty voices in the OEI planning process. The augmentation grants, which will likely include instructional design support, represent another opportunity. But to me, one of the most interesting vectors for culture-building is the course exchange course quality rubric. Every course on the exchange has to be evaluated against a rubric of evidence-backed effective online teaching practices. As the pace at which exchange courses are developed increases, OEI will not be able to keep up with demand to evaluate these courses using central staff. So they are creating a peer reviewer mechanism in which faculty on the campuses are trained on the rubric and presumably compensated to review courses that are candidates for the exchange.

This opportunity fascinates me. We know that faculty who go through an expert-supported course redesign process often experience intellectually deep and emotionally moving shifts in their teaching strategies. Is the same true when faculty are trained reviewers of their colleagues’ redesigned courses? What effect will simply exposing faculty to more and different course designs have? How will their role as reviewers and critiquers shape or enhance that effect? Can a continuously improved and updated rubric become a vector for sharing new research-supported processes across the system on an ongoing basis? Will the impact be broad and deep enough to foster new kinds of intra- and inter-campus faculty dialogs about the scholarship of teaching and learning (SoTL)? Will these cultural changes help to foster alignment around continuous operational improvement for enabling student success? This is the last mile problem of higher education. Operational excellence at student success cannot be achieved unless it is infused in the daily operations in individual classrooms. That requires affirmative faculty buy-in, support, training, and embedding in a culture that invites them into the larger conversation.

This is highly reminiscent of the cultural transition that doctors had to make from the mid-Nineteenth through the mid-Twentieth Century. In the 1840s, one could begin practicing as a physician with no medical training at all, just as one can start practicing as professor with no pedagogical training today. Doctors learned medicine from whomever they happened to train with and whatever they read in the newspaper ads about cures and treatments ranging from early antiseptics to leeches and literal snake oil, with no easy way to distinguish them. There were no major conferences or respected, peer-reviewed journals. There were no standards for quality research or convincing evidence. There were a handful of teaching hospitals and medical colleges of wildly varying quality that touched only a small minority of practicing physicians. All of these institutions, all of this social infrastructure, needed to be built and bought into by physicians before antiseptics could be differentiated from snake oil, the signal separated from the noise, regarding “progress” or “innovations” that might help their patients’ welfare.

OEI has accomplished some remarkable early successes in an extremely challenging context. But the degree to which they are able to move 114 California community colleges toward better support of student success as a group may well depend on the ability of the social infrastructure they are creating to reach faculty, be embraced by them, and and foster a culture in which academics collaborate differently and more intensively in their day-to-day work of helping students to succeed, one student at a time.

November 27, 2018
OLC 2018 SoTL Panel Further Info

I’m going to be facilitating an Empirical Educator Project-relevant panel at OLC today at 11:15 AM in Oceanic 1, followed by an EEP and EEP-curious meetup at Soomo booth (#226) at 12:15 PM in the Expo Center. The rest of this post is just a little extra information on each of the SoTL work of the panel participants’ home institutions, for those who attend the session.

CMU Eberly Center

At the intersection of faculty research, teaching, and service, the Eberly Center supports Teaching as Research. We help faculty answer compelling research questions regarding which teaching strategies are more effective at promoting learning, increasing engagement, and enhancing the learning environment. Our services provide the tools and expertise to help instructors develop research questions and study designs, identify valid and reliable data sources, analyze and interpret educational data, and present and publish research results. Read more about our research processes and findings in this site:

www.cmu.edu/teaching/teaching-as-research/index.html

UCF

At UCF, SoTL research is incentivized through an administrative Faculty Award that includes a $5,000 one-time award and a $5,000 addition to salary base. facultyexcellence.ucf.edu/recognition/scholarship-of-teaching-and-learning/

Support is provided by the UCF Faculty Center for Teaching and Learning and the Research Initiative for Teaching Effectiveness. RITE assists faculty, free of charge, with any SoTL activity within the research design to dissemination continuum.

CTU

CTU is a career-focused university encouraging the use of educational technology and SoTL research in the areas of professional scholarship and adaptive learning. Faculty (including adjunct faculty) can apply for funding through an internal website and faculty are encouraged to share their research and scholarship work with the university. Additionally, research collaboration with other institutions is supported and encouraged as demonstrated by the work with CTU and UCF.

Ole Miss Adaptive Learning SoTL Poster

http://cetl.wp2.olemiss.edu/wp-content/uploads/sites/83/2018/04/Revised-Personalized-Learning-Poster.pdf

November 15, 2018
OLC 2018 SoTL Panel Further Info

I’m going to be facilitating an Empirical Educator Project-relevant panel at OLC today at 11:15 AM in Oceanic 1, followed by an EEP and EEP-curious meetup at Soomo booth (#226) at 12:15 PM in the Expo Center. The rest of this post is just a little extra information on each of the SoTL work of the panel participants’ home institutions, for those who attend the session.

CMU Eberly Center

At the intersection of faculty research, teaching, and service, the Eberly Center supports Teaching as Research. We help faculty answer compelling research questions regarding which teaching strategies are more effective at promoting learning, increasing engagement, and enhancing the learning environment. Our services provide the tools and expertise to help instructors develop research questions and study designs, identify valid and reliable data sources, analyze and interpret educational data, and present and publish research results. Read more about our research processes and findings in this site:

www.cmu.edu/teaching/teaching-as-research/index.html

UCF

At UCF, SoTL research is incentivized through an administrative Faculty Award that includes a $5,000 one-time award and a $5,000 addition to salary base. facultyexcellence.ucf.edu/recognition/scholarship-of-teaching-and-learning/

Support is provided by the UCF Faculty Center for Teaching and Learning and the Research Initiative for Teaching Effectiveness. RITE assists faculty, free of charge, with any SoTL activity within the research design to dissemination continuum.

CTU

CTU is a career-focused university encouraging the use of educational technology and SoTL research in the areas of professional scholarship and adaptive learning. Faculty (including adjunct faculty) can apply for funding through an internal website and faculty are encouraged to share their research and scholarship work with the university. Additionally, research collaboration with other institutions is supported and encouraged as demonstrated by the work with CTU and UCF.

Ole Miss Adaptive Learning SoTL Poster

http://cetl.wp2.olemiss.edu/wp-content/uploads/sites/83/2018/04/Revised-Personalized-Learning-Poster.pdf

November 15, 2018
Toward a Methodology of Change in Higher Ed
For a couple of years now, we’ve been saying that higher education is at the beginning stages of a long transition from a philosophical commitment to student success toward an operational commitment to it. In other words, colleges and universities are beginning to grapple in earnest with how to rewire themselves so that their culture and processes are deliberately optimized and continuously tuned to support their students in getting the best education possible. This is a profound shift. It will require major changes to the ways in which academia works and the ways in which ed tech designs and markets its products. It will be very hard and take a long time. But the drivers of this change are in place.

Recently, I wrote about how our concept of Empirical Education has built into it a theory of change. The implication is that it has the backbone for a methodology of change. Our work as both analysts and consultants shown us that the increasingly aligned strategic priorities throughout the sector, when combined with the knowledge that is scattered across it, can be distilled down into a powerful yet flexible methodology for system change in education analogous to Design Thinking or one of the Agile software development methodologies. It can be a set of processes, built on a fairly small set of fundamental principles but supported by a lot of detailed craft knowledge and a rich ecosystem of supporting tools. It can be owned by no-one, although there would likely be some premiere practitioners of it. Colleges and universities could use it to redesign themselves to be more student-centric and, in the process, also more educator-centric. Product and service companies could design their offerings around it and compete based on their ability to help their academic customers better implement it.

This sense of possibility has been the animating impulse behind the Empirical Educator Project (EEP). We started with only a hazy idea of what we were building. Over the last twelve months of working with academics and ed tech product people, some aspects have become clearer. I have grown more confident in the potential of the idea even as I have grown more overwhelmed with clearer understanding of the size of the undertaking.

I am going to articulate my latest thinking about it in this post.

The time is now

There is a saying among consultants that potential clients won’t hire a consultant until and unless they both realize that they have a serious problem and come to accept that it is not a problem they can solve on their own. That holds equally true for a wide range of difficult changes that require help or cooperation, from coping with an addiction to building a functioning government to changing an institution. Higher education has been an incredibly stable system. As in, remarkably consistent over a period of about a thousand years. Historians of education tend to write about changes that take place over decades or half-centuries. There has been a looming question of whether such a slow-changing institution can adapt to such fast-changing times. Unsurprisingly, this debate has been raging for a few decades now, with relatively little sector-wide change to show for it. Is the system terminally rigid, or is it in a state of punctuated equilibrium that will shift in an appropriately dramatic amplitude once it reaches an inflection point?

I believe the latter is the case, and I believe that we are at that inflection point. Access-oriented institutions—particularly publicly funded ones—have already been under pressure for some time now to show better outcomes for students in terms of rough measures like graduation rates and time to graduation, as well as some more meaningful but difficult measures popping up on the margins such as employment and career success. On the other end of the spectrum, the elite institutions whose brands have popularly defined excellence in education for the last century or more are starting to realize that they need to adjust to changing student expectations if they are going to continue to be considered the gold standard for the next century or more. The MOOC craze was complex and problematic, but it woke the elites up to the potential for use of technology-enabled approaches to enhance their teaching practices rather than detract from it, even as their students show up on campus with increasingly high expectations for the kinds of access to knowledge, interactive experiences, and high-touch communication that technology can enable. And in the middle, private universities with decent regional reputations and tuitions that approach those of Ivy League schools are increasingly under pressure to justify their tuition with something of more permanent value to students than climbing walls and dining halls. More and more, the buzz is about innovative partnerships with employers, or about learning analytics, or about student success systems supporting better guidance counseling. In other words, we are seeing colleges and universities grope toward approaches that enable them to more reliably support student success. And they are looking for help to do it.

The focus of that previous paragraph is primarily on undergraduate education, but it also increasingly applies to graduate education. We sometimes see this problem manifest itself in financial terms, where it gets somewhat obscured by the current conversations around Online Program Management (OPM) companies. Universities often launch career-oriented graduate programs such as MBAs and MSWs because (a) they are looking for more revenue to make their institutions more sustainable, (b) online programs can scale without scaling costs like real estate and physical classrooms, and (c) they know there is a market of people who are inclined to sign up for online graduate programs that can fit with their work and family schedules while also giving them credentials that will help them advance in their career ambitions. But as the online MBA market gets saturated, universities increasingly have to find differentiators. And they can’t use climbing walls or dining halls. In the end, the only effective and durable differentiator for an online career-oriented graduate degree program is its effectiveness at helping the students achieve their goals. In this space, the immediate university driver is revenue and the immediate student goal is career advancement. So the sector tends to view this change narrowly. But if you zoom out a little, it becomes clear that the trend with graduate programs and OPMs is just one particularly clear example of where the academic institution’s financial sustainability issues are driving it toward a sharper operational focus on its mission.

Let’s turn now to the educational vendors, who are also at an inflection point across product categories. All of these companies—curricular materials providers, LMS vendors, SIS vendors, analytics vendors, and so on—they are all looking to move up the value chain and argue that their products can directly, meaningfully, and provably impact student outcomes. ((I am use phrases like “student outcomes” and “student success” interchangeably and broadly for the purposes of this post, even though I know that they can have different connotations.)) They have to, because most of the major ed tech product categories are either in danger of commodifying or in danger of failing (in the case of established product categories) to achieve meaningful market penetration (in the case of new ones).

The textbook companies hit the wall first. As students increasingly found ways to avoid buying new books (or any books), the textbook publishers raised their prices, which started a vicious cycle of reduced sell-through followed by price increases followed by further reduced sell-through followed by further price increases. This was ultimately unsustainable, particularly since the internet has made obtaining basic factual information and focused educational supplements—think YouTube—easily obtainable and free. Increasingly, publishers had to make the case that their content is somehow better than the commodity content. But better how? For a long time, the “better” publishers worked on was instructor convenience. But there’s only so far that slides, extra problem sets, and auto-graded homework can compensate for the vicious pricing cycle, particularly since the commodity materials get more organized and feature-rich over time. Eventually, the major publishers came to the conclusion that the only sustainable “better” they could shoot for is more educationally effective.

Pearson was the first out of the gate with a massive push for “efficacy.” ((Disclosure: Pearson is a sponsor of EEP.)) They have bet and are still betting the company on that strategy. But as I have written about here before, the fundamental problem is that products can’t really be “efficacious” in and of themselves unless the educators in whose class the materials are being used (a) agree with the efficacy goals that have been defined by the product developers and (b) change their teaching to work with the educational strategies designed into the products. More fundamentally, the educators have to trust the research claims of the vendors in order to even think about the product-defined efficacy goals, much less adjust their teaching strategies. Pearson’s original articulation of efficacy failed to account for any of this. They have since adjusted their course, and other curricular materials developers—most notably McGraw-Hill Education and Macmillan among the larger players—have followed suit by also focusing more on encouraging faculty to buy into research-backed teaching practices and then, having obtained that buy-in, show how their products support and implement those practices. ((Disclosure: McGraw-Hill Education is a sponsor of EEP and subscriber to our Trusted Advisor market analysis service. Macmillan is a sponsor of EEP.)) But for all their good efforts—and they are generally, good, honest efforts—these vendors are pushing string. Most academics will never take them seriously as a source of advice for considering deep and scary changes to their teaching practice.

Meanwhile in the LMS space, the developed markets have saturated and are stabilizing. New adoptions appear to be down. There are many developing markets to plumb, but they are slow and expensive to develop. So LMS vendors too have been trying to move up the value chain by talking more and more about student success. D2L has focused for some time now on the course design process and has been adding tools to its portfolio like LeaP, which is a tool for recommending personalized supplemental curricular materials. ((Disclosure: D2L is a sponsor of EEP and a subscriber to our LMS market analysis service.)) Blackboard has gone so far as to promote themselves as “your partner in change,” to the point of deprecating their flagship LMS project as “not enough.” ((Disclosure: Blackboard is a sponsor of EEP and a subscriber to our LMS market analysis service.))

And yet, the LMS companies face the same uphill battle with credibility that the textbook publishers do. By and large, academics are not going to look to their LMS providers for guidance on how to change their teaching practices. The same goes for the upstart product categories like learning analytics. Vendors will struggle to convince academics to change their teaching practices, but their products will mostly fail to demonstrate meaningful learning impact until the academics adopt practices that take full advantage of the products. All these vendors need to climb a wall of credibility with academics, but they can’t do it unless somebody throws them a rope. (Companies with significant faculty-facing service components have the best chance of swimming upstream, but that’s another post for another time.)

All the institutions in the sector—all types of colleges and universities, all types of ed tech vendors—have realized that they have a problem and are starting to realize that they can’t solve it on their own. They recognize that the core problem is that colleges and universities need to get much better at supporting student success, however their particular students may define it. They all want to get there and are starting to look to each other for help. But they don’t know how, and most of them can’t do it alone.

There’s only one stakeholder group in this picture that has not gone through the process of seeing that they have a deep problem and accepting that they need help solving it yet. Have you spotted who they are?

The people who can actually solve the problem

While the shift in incentives has reached a tipping point for the institutions, the same cannot be said for the faculty. Their graduate training is largely unchanged. Their tenure and promotion criteria are largely unchanged. The rewards and accoutrements of professional accomplishment are largely unchanged. Faculty have been given no reason to change; therefore, they don’t. Everybody knows this is true.

Or not. There are several vital aspects of this story which everybody “knows” that are either misleading or flat out wrong.

First, faculty do change. Anybody who has significant experience with the development of online learning programs or other course redesign efforts has seen it happen. They have faculty say that their experience in the redesigned class has changed the way they teach in other classes. They have watched skeptical faculty turn into preachers of the gospel. There are converts. Despite a dearth of incentives and a plethora of disincentives, despite uneven support, despite the fact that most will earn no glory for it on the other side of the closed doors of their respective classrooms, faculty do embrace pedagogical change when they have the right sorts of experiences that enable them to see the benefits.

Where are these amazing faculty members? They are everywhere and nowhere. They tend to be invisible on their home campuses, although if you ask around in different departments, you might be lucky enough to catch sight of one or three. (Or a dozen.) They have often learned the hard way that there is little benefit and significant pain involved with preaching on their home campuses, so many of them keep quiet and quietly work their magic in their own classrooms. If you want to see them in numbers, you usually have to go to one of the conferences where they congregate. I am going to one this week. One of the main activities of the participants will be crying on each other’s shoulders about how under-appreciated and under-resourced their efforts are on their respective home campuses.

It is also untrue that incentives for faculty to excel in their teaching craft remain rare. It’s still early days, but there are green shoots everywhere. Most of the time, we only hear about a small number of schools that are doing remarkable things. Arizona State University, Southern New Hampshire University, and Western Governors University, over and over again. If you’re a little more knowledgeable, you might have heard about work at University of Central Florida or Georgia State University. And if you’re paying attention to formal scholarship, you might a little about work coming out of places like Carnegie Mellon University, Duke, and Stanford. We could look a little further down the publicity pyramid at places like the University of Maryland Baltimore County. You very likely haven’t heard about the amazing work happening at diverse schools ranging from James Madison University to Coppin State University. I wouldn’t have known anything about the accomplishments of either of these institutions if I hadn’t stumbled upon them through my various travels in this very odd job of mine.

And because the news tends to focus on a few exceptional institutions, it also focuses on three contributors to success that are among the hardest to change: leadership, governance, and money. It is simply not true that the only institutions making real change have once-in-a-generation presidents, an iron grip on the faculty, and/or tons of funding. We see innovation everywhere. And everywhere it happens, it happens because institutions are finding new ways to draw on their most precious yet plentiful resource: their faculty.

There is an old term of art that deserves reviving and refreshing: the scholarship of teaching and learning (SoTL). SoTL is often seen as a grassroots effort by faculty who care about teaching to wrap it in the cloak of academic validity. If the only way that excellence in teaching will be valued by the institution is to get it into peer-reviewed journals, then let’s find a way to get it into peer-reviewed journals. In the past, institutions generally didn’t take the bait. Many treated SoTL as a pat on the head to faculty who were slaving away carrying the heaviest teaching and advising loads. “Here, you care about this teaching stuff. Have a workshop. You can pretend what you’re doing is scholarship for a while. And we’ll give you a certificate!”

That is changing. More and more institutions are realizing that faculty aren’t the problem; they are the solution. But that grassroots energy that comes from SoTL and other faculty empowerment efforts must be aligned with institutional efforts through support, incentives, and research. More and more institutions are making that connection. For example, here’s a graphic illustration of the dynamic, taken directly from Georgetown University’s Designing Our Future(s) web site:

Here are some lessons learned from Georgetown’s white paper about the progress the initiative has made so far:
A few core rules for this innovation work have emerged. First, every project has to push against some structural constraint (the 15-week semester, the credit hour, the nine-month calendar, etc.) and test variations of it. Second, projects cannot be idiosyncratic or depend on the particular interests of one talented faculty member; they have to be pilots from which we can generalize and which we might apply to other scenarios or problems. Lastly, we only fund a Red House project for one year (or the equivalent); after that, if a project is to survive, it has to be absorbed into the curriculum and faculty workload.

Beyond these basic rules we have also learned some valuable lessons about the viability of experimental and creative curricular work in a culture designed for deliberative shared governance and slow change:

We developed strong stakeholder involvement as part of our iterative design process—one that frequently included associate deans, the registrar, compliance officers, and financial aid representatives—early in each project’s development. Likewise, we communicated well and regularly with our board, alumni, and donors

We do not give ourselves as good a grade on continuous communications with faculty. Early on there were many open invitations and speaker events, and a drumbeat of updates. As the work became more intense and demanding, we focused inward, and neglected to continue to reach back out to this important community. We learned it is absolutely critical to spiral communications outward, and to be as inclusive and open as possible, especially as the work takes specific shape within a core group.

Very early on we should have established a formal faculty review and approval process for Red House pilots. We assumed we would work within the Curriculum Committee approval structures, long established for important reasons, but which do not in the end benefit a research and development initiative. Last year, a Designing the Future(s) Advisory Committee was created, with the sole mission of approving and monitoring innovation projects. This system is now working very well; it might have accelerated progress if it had been instituted earlier.
These are very early lessons, and they are somewhat Georgetown-specific. But it’s easy to see some more general principles emerge that could be useful across a wide range of educational and cultural contexts. And some of the most fascinating and remarkable changes are happening at institutions that you never read about, including some that have traditional faculty governance, few financial resources, and leaders who are extraordinary in the “normal” sense that many committed, hard-working, people-oriented academic leaders are in colleges and universities of all shapes and sizes.

I am going to write about some specific examples of this sort of organizational alignment in upcoming posts. For now, I want to spend a little time on the characteristics of a good methodology.

Toward a methodology of Empirical Education

When I think about general methodology that can be adopted and adapted across a wide range of contexts, the two models that come to mind immediately are Agile software development and Design Thinking. I’ll focus on Agile (and particularly Scrum) for the moment because I know it better, but as far as I can tell, the same basic principles apply to Design Thinking.

First, the methodology should be designed to unleash the creativity of the knowledge workers involved in the critical processes. All too often, we take really smart people and put them in a strait jacket of process. We tend to design our mission-critical processes to get us predictable results, often by controlling the human element through various management techniques. The problem arises when we ask for predictable results in an unpredictable environment, having handicapped the very smart people who are best able to minimize the problems that arise out of unforeseen circumstances while maximizing the benefits of unforeseen opportunities. There is no knowledge work I know of that has more frequent and dramatic unforeseeable challenges and opportunities than education. We wrap a lot of process around education, but it’s not the right kind of process to promote excellence by getting the most out of talented educators, just as using Gantt charts was not the right sort of process to promote excellence in software development by getting the most out of talented engineers.

At the same time, empowering knowledge workers is not the same thing as letting them do whatever they want. I have been in an Agile software development environment where the engineers interpreted Agile to mean that they decide everything. The results were not good. All Agile methods that I am familiar with have multiple roles, with each role having certain authority and responsibilities. These roles are designed to be mutually supportive, and the success or failure is very much a success or failure of the entire team and its teamwork. This is a big cultural change for many institutions, where “academic freedom” has come to be used reflexively as a shield from any demands, sometimes because some of those demands are unreasonable or unwise. There has to be a well-defined process by which student success is understood to be the collaborative responsibility of the academic team, working together as an ensemble.

These two basic principles—empowering individuals and working as teams—can generally be captured in a fairly small number of rules and roles, regardless of the flavor of Agile being practiced. And most Agile teams that get them right will function adequately while getting more satisfaction from their work—under relatively unchallenging circumstances. They may even feel that they are doing Agile well. But then there is a whole world of craft that is all about handling context-specific challenges. How do you balance functional versus non-functional requirements? How do you prioritize aging aspects of the software, a.k.a. “technical debt”? How do you manage large projects that require many Agile teams? How do you deal with extrinsic constraints on release timing (like the start of an academic term)? Agile practitioners can always improve their craft, both as individuals and as teams. Entire industries of tools and consulting have grown up around supporting excellence in that craft.

Which is utterly unlike the way in which the industries that surround education function (or fail to function) today. There is a reason for that. An industry designed to promote operational excellence of knowledge workers cannot succeed in absence of a shared understanding among the knowledge workers about what operational excellence looks like. Agile software development is a craft with a lot of consensus around the principles, a track record of results, and enough expert practitioners that knotty problems, along with their solutions, can be shared fairly efficiently across a very large and loosely organized profession. There is a lot of debate too, which is the sign of a healthy ecosystem of knowledge workers advancing the leading edge of their craft. But that debate occurs within the context of a common understanding that is woven into the culture. Practitioners in those debates are rewarded with recognition of their expertise and contribution to the field. And their employers love having these experts and reward them appropriately because their excellence at creatively applying and innovating with the methodology advances institutional goals.

With that cultural substrate in place, a tool or service vendor can come in and say, “We help you solve X sort of problem in your Empirical Education process,” and the prospective customers will, understand what is being offered, be capable of evaluating its utility, and place (monetarily quantifiable) value on that utility. That’s what we need for learning analytics, adaptive learning, or just about any whizzy, trendy ed tech thingamabob you can think of or will be thought of.

Most or all of the elements for a methodology of operational excellence in education exist in the world today. They need to be gathered, distilled, and refined into a learnable, repeatable, and adaptable practice. That is the outcome we aspire to achieve in collaboration with the participants in the EEP, not to mention support from the collective wisdom and will of higher education writ large.

Moving forward

As I wrote earlier, I will be blogging about relevant examples we are seeing, on both the institutional side and the vendor side, in the coming days. And EEP will soon be announcing the first release of some tools that can help form a foundational layer of the institutional infrastructure for Empirical Education. In the meantime, if you are going to be at the Online Learning Consortium Accelerate conference, I will be moderating an EEP-relevant panel discussion of SoTL on Thursday at 11:15 AM in Oceanic 1. From there, some of us will head to the exhibition hall, where we will have an EEP meet-up at the Soomo booth (#226) at 12:15 PM. You don’t have to be a member of the current EEP cohort to join us; the EEP-curious are welcome.
November 11, 2018