Instructure just announced the hiring of a new president. Dan Goldsmith is a software executive with experience in SaaS, international markets, and business-to-business sales, but not with education. He “will have broad responsibilities overseeing sales, marketing, product, customer experience, engineering and business development.” At the same time, COO Mitch Macfarlane is leaving. So this is a major reorganization. Taken together with the earlier departures of the company’s long-time sales and marketing executives, there has been a near complete turnover and reorganization of their top-level management.
To some extent, this is to be expected. People do start to cash out after an IPO. But the turnover and reorganization at Instructure seems quite significant—at a company that has stood out in the industry for having a tight, highly functional management team.
We’ll have more to say about these changes in the coming weeks. For now, I’ll predict that 2018 and 2019 will continue to be eventful in the higher education LMS space.
I’ve been struggling with how to characterize this year’s ASU + GSV Summit. Part of the problem is that it’s just become harder to sum up, for three reasons. First, the conference has gotten so big that it’s hard to see enough of it to get a representative sample. Second, as it has become a “must attend” conference for certain circles, it becomes harder for me to clear time away from meetings with people I know—both planned and unplanned—to be a fly on the wall. I can partly deal with that problem by bringing e-Literate colleagues who can be extra pairs of eyes and ears.
But the third problem is harder. As the conference and the culture around it mature, the trends become harder to identify because they’re no longer fashion. We’re past the Year of the MOOC and the Year of Personalized Learning and all of that. Some of the less patient money has left, and some of it has refocused on corporate talent management. (Good luck with that; I don’t think it’s any easier than edtech.) A lot of the hype has faded into the background. The conversations seem more real and more focused on tackling important educational problems in realistic ways.
Don’t get me wrong; there are still 12-year-olds in $300 jeans talking to 22-year-olds in $3,000 suits about how their math app is going disrupt education, and a few random dudes—and they are still often dudes, despite a sincere and concerted effort to make the conference more diverse—walking around wearing tinfoil headbands and loudly declaring, “I CAN READ STUDENTS’ MINDS!”
But it’s easy to get distracted by the spectacle and miss the big picture. One way to look at the summit is as a giant peer review machine. Yes, with so many submissions, the quality will be uneven. And yes, sometimes a bad product will make it through the peer review process. And yes, you may find the process and the culture to be alien and unsettling if you’re not from that world. But from what I can see, the peer review machine is getting better in the sense that the pitches from startups to investors increasingly seem to focus on things that actually make sense in terms of developing products that are educationally useful.
Coincidentally, I was reviewing some of the videos of the lightning talks from our Empirical Educator Project (EEP) summit this morning as I was thinking about this post. I was struck by how the presentation from RealizeIT, which is the type and size of company that would typically pitch at ASU + GSV, was not hugely different from the best of the pitches at that summit:
The main difference is that, for the EEP crowd, the presentation focused on the collaborative research with University of Central Florida and others to improve student impact, while a good GSV pitch might pivot off the impact visualizations to talk about how their evidence of impact has led to increased sales. Having the latter conversation between companies and funders is an important piece in the ecosystem that was missing before the ASU + GSV conference. Trying to have the former type of conversation doesn’t work easily in the same forum with the latter type, partly for reasons of cultural differences. In the early years of ASU + GSV, the conference organizers—again, with the best intentions—worked very hard to try to host both conversations, with the result often being strong negative reviews from the academics. In more recent years at the summit, the academic needs seem to be coming in more organically via the companies as intermediaries and self-selecting academics who are, for lack of a better term, bi-cultural.
The one trend that did stand out at ASU + GSV, which I think is related to the above, is a lot of activity around Online Program Management (OPM) companies. Something fairly deep is happening in this space. Right now, most of the buzz is around the debate over whether the future of OPM business models is in revenue share or fee-for-service. I think that’s a red herring, created in part by the bad blood between some individuals at Noodle Partners—one of the leading proponents of the fee-for-service model—and 2U—the darling of the revenue share crowd. The truth is that, while there is some overlap in competition, what we are seeing increasingly is a range of different companies, many but not all of which are currently labeled as “OPMs,” that offer different bundles of services and different financing models that are best suited for solving different kinds of problems. The common thread is that there is a growing range of service businesses that help colleges and universities develop strong, student-centered degree and certificate programs, either by creating new ones or strengthening existing ones. We don’t have a good term or set of subcategories for this class of businesses, of which traditional OPM is one subcategory. (And some of these business are pretending that they are not service businesses for a variety of reasons, but that’s another story.)
To me, this squishiness in the OPM market is a sign that more and more schools are serious enough about learning to be systematically better at supporting student success that they are increasingly willing to pay for help. And that change in prioritization is potentially good for everyone. And ASU + GSV has become a good vantage point for spotting that kind of change—if you look hard enough.
A few weeks ago, I posted about our case study on the governance of the Courseware in Context (CWiC) Framework. CWiC is a laudable effort to take the traditional product evaluation matrix and get beyond “magic quadrant-ism” by adding some educational context to the selection process. Which product is “best” depends a lot on your goals, capabilities, and other contextual factors. The CWiC advisory committee members explain how hard it is to integrate these factors into a product evaluation matrix. A lot of the hoped-for value of CWiC is still aspirational. There’s nothing wrong with that; it’s good to have big goals and to recognize how close to (or far from) achieving them you really are.
Sometimes just seeing your path to achieving those big aspirations is hard in and of itself. As this next case study shows, even if CWiC realizes its potential, it’s never going to be the right place to start if your goal is to meaningfully impact student outcomes and close achievement gaps. Our original goal with the National Louis University (NLU) and Acrobatiq interviews was to look at what it takes to implement courseware effectively with a vendor partner once the selection is made. In other words, what happens after you go through the process that the CWiC framework is meant to support. But we got so much more than that because NLU has been incredibly thoughtful about the program and students—the context—that impacts the use and value of the courseware. In other words, they also modeled what universities need to do before they select courseware, from designing a business/sustainability model that enables them to provide appropriate cost of an education to thinking about educational goals to figuring out where courseware does and doesn’t fit into that overall model. All of that prework has a profound impact on the working relationship with the vendor partner, the kinds of support identified as necessary to make the courseware work, and the future vision for how to continue to broaden and deepen the impact.
In the end, I think this case study shows both that a tremendous amount of work is needed to integrate courseware if you truly want to drive improvement in student outcomes and that the significant amount of planning and implementation work involved is only a small percentage of the total necessary program planning and work required. NLU has effectively redesigned the entire undergraduate experience. Their partnership with Acrobatiq works in large part because Acrobatiq understands their role in NLU’s larger goals, and because NLU maintains their focus on those goals rather than getting caught up in the notion that buying some magic product will solve all their problems.
I’m proud to have this be the final case study in this series of grant-funded stories. It neatly reflects a lot of what we believe about the ways in which universities and vendors need to work together if we want to improve higher education.
In my last post, I wrote about the goal of the Empirical Educator Project (EEP) as getting a critical mass of educators to be “socially empirical”:
Socially empirical educators view effective teaching not as an individual art but as a shared pool of knowledge and experience that everyone can learn from and contribute to. They seek out common vocabulary, methods, and standards of proof so that they can learn with their colleagues and raise the collective bar. This is the beginning of disciplinarity.
One way of thinking about the goal of EEP is to create a critical mass of socially empirical educators. They may not all arrive at one shared vocabulary, set of methods, and standard of proof. We will likely need a family of subdisciplines and empirical approaches to handle all the many meaningful contextual differences in education. But if a majority of educators see their work as embedded in a collective effort to learn together about how we can be more effective educators, then we will progress much more rapidly as a sector.
What is the value of doing this? Here again, the EEP founding cohort participants weigh in:
Too often, our conversations about education are weirdly bifurcated. On the one hand, we talk about big problems—helping massive numbers of people improve their economic status, educating a sophisticated citizenry, solving societal equity problems, and so on. On the other hand, we talk about the day-to-day work—picking tools and content, solving a developmental math program at a single college…maybe at most we have consortia working on shared problems within the group. But we only rarely and sporadically talk about the link between solving the local problems and tackling the grand challenges. We don’t have a plan for thinking globally and acting locally.
We at e-Literate contend that higher education is too focused on immediate peer groups, convinced that the most value will come from the people whose context is highly similar. There is some logic to that, given that effective education heavily dependent on many contextual details. But we seem to need a statement of common purpose around the educational part of the mission that is at once broad enough to accommodate differences and focused enough to bring together different types of institutions and individual stakeholders that have different strengths and resources to offer.
I won’t go so far as to say that EEP can offer that, but we aspire to create a collaborative environment out of which that shared vision can emerge by solving specific problems—acting locally—through collaborations that cross peer network boundaries.
We have the first batch of Empirical Educator Project (EEP) summit videos available in the form of a four-episode interview case study. You can see the full set here. I’ll be blogging about the series over the next week or so, starting today with the first episode on the definition of an empirical educator.
A lot of the currently popular terms of art that bear some relation to “empirical educator”—efficacy, evidence-based, research-backed, data-informed, etc.—tend to produce discipline-inflected responses (e.g., “It makes no sense to talk about ‘data” when you’re teaching English composition”) or suspicion due to vendor, funder, and media hype (e.g., “‘Efficacy’ is just the latest marketing buzz phrase”). And many of these terms tend to emphasize products or implementations. We wanted to put the focus on people, and we wanted the term to be inclusive of a wide range of practices that are discipline- and context-appropriate. So we introduced a new term, which always requires some norming to get people rallied around a common understanding of it.
Here’s what a sampling of summit participants had to say about the meaning of “empirical educator” after a day of work together:
The definition becomes increasingly inclusive as you listen to the complementary perspectives over the course of the video.
I think we’ll develop a more rigorous, evidence-based understanding of what it means to be an empirical educator over time—while hopefully maintaining the inclusive spirit. For now, I’ll put out my own taxonomy of levels of empirical education:
Intuitively empirical: This comes down to whether you pay attention to your students and do something differently with them based on what you observe. Do you always do the same thing, or do you have a bag of tricks that you can draw from when you see students struggle? I believe that the substantial majority of educators are empirical in this sense. They may not think of it as empiricism, but they are observing student behaviors and are adjusting their strategies based on what they see, guided by some sort of rationale for choosing which strategy to employ in different circumstances.
Mindfully empirical: Mindfully empirical educators think about how they can get the maximum amount of useful diagnostic information from day-to-day course work. They design their courses with a goal of creating many feedback loops that enable them to be adjust their teaching to the needs of the students.
Meta-cognitively empirical: Meta-cognitively empirical educators are empirical not only about how they use their existing bag of tricks but also about which tricks they should have and how effective those tricks really are. They consciously and regularly test their own assumptions about effective teaching, and they are open to trying new appropaches. My read of Lauren Herckis’ research is that the barrier of moving from mindfully empirical to meta-cognitively empirical (and to the next level, socially empirical) is where a lot of the difficult work needs to be done. Lots of educators are intuitively empirical, and the transition from there to mindfully empirical is not a huge leap. Getting them to test and challenge their deeply held beliefs about what constitutes effective teaching is a lot harder.
Socially empirical: Socially empirical educators view effective teaching not as an individual art but as a shared pool of knowledge and experience that everyone can learn from and contribute to. They seek out common vocabulary, methods, and standards of proof so that they can learn with their colleagues and raise the collective bar. This is the beginning of disciplinarity.
One way of thinking about the goal of EEP is to create a critical mass of socially empirical educators. They may not all arrive at one shared vocabulary, set of methods, and standard of proof. We will likely need a family of subdisciplines and empirical approaches to handle all the many meaningful contextual differences in education. But if a majority of educators see their work as embedded in a collective effort to learn together about how we can be more effective educators, then we will progress much more rapidly as a sector.
I think it’s also important to note that the term “educator” is intended to be more inclusive than just people who work in classrooms. Although instructors will be central change agents in any empirical education movement, anyone who is directly involved with designing or delivering a meaningful educational intervention is included in our definition of an educator. This means program administrators, instructional designers, and product designers, among others, can all be empirical educators.
We’ll have some more information to share about the Empirical Educator Project (EEP) next week, beginning with an ELI webinar on Monday at 1 PM ET. Phil and I will be moderating a star-studded panel of EEP participants:
Curtiss Barnes, Managing Director, Global Product Management & Design, Pearson
Update: I got feedback that it wasn’t clear the embedded video was a three-video playlist, so I have broken them out individually.
We have a new case study up in which we interview members of the Courseware in Context (CWiC) framework advisory committee about their work:
The focus isn’t so much on the framework itself as on the project goals and, somewhat less directly, it’s sustainability model. We’ve seen repeated failures in the market to create selection tools for curricular materials or edtech products. (Some of us are old enough to remember EduTools.) The committee members are very thoughtful about the missing element, which is context. There are, in CWiC advisory committee member Tanya Joosten’s words, “a million” different contextual variables, from classroom implementation to support to individual student needs. CWiC is an attempt to take a traditional product selection tool model and enrich it enough to account for all these contextual variables.
The committee members seem acutely aware of how big of a challenge this could be, which is part of what makes these interviews so interesting. At our recent Empirical Educator Project summit, René Kizilcec (whose research we have featured here on e-Literate) made a fascinating comment, which is that we education “desperately needs” a “science of context.” The CWiC folks are trying infuse the output we currently have from the beginnings of that science into what is essentially a tool for selection and procurement. Will it work? I don’t know. The committee members are pretty clear about the size of the challenge. But it’s the right direction.
The Empirical Educator Project (EEP) attempts to take on the other side of René’s phrase. What would it mean to have a science of context? Who would the scientists be? What cultural institutions would we need to form a consensus around what a science of context would look like such that people who care about it could cooperate on it more effectively, and such that it would attract the funding it needs to be carried out? What business process support tools, including but not limited to procurement tools, do we need to implement a science of context, and how would implementation provide input back to the scientific research? These are the questions that EEP attempts to answer. Will it work? I don’t know. But I feel confident that it’s the right direction.
