e-Literate

Present is Prologue

Category: LMS & Learning Platforms

Everything you want to know about Learning Management Systems and whatever comes after them.


  • A Note on Data Used for LMS Market Analysis

    A Note on Data Used for LMS Market Analysis

    Recently Martin Dougiamas of Moodle has questioned our data analysis for the LMS market. In some useful notes posted at Moodle.com on two recent Future Trends Forums hosted by Bryan Alexander:

    Bryan finished the Future Trend Forum by asking for Martin’s thoughts on the recent article by Phil Hill titled: “Whither Moodle?” [edited] which speculated that Moodle’s growth is slowing down and hitting a plateau.

    Martin commented that is not the case from what he is seeing and that a lot of the information contained in the article is US-based where there a lot of more LMS vendors and venture capitalists building learning platforms.

    While this description from Martin is inaccurate, the issues raised are representative of some of the questions we occasionally get about our data for our LMS market analysis service. I think it would be useful to share a deeper description in public of how our partners at LISTedTECH collect and organize the underlying data.

    What We Measure

    The market data are organized in a dataset that captures system usage on a per higher education institution basis. For most schools, a campus is equivalent to an institution. But there are also cases where there multiple campuses per university (e.g. University of Minnesota system with five campuses, or DeVry University with dozens of campuses) and the LMS decision is made at the system level. In these situations, one decision will lead to multiple institutions listed in the data. In the US, the definition of an institution is guided by unique identifiers in the Department of Education’s IPEDS data, and each region or even country has its own way of defining institution.

    The dataset goes beyond “school X uses system Y”, as it also includes dates of implementation and decommission, usage as primary or secondary system (there may be more than one system in use at a school), and hyperlinks to the public information documenting a system selection or usage. The definition of institutions includes information about its sector (public two-year, private non-profit four-year, etc) as well as student enrollments.

    How We Measure

    Looking deeper at LMS selection, there are multiple layers of data gathering at different intervals. Some of the sources:

    • Extensive search engine notification such as Google Alerts on product keywords in multiple languages;
    • URL and domain scrapers looking for system information at official school websites; and
    • Targeted human-directed searches.

    Each new data point is verified by someone using the associated hyperlinks tied to selection or usage data.

    Our North American data is essentially saturated, in that we know the vast majority of degree-granting institutions based on US Department of Education data or Canadian provincial governmental data. We have well above 90% of all schools in the dataset.

    For the global regions outside of North America, we are building up the dataset and do not have saturated coverage yet. For example, in Europe we estimate that we have 60 – 75% of institutions. We have less than that in Latin America and more than that in Oceania.

    Where feasible, we include on-the-ground subjective coverage by visiting the global regions, testing theses, finding out unique context, and finding local sources who can provide QA to our data.

    Besides our home base of North America, we have made multiple trips to Europe and Latin America thus far, and we are currently arguing about who gets to visit Australia and New Zealand.

    We plan to expand coverage to additional regions as we develop at least 30% coverage of institutions and have time to do additional research to back up our analysis.

    Degrees of Uncertainty

    Because higher education data is lumpy and based on extended implementation times, we offer the following caveats:

    • Market share information provided in percentages and trends are more reliable than absolute counts outside of North America. When we do provide absolute numbers, we advise caution for readers or subscribers to not over-interpret the absolute numbers, at least without us providing additional details to keep the data in context.
    • We typically separate North American data from Rest of World data (Europe, Latin America, Oceania) to avoid problem of North America numbers dominating aggregates and obscuring important regional differences.
    • When we have system usage information but do not have accurate implementation dates (per month or quarter), we assign these system records to June. Therefore the summer data for new systems will appear artificially high. We currently have implementation dates for approximately 75% of the listed LMS records.
    • Put another way, annual data is more reliable (i.e. without additional data collection noise) than half-year or quarterly data. The more-granular data is provided to certain subscribers, but we take great care in attempting to describe sources of “lumpiness” in the data that should be understood for any analysis.

    Overall, we have LMS data for 4,523 institutions in the US and Canada and 8,824 institutions worldwide.

    Back to the Future (Trends)

    To see these issues with an example, consider the updated chart of new implementations that led to the Future Trends Forum discussion described above.

    What is relevant to the dataset for this chart:

    • The data is based not on North American data – it is based on data from Europe, Latin America, and Oceania (Australia, New Zealand, and surrounding island countries).
    • The data comes from public sources per institution as described above and does not come from vendors;
    • The data is for primary systems – the official campus LMS; and
    • The caveats listed above should apply. Note that we identified a new trend early in 2017 (collapse of Moodle new implementations) that we can update with data through 10 months of the year – the data today is more solid than it was in early summer.

    I hope this description will answer some of the questions people have asked about our data.

  • State of Higher Ed LMS Market for US and Canada: Fall 2017 Edition

    State of Higher Ed LMS Market for US and Canada: Fall 2017 Edition

    Now that we’re past the 2017 WCET conference and headed to the EDUCAUSE conference, let’s look at updates on the institutional LMS market for North America (US and Canada). Last year we started our LMS market analysis service, where we are working with LISTedTECH to provide market data and visualizations, and our fall report for subscribers will be released in about a month. Data for 2017 presented below goes through October 1 of this year.

    We present the data “by institutions”, with market share as a percentage of the total number of institutions using each LMS as a primary system, and “by enrollments”, where we scale the institutions by their total enrollment. The latter better captures the business of the LMS market, since most licensing deals are based the number of students. We have also included data from Fall 2016 by institutions for comparison purposes.

    Some notes on the market share as defined by percentage of institutions:

    • In terms of new selections the market continues to be a two-horse race recently with Canvas by Instructure and Brightspace by D2L as the only two solutions with material gains in market share. Canvas grew from 17% to 21% , and Brightspace from 11% to 13%, of the market.
    • Outside of the big four – Blackboard Learn, Canvas, D2L Brightspace, and Moodle – no other system has more than 3% of the market measured either by percentage of institutions or percentage of enrollments. Furthermore, the category of “Others”, capturing systems not listed above and having less than 1% market share, actually dropped from 4% to 2% of the market. This indicates that continued market consolidation.
    • Schoology and BNED LoudCloud grew slightly, but not nearly to the degree of the Canvas and Brightspace.
    • Blackboard Learn fell the farthest, from 31% to 28% of of the market. The second biggest drop was Pearson LearningStudio, from 4% to 2%, based on its end-of-life notice for December 2017.
    • Moodle and Sakai both lost market share of just under 1%, not enough to show up in the rounded numbers in the table but enough to show up in our underlying data.
    • The difference in Moodle’s market share by institutions at 25% and by enrollments at 12% really shows how concentrated their usage is for smaller schools.
    • When viewing market share as defined by percentage of enrollments, Blackboard and Canvas are the most-used systems, at 37% and 27% of the market, followed by D2L at 15% and Moodle at 12%.

    Stepping back from the immediate data, let’s look at an updated LMS market share graphic, commonly known as the squid graphic, for US and Canadian higher education. The original idea remains – to give a picture of the LMS market in one page, highlighting the story of the market over time. The key to the graphic is that the width of each band represents the percentage of institutions using a particular LMS as its primary system.

    Some additional notes:

    • Blackboard and Moodle have been the market leaders as number one and two in share as they have been for the past decade. However, Canvas is quickly approaching Moodle’s market share.

    We’ll share any updates we discover at the conference this week.

  • Unizin Membership Now Set As Annual Fee Of Up To $427.5k

    Unizin Membership Now Set As Annual Fee Of Up To $427.5k

    I’ve been meaning to provide an update on Unizin now that the consortium is three years old (started officially in July 2014). Thanks to public documents from the University of Minnesota, one of the 11 founding members, we now have additional clarity on the ongoing costs to remain a member of Unizin.

    Membership Fees

    For some background, Colorado State University staff back in April 2014 described the $1,050,000 initial fee in their meeting minutes for the University Technology Fee Advisory Board:

    3. Will this decrease overall costs on our end through collaboration?
    a. We are investing $1 million up front, but there is about a 7-year payback. We are investing in a $10 million product since the other 9 universities are putting their money in as we are too. This will absolutely decrease our costs.

    One year later, when the Florida State University System joined Unizin as associate members, we noted this item from the University of Florida / Unizin Consortium Membership Agreement:

    We noted at e-Literate in our article from 2015:

    Does this mean that founding institutions that “invested” $1.050 million over three years will have to start paying annual fees of $100,000 starting in June 2017? That’s my assumption, but I’m checking to see what this clause means and will share at e-Literate.

    Update (7/17): I talked to Amin Qazi today (CEO of Unizin) who let me know that the annual membership fee for institutional members (currently the 11 schools paying $1.050 million) has not be determined yet.

    Fast forward to 2017 and we have an answer. The University of Minnesota has to submit purchases over $1 million to its board of regents for consent, and at the July 2017 meeting the new Unizin membership fees were presented:

    To Unizin, Ltd. for $1,282,500 for a three-year renewal of membership in the higher education consortium for the Office of Information Technology (OIT) for the period July 1, 2017, through June 30, 2020. The annual payment of membership fees will be covered from OIT’s central O&M funds. The FY18 budget includes planning and funding for this expense.

    That equals $427,500 per year for the next three years for the 70,000+ enrollment university. What this now makes clear is that the up-front investment in Unizin was not a one-time fee broken up into three easy payments. Unizin member has an ongoing annual fee set in three-year periods.

    I again asked Amin Qazi for clarification, including whether all Unizin members were now paying the higher fee ($427.5k vs. $350k for initial three years). Amin confirmed via email:

    Unizin is a non-profit organization and seeks to cover its costs. We have found that our cost to provide our services and tools somewhat scale with the size of the institution. The Unizin Founding Member Fees have been adjusted after the initial three year period. So while larger institutions do pay more, smaller institutions pay less. We anticipate further adjustments as we grow and are able to recognize even greater economies of scale.

    I would then assume that the University of Minnesota, along with University of Michigan and Penn State University, are paying at the highest level and more than $350k, and that smaller schools like the University of Iowa and the University of Nebraska are paying less than $350k.

    LMS Fees

    The same University of Minnesota document also describes their costs for the Canvas ((Disclosure: Instructure is a subscriber to our market analysis service.)) LMS based on the Unizin agreement.

    To Unizin, Ltd. for $5,023,000 for a purchase of Canvas Learning Management System (LMS) for the Office of Information Technology (OIT) for the period July 1, 2017 through June 30, 2022. [snip]

    Unizen [sic], on behalf of its member institutions, conducted a competitive Request for Proposal followed by a detailed evaluation process. Through this process Canvas by Instructure was selected as a Learning Management System (LMS). The University then conducted a two year pilot of Canvas and a majority of the stakeholders preferred Canvas to the University’s current LMS, Moodle. Most of the Big Ten schools have adopted or are adopting Canvas.

    The University receives an additional 30% discount by purchasing Canvas through Unizen [sic] rather than purchasing directly through Infrastructure [sic] and 3% caps on annual increases, rather than 5%, has been negotiated.

    This five-year deal comes out to $12 – $14 per student per year. The document does not specify what level of support they have chosen, although they describe a “dedicated test server”.

    New Associate Members

    In other news, Unizin announced in July that the University of Nebraska system has joined as associate members.

    The Unizin Consortium is thrilled to welcome the full University of Nebraska system, bringing the total number of institutions in the consortium to 25. With the addition, the University of Nebraska at Kearney, University of Nebraska at Omaha, and University of Nebraska Medical Center join Unizin Founding Member the University of Nebraska Lincoln.

    Note that associate members do not pay the same amount as full members. In Florida, the State University System deal costs each associate member $100k per year.

    We’ll likely give updates at e-Literate after the EDUCAUSE conference on how Unizin has evolved in terms of services and potential new members. But for now we at least have more clarity on the financial terms of the consortium.

  • University of Wisconsin System to Migrate From D2L Brightspace to Canvas LMS

    University of Wisconsin System to Migrate From D2L Brightspace to Canvas LMS

    In one of the most significant LMS selection projects of the past few years, the University of Wisconsin System (UWS) has chosen to migrate from D2L’s Brightspace to Canvas as its centrally-supported Learning Management System (LMS). ((Disclosure: UWS, UW Madison, Instructure, and D2L are all subscribers to our market analysis service, and we aided UWS in the needs analysis portion of this project.)) UW Madison already moved to Canvas as part of its Unizin membership, but now the rest of the 180,000 student, 26 campus system will also make the change.

    The decision was first noted on the UWS procurement portal and in a investor analysis note from Raymond James. A representative from UWS confirmed the news and added that “Canvas has been issued the Notice of Intent to Award and a final contract is going to the UW System Board of Regents for formal approval in October”.

    The UWS project page describes the process leading up to the LMS selection, starting with needs analysis kickoff in 2015.

    The Learning Environment Needs Analysis (LENA) project was undertaken as a continuation of a multi-year UW System effort to: 1) understand the current and future learning technology landscape, 2) uncover the wants and needs of UW System institutions with regard to academic technologies that support teaching and learning, and 3) identify gaps that exist in supporting teaching and learning through academic technology. The results of the LENA project were presented to the Learn@UW Executive Committee, along with a recommendation that the Committee charter the process for planning to move into a next generation learning environment for the UW System. The intention was that through this process, UW System would discover potential paths forward to support such an environment.

    Last year UWS developed the request for proposal (RFP) requirements list, and the formal RFP was released in January of 2017.

    Beyond the size of the system, UWS decision is significant due to it being the first major customer of Desire2Learn (as the company was known prior to 2014). Back in 2002 / 2003, most LMS decisions were framed as Blackboard vs. WebCT, and when UWS selected little-known D2L, it sent shock waves through the market. The decision really put D2L on the map as a true contender, and they followed up with wins at the University of Iowa ((Disclosure: In previous consulting company I advised U Iowa on their LMS selection.)), the Ohio State University, Minnesota State Colleges and Universities, and the University System of Georgia.

    We have noted several times at e-Literate and as part of the e-Literate Big Picture: LMS market analysis service that D2L has an impressive record of client retention. The company has been a fierce competitor in keeping customers, as seen when the Colorado Community College System recently chose to remain on D2L Brightspace after their LMS selection process. ((Disclosure: CCCS is a subscriber to our market analysis service. I also advised CCCS when they originally chose D2L in 2008.)) This loss of UWS is the biggest setback for the company in terms of losing clients, and it is a major win for Instructure’s Canvas system.

    Expect more market news to come out in the next month based on WCET and ECUCAUSE conferences.

  • Response on LMS Market Size Analysis

    Response on LMS Market Size Analysis

    Josh Kim wrote a post yesterday at Inside Higher Ed questioning some claims about the LMS market size.

    The first result that comes up in Google is from Zion Market Research. Their estimate is that the global LMS market is valued at $5.19 billion in 2016, and expected to reach $19 billion by 2022.

    According to Statista, the LMS market in 2016 was valued at $3.21 billion. CNBC claims that the learning management system market is worth $5.2 billion today, and will grow to $16 billion in four years.

    Can these numbers be right?

    Josh then did a quick analysis based on Instructure’s revenue and came up with much smaller numbers.

    If we assume that the cost of Canvas is about average for LMS companies, we can get an estimate of the total market size by multiplying Instructure’s revenues by five. This gives us an estimate for the total size of the LMS market $555 million.

    He extended this by 50% based on Instructure growth to give an estimate of $555 – $832 million, which is way below the big numbers claimed by Zion, MarketsandMarkets, and other market analysts.

    The biggest difference here is that all of these analysts combine corporate learning markets with academic LMS usage, and corporate learning platforms account for somewhere between $2 – $4 billion, with best estimates (although outdated) coming from Bersin. This point has already been pointed out in the comments at IHE.

    tl;dr – the ~$5 billion number is reasonable if you combine academic and corporate LMS markets.

    There are two caveats, however.

    • First, the market growth claims are marketing fluff to generate interest in buying the reports, as companies love to show growth potential to investors. Anything close to $20 billion in a few years is ludicrous.
    • Second, as Michael commented on this approach based on a post I wrote criticizing MarketsandMarkets:

    If anything, my Phil understates the case here. Lumping higher ed, K12, and corporate LMSs into the same category is a little bit like lumping railroad cars together with automobiles because they are both called cars, have wheels, and carry things and/or people from one place to another. On top of that, nobody has decent data on the size of the global market, never mind the growth of it. MarketsandMarkets’ “analysis” effectively gives us made-up numbers about a mythical automobile/train car market.

    So a better question is what is the size of the global academic LMS markets, combining K-12 and postsecondary?

    One issue from the IHE post is that Josh used our 20% market share as basis for extrapolating Instructure’s market share, but that number is only for North American higher education. Globally, Instructure’s market share is much smaller as shown in our recent analysis (data provided by our partners LISTedTECH).

    Furthermore, while Canvas by Instructure and Schoology seem to be leading vendors for new adoptions in the K-12 market, at least in the US, no one has reasonable measures of market share in this area. Long and short – we need a different approach than extrapolation from Instructure alone.

    LISTedTECH estimated the global LMS market just in higher education as $1.7 billion based on (# of students) * (average cost of LMS per student) = annual spending approach. In the US, the best estimates I’ve seen in that K-12 learning platform markets are worth 30 – 40% of higher ed market. This likely is smaller globally, as anecdotally there is not a big emphasis on LMS usage outside North American and Northern Europe. So one very rough estimate is that academic LMS market is worth approximately $2 billion per year.

    If you want to use the company revenue method, then I would use additional data. Blackboard’s LMS revenue is likely $450 – $550 million per year based on reviews of corporate debt filings and extrapolation from their time as a public company. D2L’s revenue is smaller than Instructure’s, so let’s assume the two companies combined have $200 million revenue. Note that in all of these companies this is global revenue for K-12 and higher ed combined. The question is how to scale this $700 million combined revenue from all three companies. Is it reasonable that global spending for academic LMS could be 2 – 3x larger than this number? When you add in all of the Moodle hosting – both Moodle Partners and unofficial Moodle hosting – as well as Chinese market adoption, European providers, and K-12 players like Schoology and Powerschool/Haiku, this seems at least believable.

    So my highly-researched, thoroughly-documented estimate of the global academic LMS market is $1.5 – 2.0 billion per year. And it is forecast to grow to $8.2134526 billion by June 3, 2022.

    Note to investors – ignore that forecast.

  • InstructureCon 2017: Culture as a competitive weapon

    InstructureCon 2017: Culture as a competitive weapon

    In our recent coverage of BbWorld and D2L Fusion, Michael and I came away impressed. Both companies are improving and doing so in ways showing them listening to customers, empathizing, and developing product enhancements with more engaging user experiences. Both companies are doing so by attempting to change their company cultures to support these behaviors. Their challenge, however, is that Instructure is already there – their company culture has supported simplicity, customer-centric empathy, and openness from day one. And what struck me (again) at this year’s InstructureCon was how well the company is managed and how consistent is their approach, over time and throughout the organization.

    Instructure launched its Canvas LMS back in 2010, and it is remarkable how Michael’s description from June 2010 could still be applied today.

    If I had to summarize Instructure’s strategy in one sentence, it would be “They use the lessons learned by consumer web companies to clear the clutter out of LMS software design and business model.” They’re not focusing particularly on open education or analytics or any other hot topics in online education, although they are aware of these and do pay some attention to them. Rather, they are looking at core use cases and trying to make them as simple as possible, throwing out some outdated LMS design assumptions in the process.

    From the early days when they had dozens of employees to today, when they have over 1,100 employees, the strategy is the same and the company culture is the same. This despite the departure of both founders and the 2015 IPO and all the resultant pressure from investors.

    This is Instructure’s secret. They continue to manage the company to support the strategy of clearing the clutter out of LMS software design and business models.

    Product News

    The biggest product news at InstructureCon this year:

    • Release of Gauge, an assessment management platform for K-12 usage, which might impact higher ed in the future
    • Redesigned Quiz engine and Gradebook, leading to what we saw as the biggest interest from conference attendees
    • Release of blueprint courses
    • Investment in back-end platform design (e.g. microservices) to increase speed of design improvements
    • Reintroduction of analytics – visualizations initially targeted at faculty usage
    • Introduction of Alexa integration

    The focus on assessments, quizzes and grade books is both problematic and encouraging. Problematic for the industry as we spend too much time on measurements and grading and accountability and too little time on learning and continuous formative assessment. Yes, the quizzing engine can be used for formative assessment, but in most cases the usage is for summative assessment. But even with this viewpoint, it is encouraging that the primary focus of these product enhancements to Canvas is on reducing the time spent on the grading activities. It is a fact of life that an enormous amount of effort is spent by faculty, particularly in the US, on managing grades. It is good to see tools that directly reduce the pain in this area.

    In a standing-room only session, Instructure’s Christi Wruck explained the initial phase of a multi-year effort to redesign and improve the grade book. Most features shown were of the nature of reducing the number of clicks or simplifying the navigation of grading activities. I won’t go into the minutiae here, but this session seemed to generate the most excitement at the conference.

    The (re)introduction of analytics surprised me, as I had seen most public efforts in Canvas lately focusing on getting data out to let customers do their own analytics. The initial analytics in Canvas were introduced in 2012, and there were and are a few cool visualizations for faculty mostly, but there is little meat on the bones. There never seemed to be a real product strategy on the developing role of analytics within an LMS nor a full team to work on this challenge. In the meantime they rolled out Canvas Data, which we have talked to staff from several colleges and universities who are actively playing with and exploring the data. But this year Instructure described how they’ve been hiring a new analytics team and working on new visualizations. As with the previous effort, the initial focus is on helping faculty get a better view of how students are performing. We will have to watch this current focus on analytics to see if a real strategy develops.

    Blueprint Courses is a overdue features set to allow course design templates or components across courses or shared between instructors. This capability finally provides direct support for centralized course design models involving master courses and multiple course shells as typically used in large online programs. While this capability could be considered late, conference attendees seemed to be quite impressed with the design of the new features.

    Market News

    I shared in my post on D2L’s Fusion conference how their Brightspace LMS has made impressive wins recently, particularly for schools with large online programs. Canvas continues to gain market share and lead all competitors in new implementations (schools choosing a new LMS), but over the past 12 months they’ve allowed D2L to at least become a clear competitor.

    Michael also shared a view from the North American (US and Canada) market for higher education showing that most of Blackboard’s “wins” over the past few years have come from ANGEL end-of-life migrations. Looking at that same image, however, it becomes even more obvious just how well Canvas has performed for new implementations. It is not just that Canvas has the lion’s share of new implementations, it’s also the breadth of migrations coming from Blackboard Learn, Moodle, ANGEL, LearningStudio, Sakai, and even several from D2L Brightspace.

    Instructure has recorded strong international grown in just the past two years, and this interest was evident at the conference. The most notable difference between the early users conferences and this year’s event was the expansion of the customer base. There must be similar numbers of K-12 attendees as there are higher education attendees, and we’re now seeing clients from Europe, Latin America, and Australia & New Zealand.

    Just 2-3 years ago Canvas as an LMS brand was virtually unknown outside of the US and Canada, but now Canvas is winning more than any others in North America, Europe, Latin America, and Oceania.

    Challenges

    Instructure certainly has challenges to face. In terms of market share, it will be very interesting to look at data for the end of 2017 to see if D2L has maintained its momentum in new implementations. And being a publicly-traded company, investors care not just that Instructure is growing but also whether Instructure is growing fast enough in multiple markets. That’s a lot of pressure.

    On the product front, Canvas was late to the game in terms of supporting mastery learning and associated competency-based education initiatives, although their introduction of MasteryPaths is showing some ability to catch up. Likewise, they certainly are behind their competitors in terms of learning analytics and associated research. And I would even say that they are late in supporting centralized course design models. In all three areas, there are initiatives introduced over the past 12 months to address these product needs. The challenge is for the company to catch up with competitors not necessarily in terms of full feature sets, but more in terms of supporting basic use cases and doing it well.

    When I asked various Instructure management at the conference whom they saw as their greatest competitor, the most common answer was themselves. The company knows they have work to do, and there is a real sense of “we have to do better” that permeates the company. For Instructure’s competitors, the good news is that Canvas has weaknesses. The bad news is that the company knows it.

    Back to Culture

    But the real event of InstructureCon is not really based on product news or market share. As Michael noted last year:

    Throughout the conference, I asked a number of attendees what percentage of their motivation for coming was to find out what the company was planning to develop in the next year. The range of answers averaged between 10% and 20%. They all told me that, while they were looking forward to the roadmap sessions later in the week, they didn’t expect any big surprises and weren’t all that focused on finding out what the developers had been doing. I have never been at an LMS conference where that was true, including Sakai conferences and Moodle Moots.

    Consider the Instructure company profile at Glassdoor. Last year the company was rated #4 best place to work for all small and medium-sized businesses, and Josh Coates was rated #15 for all CEOs in the same category. But also look at the video that Instructure management chose to include in the “Why Work For Us?” section. Other companies tend to put up platitude-filled dreck that tells you very little about their company. But Instructure puts up a 7+ minute video where they talk primarily about company values. In the video, they talk about the behavior the company does and doesn’t want – and even nods that they’ll get rid of people who have have these behaviors – and they avoid all the pseudo-inspirational messages.

    While Canvas is a market-changing platform, that is not really Instructure’s primary strength. Instructure as a company is the real strength.

  • BbWorld Report: Blackboard May Be Turning Around

    BbWorld Report: Blackboard May Be Turning Around

    We’ve written similar headlines after past BbWorlds only to be disappointed, so it’s prudent to be cautious. We also need to be clear on what “turning around” does and does not mean. That said, this time feels different. ((Disclosure: Blackboard is a subscriber to our LMS analysis service.))

    Kinds of Evidence of LMS Supplier Health

    Given the cautions above, it’s worth taking some time to look at the types of evidence we gather and what each type can or cannot tell us before diving into the conference analysis:

    1. Changes in adoptions and market share: For investors and competitors, these are the measures you are trying to predict (from among the measures that we usually talk about at e-Literate). The other measures just help to anticipate changes in these ones. For customers and prospective customers, these indicators are useful but less dispositive. They provide a reasonably good sense of how stable the provider is and how well received the product and provider combination are being received by the wide world of current and potential customers. For all audiences, they are trailing indicators. They provide hard, objective data about decisions that colleges and universities have made but not about decisions that they are about to make.
    2. LMS evaluation processes and RFPs: We can learn a lot about imminent change in adoptions and market share by what happens when colleges and universities start LMS evaluations. Which LMS vendor’s current customers are going out to bid most often? When they go out to bid, which LMSs do they decide to evaluate seriously and which ones do they skip? What kinds of questions do they ask? How well do the vendors respond to the evaluators’ questions, and what do the evaluators make of the vendors’ answers? The evaluation data aren’t always good predictors of how happy the schools will be with their choices—a lot depends on how well they run their evaluation processes—but they do give us some indications about how well the vendors understand their customers’ and prospective customers’ needs and perspectives, as well as some information about how well they are executing as a company. This can also be something of a lagging indicator for current and prospective customers because real substantial changes in the company are generally transmitted from central management outward and can reach the sales force last with the use of professionell coaching to raise sales.
    3. Customer sentiment: Are customers happy? Do they feel like their supplier is responsive? Have they noticed a change in responsiveness (good or bad)? How have their recent experiences been with new releases and support services? Customer sentiment tells us how the company is performing for customers right now, but it’s hard to gather in more than an impressionistic way.
    4. New features and other anouncements: There are two primary types of questions these announcements can help answer for current and prospective customers. First, is the supplier filling gaps or fixing problems that will impact customer satisfaction (and therefore demonstrating awareness of the areas where they are underperforming)? Second, what does the pattern of announcements tell us about where the supplier think customers’ new and future needs will be and which of those needs they think they can fulfill? New announcements are a leading indicator of company direction. We will occasionally provide our initial opinions about the quality of the new features, but those should be taken with a grain of salt. We don’t believe we can get a reliable read on the “quality” of any feature until a variety of customers have used it in real-world situations.
    5. Management public and private presentations and discussions: This is probably the most subjective but also potentially the most revealing leading indicator of both company focus and likelihood that their quality of execution will improve or deteriorate. It has the most value when it is interpreted in the context of the previous four types of indicators.

    Of these indicators, we don’t get a lot of new information on the first two at LMS conferences. We get our data on the first one primarily through our data partnership with LISTedTECH and the second one primarily through a combination of the LISTedTECH partnership and our experiences consulting for colleges and universities on their LMS RFP processes. So I’m going to give a brief(ish) summary of these two and then spend the bulk of the post on the last three.

    Context: Adoption and RFPs

    Let’s be clear: In the United States and Canada, Blackboard is playing defense. For now, they are focused on reducing the number of current clients who go to RFP and, of those who do, increasing the percentage who stay with Blackboard. Winning new clients is something they’d like to do, of course, but they’re more focused in the short term on not losing clients. Everybody knows it. Multiple senior Blackboard executives acknowledged the fact or even volunteered it to me on the record at the conference. They are not getting many new implementations:

    New Implementations NA CC

    And of those new implentations they are getting, most are conversions from their legacy ANGEL platform:

    Chord NA CC

    In terms of evaluations, we haven’t seen evidence of Blackboard bottoming out yet. When non-Blackboard customers go to RFP, many of them don’t include Blackboard on their candidate short list. Those that do generally don’t pick Blackboard. (I’ll have more to say on this later in the post.) Meanwhile, Blackboard customers continue to go to RFP. A subset of those have already decided that they will not consider Blackboard. This should not be interpreted as a clear sign that Blackboard isn’t improving; some customers just reach the end of their patience and are no longer persuadable. What it does suggest is that, if there is substantial improvement, it is relatively recent. Nevertheless, we’re not yet seeing Blackboard change their win rate the way we are beginning to see it with D2L.

    There are two major caveats to all of this. First, Blackboard scored a major win with the University of Phoenix’s new adoption of not just Blackboard in general but Ultra in particular. ((Disclosure: University of Phoenix is a consulting client of MindWires.)) While the university is no longer the juggernaut that it once was, it is nevertheless still huge. We will continue to consider them a prospective client until they have actually migrated at scale. Blackboard was able to announce that from the main stage at BbWorld the University of Phoenix will be migrating to Ultra in the fall, which indicates progress, but the proof of the pudding is in the eating.

    The second major caveat is that we are about to enter a new school year and, with it, a new round of RFPs. The migration pattern may change going forward. This is where the forward-looking indicators at the conference may provide us with some clues.

    Customer Satisfaction

    This BbWorld seemed to lack the same simmering discontent that characterized their conferences in the recent past. The attitude seemed neutral-positive. I heard consistently that more problems are being fixed with 9.x and the feature gap that have caused some schools to pass on Ultra is narrowing. There had been some issues with early migrations to SaaS that scared some customers away from considering it for the time being, but nobody I talked to was ruling it out; they were just waiting to be sure that the kinks were worked out first. Schools who migrated more recently seemed to have a better time of this. By the way, the migration issues were brought up by Blackboard executives on the main stage. As with the admission of playing defense on keeping customers, this is an example of a new public honestly that I observed from the company. I’ll have more evidence of this later in the post.

    How many customers have migrated to SaaS? Blackboard claims a little over 200 have made the switch so far with over a hundred more either planning to migrate or piloting:

    Screenshot 2017-08-12 12.52.25

    So that’s a good sign.

    By the way, here’s a third data point in terms of Blackboard’s honesty: One Blackboard executive, after bringing up the AWS competency certification that’s noted on the slide above, volunteered, “That’s not a differentiator. Our competitors have this certification too. It’s more that it would be a bad sign if we didn’t get it.”

    Huh.

    Late in the conference, I was able to speak with several customers who had had private meetings with Blackboard during the week. The common themes were improvement, honesty, responsiveness, and not-there-yet-but-getting-there-pretty-fast.

    I have one other observation that doesn’t fall squarely under the heading of customer satisfaction but is related and also foreshadows some of the other observations I’m going to cover in this post. I went to several analytics sessions at the conference, including one at the Moodlemoot—yes, there was a Moodlemoot inside BbWorld; more on that later—and the discussions were interesting. I can’t remember being at an LMS conference where the Q&A portion of the analytics presentations were fulsome debates about the value, adoption, and ethics of learning analytics rather than on product features. But that’s exactly what I saw at BbWorld this year. It was almost as if I was at a conference that was about teaching and learning. The only other LMS community where I’ve seen multiple conference talks that were both grounded in pedagogy and theory focused (as opposed to “this is how to implement this pedagogical approach using this tool”) is (ironically enough) the Moodle community.

    Announcements

    Blackboard’s slides showing recent progress on both Original Experience for Learn and Ultra have a steady-as-she-goes feel to them:

    Screenshot 2017-08-12 13.25.20

    Screenshot 2017-08-12 13.25.46

    The Ultra mix of new features is odd in an interesting way. On the one hand, the fact that the company only added fill-in-the-blank test questions and media capabilities in the rich text editor last quarter screams “Caution: Wet Paint.” On the other hand, “discussion insights” is an embedded analytics capability unique to Blackboard (as far as I know) that helps instructors sort through active discussions in large classes. The latter may be a requirement for their flagship Ultra client—the University of Phoenix—or an indicator that Blackboard is thinking differently about what they want to be considered a fundamental differentiator for Learn. Or both.

    Blackboard Collaborate, their webconferencing platform, got a lot of love at the conference too. Our early experiences with the relatively new “Ultra” version of the platform were frankly rocky, but I heard nothing but raves about it from customers. And the company is clearly putting a lot of energy into it:

    Collaborate

    Blackboard has long been a portfolio company with lots of stuff to sell, but this year was the first time I’ve seen them walking the walk on truly integrating those products. For example, I saw a demo of their new Blackboard Instructor mobile app on a tablet which showed a workflow of a professor sending an announcement to students reminding them that a synchronous session was about to start and then launching an embedded Collaborate session. It was pretty slick.

    In some ways, this fits a pattern that Phil noted in his D2L Fusion post that Instructure’s competitors are finally catching on to the notion that ease-of-use is not just a marketing slogan or another bullet point in a long list of bullet points. Perhaps the most revealing moment of the conference in this regard was when Blackboard’s VP of Teaching and Learning Phill Miller showed me this graph of the usage of Blackboard’s SafeAssign antiplagiarism tool:

    SafeAssign

    What happened in 2016? Blackboard integrated SafeAssign into the core grading workflow.

    I know, I know. You’re thinking, “Wait. You mean it wasn’t for all this time?” Nope. And that’s the point. Quantified, even. The value of a feature is only realized when the feature is used, and how often it is used depends heavily on how usable it is. In some cases dramatically so. All the LMS providers are now working to raise their respective games in terms of usability. But Blackboard in particular has some opportunities to increase value because of the breadth of their product suite. To the degree that they can simply improve workflows through better integration between their products, they can unlock a lot of latent value fairly quickly.

    But let’s return to that new Instructor app for a moment. The emphasis is on improving instructor/student communication. Their separate (and older) speed grader-equivalent app will be merged with the new instructor app, but there appears to be a real company-wide focus on connecting humans with other humans in an educational context. This theme carried over into their analytics products—both embedded and stand-alone—which move away from Blackboard’s historic emphasis on reporting and beyond the industry’s fixation with retention early warning into the day-to-day business of helping busy instructors catch important details that they might have otherwise missed. The aforementioned discussion insights is one example. Another is their advisor analytics dashboard, which helps students’ advisors get increased visibility into how the students are doing in all of their current classes.

    Oh yes, and did I mention that there was a Moodlemoot inside BbWorld? There was! There were even slides, presented by Blackboard employees, during Blackboard sessions, about their Moodle-related products. Here’s one:

    Moodlerooms

    Blackboard has built a large part of its global business by buying up major Moodle hosting and support providers in large swathes of the world. As a result, they contribute a majority of the revenues that fund continuing Moodle development by Moodle HQ. Blackboard has historically downplayed this relationship inside the United States—even though they purchased the US’s largest Moodle support provider—for fear of cannibalizing their Learn business. That policy has apparently changed. Moodle even got several prominent mentions in the BbWorld keynote.

    I was able to spend a little time at the Moodlemoot, though not nearly as much time as I would have preferred. It was small and the vibe was a little glum. This isn’t surprising. First, Moodle adoption has been losing steam for a while here (as well as in Europe, though that change is more recent).

    Moodle NA

    Second, vocal elements of the Moodle community, including some in leadership positions, tend to be anti-vendor in general and anti-Blackboard in particular. Having a major US Moodlemoot fit inside a single (admittedly large) room at BbWorld had to be a hard pill to swallow. And Blackboard, for its part, did not always appear to execute well on supporting the moot. I found the Moodle session listings in the BbWorld app to be confusing. Nevertheless, there certainly seems to have been a major sea change at Blackboard regarding promoting Moodle in North America. Time will tell whether the increased efforts toward a more visible and cooperative relationship will bear fruit.

    The last announcement piece I’ll mention isn’t really new to BbWorld so much as it is new since last BbWorld. Blackboard was heavily promoting Ally, the content accessibility tool the company acquired within the last year. Interest appeared to be huge, with overflowing crowds at the sessions.

    So what does all this add up to? I’d say a few things:

    • CEO Bill Ballhaus must have succeeded in convincing the company’s private equity owners to allow him to invest in accelerating product development. There’s no other way all these announcements would have been possible. That’s definitely new and a positive leading indicator.
    • The company is thinking about ways to combine its portfolio of products (and services) to meet customer needs. That’s also new, and a differentiator.
    • Another differentiator is the level of sophistication that Blackboard is bringing to learning analytics, both in terms of the feature set and in terms of the conversations they are having with customers.
    • Both the announcements and the customer sentiment indicate that the company is getting better at both listening and executing based on what they’ve heard.
    • None of this changes the fact that Blackboard is still playing defense, but it does suggest that they may be playing better defense and preparing a strategy that will enable them to go on offense.

    Management Public and Private Presentations and Discussions

    This is the area where some of the most dramatic changes were visible. For starters, the marketing messaging in the keynote was by far the most subtle and sophisticated that I’ve ever seen from Blackboard. Two new taglines were introduced. The first one, “Simply Powerful,” wasn’t really new but rather a revival of the old ANGEL tagline. (I think I still may have that T-shirt, although I doubt I could squeeze into it anymore.) Back in the ANGEL days, the subtext was, “ANGEL is simpler than Blackboard, but it’s also powerful.” In today’s context, the subtext is flipped on its head: “Blackboard is more powerful than Canvas, but it’s also simple.”

    The other new tagline was “Your Partners in Change.” There’s a lot going on here. First, this line was projected up on the screen in huge letters as Bill Ballhaus talked about this year being the 20th anniversary of Blackboard. Also on the screen in the background was a picture of NASA’s Pathfinder spacecraft, which landed on Mars the same year that Blackboard was founded. Ballhaus is an aeronautical engineer by training, a fact that he made very plain in his schpiel. Part of the subtext was, “Yes, we’re Blackboard, but not that Blackboard. And I’m the CEO of Blackboard, but not that CEO.” “Partner” was as important as “change,” because it contrasted with the hubris of the last two CEOs. It also provided a cohesive identity for the company as one that provides an integrated portfolio of products and services that can help its customers respond to changing times.

    But the star of the keynote was not Bill Ballhaus but Blackboard’s Chief Strategy Officer, Katie Blot. Ballhaus acted as host and referred to himself as “chief client advocate,” but he quickly ceded the spotlight to Blot for the substance of the keynote.

    A side note: All of the top three LMS providers in terms of US and Canadian market share have powerful, competent women on their senior leadership teams. Phil mentioned D2L’s Cheryl Ainoa in his recent post. I have mentioned Instructure’s Misty Frost in the past. Like Ainoa and Frost, Katie Blot has a role at her company that is broader than her title suggests. It was good to see her get the spotlight.

    And she did not disappoint. Blot is not an ed tech industry careerist, having come to Blackboard from her previous gig working at the US Department of Education. She left behind former CEO Jay Bhatt’s absurdly grandiose claims that Blackboard would change the world single-handedly—”Your Partners in Change”—while maintaining upbeat energy. Her talk was substantive and thematic, punctuated by video interviews with various senior executives about specific product developments. It was not over the top, gross, or cringe-inducing in any way. In fact, it was…dare I say…quite good.

    Beyond that, the main themes I noticed were honesty and consistency. I’ve mentioned the former already and have more detail to add, but let me first address the latter. As we’ve mentioned in the past, one method we have for evaluating vendors at their conferences is asking a lot of different people the same questions and seeing if we get the same answers. This is a particularly critical litmus test for Blackboard given the mess it has to clean up regarding the confusion between it’s SaaS options and Ultra. I asked a lot of random Blackboard employees about how Ultra is going. Consistently, the answer I got was something like the following:

    Let’s back up and first talk about SaaS….

    [Tells a story about good progress with SaaS, adoption, often mentioning the bump they hit with earlier adopters in the process.]

    Now within that context, let’s talk about adoption of the Ultra experience.

    [Talks about how each customer has their own must-haves before they will even consider Ultra, how Blackboard has a prioritized punch list, and how they have dramatically increased the number of scrum teams working on it to make sure they can meet their commitments to work their way through that punch list in a reasonable time frame.]

    This is a pretty dramatic contrast to two years ago, or even a few months ago when Phil was at a Blackboard conference in Europe. So the company is definitely getting on the same page. Out in the field, we are not seeing the same consistency among the sales representatives during RFP presentations. When I brought this up with Bill Ballhaus, he acknowledged it without hesitation and went on to describe steps the company is taking to correct the problem. (There’s that honesty thing again.)

    There was also a return of Ray Henderson’s progress report card by both Phill Miller and Blackboard’s Chief Product Officer Tim Tomlinson. I believe it’s no coincidence that both of these men are former ANGEListas. There was something of a minor ANGEL take-under at Blackboard when Henderson was President and Chief Technical Officer there. That change stalled out under Bhatt but has apparently been revived under Ballhaus. Miller and Tomlinson have both been promoted, and the center of gravity for Learn development has moved to Indianapolis, the former home of ANGEL. Henderson’s public approach could be summed up as something like “make commitments, measure your progress, and tell the truth.” I saw many signs of a similar philosophy taking root in Ballhaus’s Blackboard.

    The last thing I’ll say—and this is probably the most subjective assessment of this post—is that the employees seemed, for lack of a better word, happy. Not forced, conference-host happy but normal people happy. I-love-my-work-and-like-my-colleagues happy. I have observed lots of folks working at organizations that are under stress or dysfunctional. I have seen them from the inside as well as from the outside of those organizations. There’s a vibe that’s unmistakable. The Blackboard folks I talked to didn’t seem to have that vibe.

    Bottom line: Blackboard’s adoption trend line is undeniably down and likely will continue in that direction for at least another 12 months (if you factor out the likely University of Phoenix implementation, which will skew the numbers). But early and subjective signs suggest a positive change in direction inside the company—possibly a rapid one—that may become more visible externally between now and this time next year.