Instructurecon 2016: Why This Company is Still Formidable (and Misunderstood)

When I talk to Instructure competitors or critics, I usually hear two complaints about them:

1. They “open wash” or are “fauxpen.”
2. They have no vision.

Having attended Instructurecon 2016 a couple of weeks ago, my answer to the first complaint is that the critics don’t understand what Instructure and their customers mean by “openness.” In some respects, Canvas is the most open ed tech project I know of in ways that strongly differentiate it from its competitors. Full stop. The second complaint is more plausible. But if I were competing against Instructure, I wouldn’t take too much comfort in that.

Openness

Most criticisms of Instructure’s fauxpenness come from Sakai or Moodle fans. There are two flavors of this critique. The first faults the company for having an “open core” model, meaning that not all of the software is open source. The basic, feature-limited version is released under an open license, but to get the deluxe version, you have to pay a fee and the source code is not open. Think of open core as a variant of “freemium.” The second criticism is that, while (most of) the source code is open, it is nearly all developed by Instructure employees. It doesn’t have a “real” open source community in the conventional sense.

These are not the kinds of openness that Instructure customers care about. They don’t see a practical difference in 95% open source code versus 100% open source code, as long as they can understand that code well enough to be able to do what they need to with it. Nor do they care who develops it, as long as they can count on it continuing to be developed well. I don’t mean to trivialize these kinds of openness. They can matter quite a bit to some organizations in some situations. But they don’t matter very much to Instructure customers, who place higher value on different kinds of openness.

I didn’t know it in the moment, but the best illustration of this difference was the attendee reaction to CEO Josh Coates’ keynote. In the past, I criticized former Blackboard CEO Jay Bhatt for his odd BbWorld presentation:

CEO Jay Bhatt ran through a whole long list of accomplishments for the year, but he only gave each one a few seconds as he rattled through the checklist. He mentioned that the company has a new mission statement but didn’t bother to explain it. It took nearly an hour of mostly talking about big macro trends in education and generalities about the categories of goals that the company has set before he finally got around to new product announcements. And then commenced what I can only describe as a carpet bombing run of announcements—a series of explosions that were over by the time you realized that they had started, leaving you to wonder what the heck had just happened. Vice President of User Experience Stephanie Weeks gave a 10-minute talk that was mostly platitudes and generalities about goals for students while some truly significant UX work that her team had done played on the video screen in the background, largely unexplained. There was something mentioned about cloud. Collaborate without a Java plugin! A new mobile app. Wait, another new mobile app, but something about jobs. Wait! Go back to the last slide! I think that was…. Is it over already? It seemed like simultaneously the longest and shortest keynote ever.

Phil and I had a chance to talk to Jay about it later in the day and asked him (politely) what he was thinking. He said, “I don’t view BbWorld as a selling conference. At all.”

Josh’s keynote was far weirder than Jay’s. He told a rambling story about his college days and desperately searching for a course he could pass that would fulfill a final degree requirement so he could graduate. He cracked a bunch of jokes, some of which were funny. He talked about his personal view of the world. By the end, you could get the sense that he was trying to tell you something that was important to him about the company’s values. But I don’t think he spent more than five minutes talking explicitly about Instructure, and I don’t remember a single comment about roadmaps or product releases. In fact, I don’t recall anything being discussed about product road map or milestones on the main stage during the entire first day of the conference.

But in stark contrast to BbWorld, none of the customers I spoke to at Instructurecon seemed to mind. Why? Because they already knew what the company was doing. Throughout the conference, I asked a number of attendees what percentage of their motivation for coming was to find out what the company was planning to develop in the next year. The range of answers averaged between 10% and 20%. They all told me that, while they were looking forward to the roadmap sessions later in the week, they didn’t expect any big surprises and weren’t all that focused on finding out what the developers had been doing. I have never been at an LMS conference where that was true, including Sakai conferences and Moodle Moots.

What is Instructure doing differently? There are clues, but you have to know where to look and how to read them. For example, take a look at this slide from VP of Product Mitch Benson’s road map presentation that describes their software development lifecycle:

Mitch spent maybe a minute on this slide. It whizzed by. But note the words on the bottom. That’s Agile software development terminology. Most people who have done a little Agile are familiar with epics and stories. An “epic” is a reasonably small feature that makes sense to a customer, like the ability to add a comment to a graded assignment. A “user story” is more of a micro-feature that makes the main epic-level feature work. For example one user story included in the epic of adding a comment to a graded assignment be the ability to edit that comment once it is saved. Agile development teams use epics and stories all the time to prioritize software development tasks.

Less widely used is “theme.” This is a higher level of organization to start thinking about goals. Let’s look at another slide from the same presentation to see how Instructure uses the term:

One theme the Canvas product team will be working on for the 2016-2017 development year is “achievement-driven.” They see the functional areas in which this theme will appear as master courses; quizzes; outcomes; data and live events; and gradebook. Once a year at Instructurecon, the company will announce these high-level themes. And by the way, after they announced the themes this year, they then had representatives from various user groups come up on stage and describe their top priorities, risking the possibility that the priorities customers asked for—in front of everyone, on main stage—looked nothing like the roadmap that Instructure had just presented. As it turned out, there was a fair bit of alignment, and the areas where the user groups differed effectively highlighted the diversity of the needs of the user base rather than making the company look clueless about their customers’ needs.

Anyway, after the themes are announced, the product team then goes off to conduct their customer research and design planning. At some point, a number of months later, the product team releases the user stories and screen designs. Publicly. Not just to select interest groups. Not just to paying customers. To everyone. These design artifacts amount to both a public announcement of and commitment to what is coming as well as an invitation for customers to provide ongoing input to developers as the features are being developed. This is not something special that Instructure does only with cutting edge ideas they are trying out. It’s the way they do all their Canvas development. It works because (a) the company is incredibly good at getting the word out to distracted customers about work in development and (b) customers trust the company to keep its commitments. Early screen designs by vendors are often derided as “vaporware.” When a customer uses that term, it means “I don’t trust you to deliver on your promises.” I could find none of that at Instructurecon, and I was actively looking for it. The most remarkable thing about the conference was the extremely low level of overall customer anxiety.

Instructure’s particular take on openness shows up in other ways too. For example, by lowering customer anxiety regarding future uncertainty, the company is able to create a lot of space at the conference for networking. The primary activity at Instructurecon seems to be sitting down over meals, at parties, or in lounge chairs and chatting with folks. This is true in of most conferences in some sense. The difference is that Instructure creates an environment in which customers don’t feel torn between attending sessions in hopes of gleaning scraps of information about product development and having high-value informal and opportunistic conversations. Instructurecon is not really a conference so much as it is an extremely well executed social event.

And the hosts are everpresent, always there to grease the wheels of conversation without being conspicuous about it. Particularly at meals, there was almost always at least one Intructure employee sitting with the customers at any given table. It could be hard to tell the difference at first sometimes because there wasn’t an atmosphere of anxious customers quizzing or complaining to the vendor. It was just folks chatting. Phil and I do a lot of roaming and random table-sitting when we are at these conferences. In fact, we view it as one of the most important activities for our analysis. We listen to a lot of conversations and ask a lot of questions. The one thing we’re looking for above else is consistency or inconsistency. Are we hearing different customers complaining about the same problem in different conversations? That’s meaningful. Are different employees saying different things in reply? That’s meaningful too. Are they all saying the same thing? That’s even more meaningful (and unusual). There was a shocking amount of consistency among both customers and Instructure employees at Instructurecon.

And a lot of honesty. For example, while I didn’t hear a lot of complaints about functionality, the one I heard consistently enough to call it a pattern was about tests and quizzes. And the answer I heard from Instructure employees, multiple times in multiple places, always went something like this:

Yes, we’ve been hearing a lot of complaints about that. We know we have to improve it.

That’s it. No dodging. No polite non-answers. No rationalizations. No “OK, I hear you, but….” Just, “Yup, we know it’s a problem and we owe you a fix.” What didn’t happen next was equally striking. I didn’t hear customers say, “Yeah yeah, we’ve been hearing you say that for three years now and nothing has changed.” The conversation just moved on. Customers were satisfied with the answer. The quiz problem was even raised, quite bluntly, from the main stage by a senior executive. But that happened on the last day of the conference. The employees I heard addressing the issue at table conversations had not been taking their cues from something they had heard an executive say publicly already that week. They just all knew there was a problem, and they also knew the right way to respond was to own up to that problem and make no excuses. Their customers see that, understand it, and respect it.

That, my friends, is a kind of openness that is both very valuable to the adopting schools and very difficult to achieve. I have participated in multiple open source communities that, while they were open in the senses that nothing is secret and all code is openly licensed and yadda yadda yadda, they are mostly opaque to anyone who is not in the inner circle and investing large amounts of time in staying current. Because transparency is incredibly hard, and building real personal connections and trust that transfer to an entire organization is even harder.

So, in retrospect, Josh’s rambling keynote about how he accidentally learned the value of openness by trying to dodge a graduation requirement as an undergraduate? Not so weird. At least, not as weird as it seemed to me as a first timer at Instructurecon and an outsider to the Canvas community. It revealed an essential characteristic of the way that Instructure does business and their single strongest advantage in the market. Competitors who fail to understand this about them will continue to lose to them.

Playing “Small Ball”

Phil recently described Moodle’s strategy, based on his recent attendance of a Moodle Moot, as playing “small ball.” The same could be said of Instructure’s Canvas development strategy over the last few years. Lots of incremental improvement, but nothing terribly ambitious. I raised this concern to a number of Instructure employees at all levels of the company—again, in separate conversations—and once again got remarkably consistent reactions. Sometimes with a pause and a head cock, other times immediately and without hesitation. But the answers were all generally along the lines of, “I can see why you’d say that” or “That’s probably fair.” One or two employees pushed back a little, but mainly just to establish that they thought some of those incremental changes have been meaningful ones. They didn’t deny the incrementalism itself. Add to this apparent consensus the fact that there has been a lot of churn in the senior ranks of the company’s product management over the past few years, and it raises the question of whether the company has the capacity to innovate substantially beyond the bounds of the founders’ original vision for the product.

The latest reorganization of the product team now has long-time insider Mitch Macfarlane as SVP of Product and Customer Experience. Once again, the answers I got from quizzing random employees about how they feel about the move was surprisingly consistent. They feel good. (I think some of them may feel a little relieved too, based on their body language and tone, but that’s my interpretation rather than anything that was expressed explicitly.) They seem to think that we will see improvement in both the speed and magnitude of product improvement and innovation.

But the proof of the pudding is in the eating. The first test for Instructure will be Mastery Paths, which was announced to be in beta at the conference and is scheduled to be released this quarter. How “innovative” or “ambitious” is this work? Selective or conditional release functionality has been around in mainstream LMS platforms at least since WebCT Vista. These features are getting new attention because of the increasing interest in CBE and mastery learning. You could argue that Mastery Paths is Instructure playing catch-up. And that’s undeniably true. But it may also turn out to be only part of the story—depending on the details of the released product. Experience with existing implementations tells us it’s hard to build this kind of authoring functionality in a way that is easy enough to use that it gets broad adoption. More often than not, anything beyond the most basic gates or triggers only gets used by instructional designers and some hard-core early adopter faculty members. Instructure has set a goal of making mastery learning authoring functionality easy enough to use that it will be broadly adopted.

At the risk of leaning on a badly abused comparator, Instructure’s philosophy of product development is similar to Apple’s in the sense that both companies focus popularizing functionality already in the early adopter market by improving ease of use to the point where it feels different in kind rather than degree. Apple didn’t build the first MP3 player, for example. They just built the first one that lots and lots of people judged simple enough to be worth the trouble. This is the kind of step function in usability that Instructure claims they want to bring to mastery- or competency-based authoring. It’s certainly not the only possible approach to innovation in the LMS market. For example, D2L’s strategy is to bring sexier functionality like adaptive learning and gamification to the LMS product category. “Innovation” means different things to these two companies.

We’ll see if Instructure succeeds in bringing their style of innovation to Mastery Paths, and we will also see whether they bring other ambitious improvements to their shipping product in the next 12 to 24 months.