The Future of Sakai: My View

As a member of the Sakai Foundation Board, I have at least the appearance of conflict of interest in any analysis I write about Sakai. That’s why I encouraged Phil to write his own, independent analysis of the recent developments around Sakai OAE. I was happy to support him as one source for the article but not as editor. In fact, in keeping with his previous coverage of the Sakai OAE woes, Phil did not show me his post draft before he published it. This is as it should be. We strive for objectivity and independence here at e-Literate.

But now that Phil has done his work, I would like to offer up my own personal reflections on the current state of the Sakai software and community and where they could go from here. In the process, I will do my best to make my relationship to the project clear.

Sakai and Me

My opinion of Sakai in the early days was not high. Back in 2005, my post reviewing the release of version 2.0 of the software started with the sentence, “Given that Sakai 1.5 was a feature-impoverished, unusable wreck, I fully expected 2.0 to be unusable as well.” I said some nice things in the middle of that post about the pace of progress from 1.5 to 2.0. But further down, after writing about how the discussion board was “unusably” bad and complaining about the total lack of support for groups, I went on to write,

To me, what this says is that (a) distance learning teachers have not played a significant role in the interface design and (b) the four big institutions at the core of the Sakai project aren’t giving a whole lot of thought to using Sakai beyond basic web-enhancement of F2F classes. Because nobody who had any serious intentions to teach hybrid or fully online class would let these two problems persist. They are both deal-breakers. And since other areas of the UI have clearly gotten some work, it leads me to worry that the folks driving Sakai are really just that clueless about what makes for a great distance learning platform. (Or, alternatively, they don’t care that much. My experience is that the degree to which a school is committed to fully online learning is inversely proportional to the popularity of their sports teams. If campus life is a big part of the school’s brand value, then selling distance learning just doesn’t appeal that much.)

Not content to just throw stones from a distance, I eventually subscribed to the Sakai listservs and aired my concerns there directly. I was so relentlessly critical that, at one point, an exasperated Brad Wheeler wrote me an email asking me to lay off. Which I did not do. Meanwhile, in my role at the State University of New York, I was part of the evaluation team that rejected Sakai as the right solution for a system-wide LMS, despite the fact that senior executives in SUNY wanted the answer to be Sakai. At one point, during a public presentation by IU’s Rob Lowden on the virtues of Sakai, an exasperated SUNY Learning Network Director David Porush asked me to refrain from asking questions that were putting Sakai in an unflattering light. Which I did not do.

It is fair to say that I was not a fan. And yet, there were aspects of Sakai that I found remarkable. First was the idea of an open source community of educators designing and building educational software together. While I am not and never have been anti-capitalist, I do believe that it is important for credible, not-for-profit open source alternatives to exist in the educational software market as a counter-balance to commercial forces. Looking back, it is clear to me that the existence of Sakai and Moodle together put pricing pressure on the for-profit LMS players. They also gave universities a sense that they had someplace to go at a time when Blackboard was buying up some for-profit competitors and suing the rest.

Second, there were some deeply interesting ideas in the architecture of Sakai, chief of which was modularity. In this era of ubiquitous RESTful APIs and maturing integration standards, we take the idea of the mashup for granted. But back then, it wasn’t obvious. Along with early work by Chris Vento on what would eventually become the IMS LTI standard and research coming out of the JISC on the e-Learning Framework, the Sakai approach inspired Patrick Masson and me to propose that SUNY should use pieces of Sakai to build what we called the Learning Management Operating System, or LMOS. Conceptually, it was not terribly different from what Blackboard proposes to do today with their xp framework; namely, separating “tools” into separate and swappable learning services. Don’t like the discussion board in your system? No problem. Just swap in another one. I will have more to say on this idea later in this post. For now, the point is that the Sakai architects were onto something. As imperfect a start as they had, they showed promise of providing both economic protection and valuable innovation to the academic community.

It was at this moment, when I was still critical of Sakai, had rejected it as a viable option for my institution, but was promoting the value of some of the ideas in the software and the community, that Ian Dolphin—then a member of the Foundation Board—nominated me to stand for election to the Board. The flattery of it aside, Ian’s drive to attract a diversity of opinions and perspectives into the community made a strong impression with me and began to change my impression of the Sakai leadership (whom I had at one point referred to as “brahmins” and a “technocracy of developers”). I didn’t get elected that year, but I did start paying more attention to the diversity of voices in the project rather than just the ones that irritated me the most. I began to see it as the community that it is.

Several years later, when I was working for Oracle, I ran for the Board again. I was elected that time. Like SUNY, Oracle’s interest in Sakai was somewhat indirect. They were (and are) a commercial partner, but not in the way that Unicon, Longsight, and rSmart are commercial partners. They didn’t have nearly the same skin in the game. My current employer has even fewer specific ties to Sakai than Oracle did. Cengage is a partner of all the major LMS players and, unlike Oracle, has no reason to tilt toward entrants that use their database or are written in Java. So I don’t have any direct and tangible stake in the Sakai’s success the way many other Sakai community members and almost all other Sakai board members do. I have served on the Sakai Foundation Board mainly because I think what the community does is important for education. I have never thought it was anywhere close to perfect in how it goes about achieving its aims; nor do I now. I have never thought it was important for Sakai software to “win” in the LMS space by being a dominant player; nor do I now. I support Sakai because I think it is important to have a credible and viable non-commercial alternative in the LMS space in order to protect the choices of schools, because I think a software development group like Sakai’s can generate important innovations by being so close to the teachers and students who use the software, and because I believe that the software produced by this community can serve the needs of a significant portion of the academic community well. The recent events with OAE have not changed my feelings in this regard.

Sakai OAE

I have generally been a fan from afar of OAE. As far as I am concerned, it remains one of the boldest and most interesting efforts I have seen to re-imagine the institutionally supported learning platform. I also believe that it has been significantly more influential than its current level of practical success would indicate. I know for a fact that Blackboard employees have spent significant amounts of time picking through the OAE source code. And I suspect that Martin Dougiamas’ decision to start thinking about Moodle 3 as a complete redesign was at least partly inspired by Sakai’s revisioning work in OAE. But I have never been an active participant or even close observer of the project’s day-to-day operations. As is often the case when a high-profile effort runs into trouble, there is a lot of Monday morning quarterbacking going on right now. Since I have not been following the play-by-play, I can shed no new light at that level. But I do have some high-level observations and speculations about what might have gone wrong and what it means for future open source educational technology efforts.

Sakai OAE was started as an experiment by Cambridge University. Cambridge doesn’t teach classes the same way that most United States universities do, and they are particularly ill-served by the structure of a traditional LMS. In Cambridge and other schools that follow the tutorial model, lectures are decoupled from classes. Students work individually and in small groups with their tutors to define their curricula. As part of that effort, they may go to lectures by their professor, another professor in their department, or a professor in an entirely different department. This pedagogic structure, although it is very old, bears a distinct resemblance to the idea of a personal learning network. The structure of the traditional LMS, where everything attaches to the class, simply does not work here. So Cambridge began developing a system that would work better for them and sharing it with the Sakai community.

A number of folks got excited by what they saw. And since it came at a time when enthusiasm for and investment in the traditional Sakai CLE product was beginning to slump, Cambridge’s experiment got positioned as “Sakai 3,” the future of the Sakai platform. This was the first mistake. Beyond the fact that doing so alienated the hard-working people who were keeping their schools’ mission-critical current-generation Sakai platform viable, it also tried to put a square peg into a round hole. Cambridge’s experiment was interesting precisely because it was not an LMS. And while the project vision was to eventually support the same functional areas that an LMS does, the heart of the project, and the area that most urgently needed to be developed, was the area that was different from an LMS. Making the short-term strategic goal of the project to become a full functional replacement for the current Sakai software was bad for both projects. The Foundation corrected course on this decision about a year in, changing the name of the Cambridge-born project from “Sakai 3″ to Sakai Open Academic Environment (OAE),” but some of the deeper roots of the problem went uncorrected. Fundamentally, there was not agreement among the participating institutions about what the goals of OAE were. From my view at a distance, it looked to me like Indiana and Michigan wanted the next generation of the current Sakai CLE product, Charles Sturt wanted both what Michigan and Indiana wanted and the unLMS vision that Cambridge had originally pursued, and Berkeley wanted a portal. I’m not entirely clear on Georgia Tech’s goals, although I suspect that they were closer to the original vision. NYU was perhaps most closely aligned with the Cambridge vision. In some ways what I’ve given here is a bit of a caricature of the motivations. Schools came to the table, in part, because they were inspired by what they saw in the vision. But as that vision got filtered by the participants through their respective local contexts and tactical needs, views on project goals appear to have diverged significantly. This factor alone would have made meeting a coherent set of project goals difficult to achieve.

On top of that, there appear to have been some serious flaws in the development process. Again, I write this from a distance, not having been close to the project. But the accounts coming out of the group are consistent in their reporting that performance problems were identified early but not addressed until much later. This was a cutting edge project in a dozen different ways. It made much heavier use of client-side technologies than any previous effort by the community. It drew on new, web-scale architectural principals and technologies like NoSQL. In other words, the project was operating in a modern, consumer-web development world of today rather than the enterprise development world that dominated when Sakai CLE grew up. It was therefore high-risk. Agile software development techniques have grown in popularity precisely because this kind of highly dynamic development environment with new goals and new technologies has become the norm rather than the exception. The OAE project team professed to be using Agile methods. But the entire raison d’etre of Agile is to address risk sooner in smaller chunks. Whatever Agile method the project team was practicing, the fact that known architectural risks lingered until they became so expensive to resolve suggests that the project was not following Agile principles. This, in turn, may have been caused by a conflict between process and governance. I emphasize that this is speculation on my part. I didn’t observe the day-to-day workings of the project, so I am left to guess from the evidence I see after the fact. But as a community, Sakai tends to focus much more on governance than on process. One thing is clear: If performance was an existential risk to the project—and it is now abundantly clear that it was—then all work on functionality should have been deprioritized in favor of reducing that risk until the team had measurable evidence that they had performance under control. That apparently didn’t happen.

This problem was exacerbated by some drift from the project’s original philosophy to build only the portions of the code that were unique to the needs of higher education, using as much generic open source code as possible. The team had originally chosen to use Apache Jackrabbit as their back end, on the theory that the LMS was really managing content at the back end, and that a content storage structure like the Java Content Repository, implemented in Jackrabbit, could meet their needs. It was a fascinating idea. But it turned out that the needs of enterprise content management, which supports a lot of reading content and not so much writing, is very different than what OAE needed, which was a world in which everybody is a writer. By this point, enough code had been written based on the premise of Jackrabbit that abandoning it was expensive. So Ian Boston wrote a custom NoSQL back end solution that was designed to ease the pain of that transition. The trouble was, it was a custom back end. This was more code for the team to support, and tricky code at that. On top of that, there was more risk since nobody else had ever used it. As it turns out after months of additional work, the custom solution didn’t meet the scalability needs either. As a result, project sponsors lost confidence.

I want to be very careful about what I write next. A number of people, including me at times, have had a false sense of optimism about OAE. In addition, I have no official word yet about what the project team plans to do next from a planning or architectural perspective. (I applaud their prudence in thinking through their next steps, and maybe doing some early testing, before making any public announcements.) But based solely on the analysis above and on Ian Boston’s (unofficial) comment regarding the proposed change in architectural direction, it is plausible to me that Sakai OAE has a better chance at success now than it has had in a long time. The participating schools that were least in alignment with the project’s original vision have left. The community’s expectations regarding Sakai OAE being a CLE replacement in the short term have been reset. And if the direction described in Ian’s comment proves to be similar to the one that the project takes, then the risk profile and code surface area of the project could both be reduced significantly. Node.js and Cassandra are both relatively widely adopted technologies for building consumer web products. To the degree that the project can just do what lots of other folks are doing on the back end and focus their innovation mostly on the front end, that should enable them to do more with less.

There has been a myth in the Sakai community that you need a ton of money to build anything worthwhile in software. That may have been true ten years ago, but it is not true today. As Phil Hill and I have both been emphasizing here at e-Literate, the cost of developing software has dropped by at least an order of magnitude in the last decade. Instructure built an entire, viable, competitive LMS with vastly less than some of the OAE project participants seemed to think they needed. In fact, there is a strong sense among venture capitalists that it can be harmful to give a start-up to much money too soon, because excellence comes from the focus that having to make hard choices forces on you. Furthermore, I suspect that the failures in OAE risk management may have been a direct result of the drive to get more money into the project and the tensions that resulted by having too many diverse tactical demands being placed on it. The remaining project team now has years of user research, front end code, and experience. If they are able to maintain focus and manage risk, there is no reason to believe that they don’t have the resources they need to turn the original Cambridge vision into a successful product. Whether that product turns out to be a replacement for the LMS or something that fits into the academic ecosystem in a different way is another question entirely. I don’t know the answer to that question, and I don’t think the project team should focus on answering it in the short term. Instead, they should strive to meet some real, urgent, and well defined needs of a coherent and significant group of academic stakeholders. Focus is everything.

In my view, the biggest threat to OAE now is not its resourcing but its customer base. Having too many customers with urgent but divergent needs is deadly to an early-stage product. But having no customers with urgent needs is equally deadly. OAE needs to be continuously tested against the real-world, practical needs of a coherent set of customers. If the remaining OAE participants walk away from the goal of implementing for real-world uses at some reasonable scale, then the project will likely fail.

Sakai CLE

There are some in the Sakai community who appear to believe that attention and resources will automatically return to Sakai CLE now that OAE’s star has fallen. They are almost certainly mistaken.

The very modularity of the project that I touted early in this post is actually a second-order effect of an aspect of the community that is coming back to haunt CLE. Forty-four years ago, an engineer by the name of Mel Conway wrote down an observation that came to be known as Conway’s Law:

“[O]rganizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.“

In other words, if a company has three divisions that are building a piece of software together, that software will have three modules. And the quality of the interfaces of the modules will be partially determined by the quality of the communication between the three divisions.

In the early days of Sakai, one of the things that made the coalition work was an agreement that each school would do its own thing to a certain degree. The modularity of the software was designed to enable the project to hold together with a lower degree of consensus about the details. If School A didn’t like the discussion module that School B wrote, then School A had the option write their own or integrate a third-party option. But as a result, there was never a particularly strong functional vision for Sakai. The closest the community came was to set a goal that the system should support academic collaboration as well as formal classes. This goal is manifest in the product’s flexible permissions structure as well as the name choice of Collaborative Learning Environment (CLE) rather than LMS. But other than that, Sakai CLE really was and is a solid but mostly conventional LMS. That was perfectly fine in the days when the changes in the competitive landscape were slow and being supported by an open source community was a more significant differentiator than many of the functional variations between LMS products. Sakai CLE has been competitive. But today, the various LMS products are differentiating themselves more significantly and more rapidly. Unless CLE develops a compelling vision that excites stakeholders and solves real and urgent unsolved problems, then Sakai schools will be more likely to move to other platforms than to make a significant investment in updating Sakai.

Does it make sense to put this kind of investment into a current-generation LMS architecture? The market offers a diversity of answers. Instructure was built from the ground-up on a consumer web architecture and is doing well. Blackboard’s xp strategy has them swapping out their current-generation architecture one piece at time over an undefined period of time. Moodle is planning a re-architecture of their platform with Moodle 3, but it is years in the future. As far as I know, nobody has written a single line of code for Moodle 3 yet. Desire2Learn seems quite happy with their current-generation architecture and has not given even a hint that they have plans to replatform. Given all this, there is no particular reason to believe that Sakai CLE is inevitably at the end of its run. To my mind, the strength of a functional vision and the leadership to sell investment in that vision is going to be a stronger determiner of the future intermediate-term future of CLE than its technical underpinnings.

One interesting possibility is related both the LMOS idea I mentioned earlier and Blackboard’s xp strategy, which I described in a recent post. Blackboard is essentially rebuilding their LMS one piece at a time using consumer web technologies like Node.js and a NoSQL back end solution. They are also providing these pieces as true multi-tenant SaaS; I think that’s another critical piece the Sakai community has been neglecting, but that’s another post for another time. The main point is that they are rebuilding parts of the LMS one at a time, and they are making heavy use of the IMS LTI standard to do it. Notably, Sakai CLE had strong support for LTI long before Blackboard did, in part because of Chuck Severence’s role as the LTI evangelist and in part because it was an essential component of the CLE/OAE hybrid strategy. There is no reason why Sakai CLE core couldn’t evolve into that enterprise class identity and grade hub that Blackboard Learn is moving toward, and that new projects focusing on innovating in various parts of the academic experience couldn’t spring up and integrate with CLE. This is not so different from the original CLE/OAE strategy. Indeed, OAE could yet be one of those projects that integrates with the hub. But regardless of whether the community adopts this vision or some other vision, Sakai CLE needs some vision to move forward and thrive as a core project. In order to get resources, it will first need leadership.

The Role of the Foundation

One question that comes up when a project hits a crisis is whether the Board has been doing its job. Couldn’t the Board have prevented these problems from happening? In my view, the answer is largely “no.” The Foundation is a legal structure designed to make it easier for schools and other stakeholders to collaborate on open source software development processes. It is designed to lower transaction costs, reducing the friction of collaboration. The Board exists to ensure the health of the Foundation. But there is no reason to believe that people who are a layer removed from an individual project would be more effective managers of it than the people who are involved with it every day and have skin in the game. You simply can’t force coherence and viability on a community-based open source project from the outside. And we should be clear that the Foundation Board, as an entity, is largely on the outside of individual projects within the Foundation.

What the Foundation can do is offer more guidance regarding best practices and more focus on making sure that projects are doing the kinds of things that tend to lead to project success in general. The Board could, for example, ask for risk analysis and mitigation reports from early-stage projects like OAE. It wouldn’t necessarily guarantee that the project leadership would do the right thing, but it would at least ensure that they are asking the right questions. Likewise, the Board could ask CLE stakeholders to have a discussion about functional product vision. It wouldn’t guarantee the group will come up with a good answer, or even any answer at all, but it would at least call attention to the question. This need for the Board to get better as facilitator (as opposed to a governor) of open source projects is one of the reasons why I am excited about the proposed merger with the Jasig Foundation that will be voted on by the community soon. The goal is to become the Apache Foundation of educational software. Apache projects don’t always succeed, but their membership in the Apache Foundation gives them a higher chance of success. That, I think, is the best that any open source foundation can do.

To sum up, I believe that Sakai OAE, CLE, and the foundation itself are at a critical inflection point. I am hopeful that all three can survive and thrive. But the story has yet to be written. In the end, it will all come down to individual members of the community stepping up. In the end, it always does.