Phil has been a busy boy, putting out two pieces about some Blackboard research that had gotten some negative responses and two more on a horrifically bad LMS outage for UC Davis and other universities using support vendor Scriba.
Our main schtick here at e-Literate is to get beyond the headlines. We try to explain what is actually happening and why it is important. Often, this involves puncturing hype like “robot tutors in the sky,” not by heaping them with scorn but by showing the gap between the hype and reality. But there are also times when we do the same for anti-hype. For example, there is a popular narrative that Blackboard is evil and bad and stupid, partly based on the massive brand damage the company did to itself a decade ago (and two CEOs ago). As a result, when they do…well…anything, it tends to be interpreted in the worst possible light. Personally, I’m glad that Blackboard is sharing the findings of their product research. I don’t know of many ed tech companies that are doing that. This is different than putting out a white paper about some study the company did showing how awesome their product is. Rather, it shows us how they are trying to understand their customers’ needs. Sharing that back with the world helps us, in turn, understand how Blackboard thinks. It is an important kind of transparency. As Phil’s second post on the topic highlights, we do have to understand that this kind of study is different than an academic study in order to draw the right conclusions from it. But that’s fine.
Likewise, the Scriba outage story plays into a “Sakai is dying” narrative. As it happens, I was at conference for the foundation that hosts Sakai last week and got something of a rundown on what was going on, including a preview of the release scheduled to come out this summer. (Oddly, nobody mentioned the outage, which must have been in its early stages at that point.) The Sakai sustainability story is complicated enough to merit a separate post, which I will get to later this week. In the meantime, I’d like to put the Scriba outage in context of how open source works in general and how the Sakai ecosystem works in particular. Because it is very easy for this story to reinforce preconceptions rather than looking at it carefully on its own merits.
Phil wrote,
There is an interesting angle here in that Sakai is open source yet data is not easily recoverable.
Right. Open source can help with problems like the Scriba outage. As far as I can tell, it did help UC Davis. But it’s not that simple. There is a lesson to be learned here for all LMS-using schools, but it’s not “open source doesn’t provide the flexibility that it is supposed to provide”; nor is it “Sakai is dying.”
The Theoretical vs. the Practical
One theoretical benefit of open source is that, if you don’t like your support provider, you can move to another one (or self-host) and keep your platform. If your provider were to suddenly disappear, as Scriba did, you would still be able to run it yourself or to transfer to another provider. This one of two main reasons that Instructure released most of Canvas as open source in 2011. The company was young and perceived as risky at the time. Releasing the code as open source was similar to putting it in escrow to assure clients that they would have something even if Instructure went belly-up.1 UC Davis was eventually able to stand up their own version of Sakai in part because Sakai is open source and the school therefore had access to a copy of the code.
But a number of other pieces also needed to be in place in order for this strategy to work well, or at all. First, Davis had to have a current local backup of their data. Running the LMS is all well and good, but what they really needed was their course data…you know, like student grades and stuff. One lesson here for any school, regardless of the brand or license of your LMS, is always have a current off-site backup of your data in a location not controlled by your vendor. Davis apparently had that. Second, you need the ability to run the system locally (or have some support vendor ready to run it for you). Davis used to host Sakai themselves and, being an R1 university of substantial size, obviously has a capable IT department. So they had the theoretical capability. The situation may have been complicated by the fact that their Sakai installation was apparently non-standard. We know that that the Davis installation runs on a different database than is typical. I would not be surprised to hear that they also have substantial customizations. One good thing about open source is that you can do anything you want with it. One bad thing about open source is that you can do anything you want with it.
Bottom line: Because (a) Sakai is open source, (b) Davis performed regular data backups on a site that was not controlled by the support vendor, and (c) Davis had the necessary technical capabilities in-house, they were able to scramble and get a local version kinda sorta running, on short notice, until Scriba re-emerged. On the flip side, there may have been Davis-specific complications that made this task harder
The Support Vendor Ecosystem
So that’s disaster recovery. What about moving to a different support vendor? Well, that depends. Are there any? And would you trust the ones you have? These questions, like many about open source, are project-specific and change over time.
Once upon a time, there were three major Sakai support vendors in the United States: rSmart, Unicon, and Longsight. Sakai was originally started by a handful of big schools that relied on their own staff for support, hosting, and development. The community of adopting schools was slow to expand to those that needed help to run or host the platform. As a result, three support vendors was both too many and too few. It was too many because there just wasn’t enough business to keep them all thriving. Unicon still does some Sakai work, particularly in the area of custom development, but they never did a lot of hosting work. rSmart decided to focus on Kuali—a move that ultimately proved disastrous to them—and therefore sold off their Sakai business to a company that then sold it to Scriba, the very one that caused the UC Davis disaster. The last hosting vendor standing in the US at the moment is Longsight. If there is only one, or even only two or three, the theoretical benefits of moving to another support vendor has limited practical value, particularly for schools that do not have the capability to self-host.
That said, you would need to know a lot more than this to draw firm conclusions on the future sustainability of Sakai. How many self-hosted schools are there? How many are hosted at Longsight? Or at other, smaller vendors? How much is going on internationally, with colleges and universities that are largely invisible to mainstream US-centric analysis? And how healthy is the development team? A note on that last point: An open source project can have a healthy community of developers that is disproportionate to the number of adoptees, since developer resources are not funded solely by licensing and support revenues. Therefore, adoption footprint is not as direct a measure of the health of the platform as it is for proprietary products.
More on that, and on the overall state of Sakai, soon.
- The other reason was branding; Instructure wanted to establish themselves as “the good guys” and more generally “the opposite of Blackboard”. [↩]
pmasson says
I can’t think of a worse example to highlight the benefit of vendor neutrality inherent to open source than Cavas. Instructure has an “open core” business model, which, “primarily involves offering a ‘core’ or feature-limited version of a software product as free and open-source software, while offering “commercial” versions or add-ons as proprietary software.” [1] (note, I would state “proprietary” and not “commercial” in that description.)
As previous Open Source Initiative President Simon Phipps has explained, “it does not deliver and sustain the principle that delivers cost savings and flexibility to the customer – software freedom.” [2] Phipps explains the “core” product is released with an open source license, however that core product, in-and-of-itself, is insufficient to provide real value nor depth of service, and must be supplemented with other proprietary features (often offered by the company with the copyright for the core open source product). This extended platform (open core + proprietary add-ons) creates risk and vulnerabilities as the organization becomes dependent on the tools, and thus supplier. Phipps continues, “using the package locks you in to the supplier. If they prove a bad choice as a supplier, or if your business needs change, you have no real choice beyond ‘take it or leave it'”.
Indeed, I know of no options other than Instructure who provides hosting/support for Canvas. I do find general hosting services that include Canvas’ core product in a c-Panel/WHM system, but none related to the add-ons (maybe I am ignorant of these, let me know).
Ironically, I understand UC Davis (my alma mater) is migrating to Cavas from Sakai. I would think this experience would make The Ags, leery of such dependencies: out of the frying pan, in to the fire, “whatev!”
Michael does make excellent points about the responsibilities of customers related to back-ups and customizations. These are both very important aspects of managing on IT portfolio that are often neglected. However, this issue is not unique to open source software. Institutions should have disaster recovery for all of their data, not just open source systems. Customers should understand migration issues prior to contracting, through procurement: how many institutions include questions about migration *from* the new system (not just to the new system) in their RFP’s?
Regarding customizations, after 25 years in higher education, I can attest to many examples of customizations made by institutions to their COTS software. Do you know of any institution that thinks they *aren’t* unique, and can run with software, “out-of-the-box?” We all know any company will readily agree to a professional services contract around custom development (I’ll admit I’ve done it). Therefore, the advice Michael gives around customization is just as applicable to proprietary as open source software–oh, and let’s not forget about in-house development.
I also agree with Michael that vendor choice should be part of the due-diligence of selecting an open source system and support provider / vendor. Although it does make me chuckle that, decision-makers will point to the lack of alternative hosts (only two or three) for an open source platform as a significant risk (of not only the platform, but open source software generally), but then readily get in bed with a single- source vendor for proprietary software. But let’s not forget, there is always the option to host locally. I’d offer another option that rarely gets attention: contract with another campus/system which is already running the open source option and thus has the local expertise/infrastructure to host your institution’s open source platform.
I’ll echo the Open Source Initiative’s* query again when Phil first posted on this , why aren’t we asking UC Davis how they found themselves without contractual leverage to force their hosting provider to support them through the life of their deployment? In reality this exact same situation could be happening with a proprietary platform, “Hey why is Canvas down?” … “You’re leaving us, whatev!”
Thanks,
Patrick
(*Fair disclosure, I am the GM and Director of the OSI)
1. https://en.wikipedia.org/wiki/Open_core#cite_note-Riehle-2
2. http://www.computerworlduk.com/blogs/simon-says/open-core-is-bad-for-you-3569652/
pmasson says
To Chris Munzo above…
True, “contractual leverage has few short-term benefits if your hosting provider goes dark.” And this is not an issue specific to open source, but rather a consideration for assessing any vendor. All my points are simply to highlight that this entire issue has nothing to do with open source software, the development models associated with it, the project or foundation that supports it–it is a vendor support issue with lessons to be learned much broader than what has been the focus of many posts and even parts of the original articles of e-Literate.
pmasson says
BTW, no one has answered my questions about what was the contractual status between UC Davis and the vendor.
Michael Feldstein says
Probably because we don’t know the answers to your questions at the moment.
Rodney Tamblyn (@ocnb3) says
Organisations should first have a risk plan for important systems, which at a minimum identifies a range of risks that could occur (fire, getting hacked, vendor failure etc), the impact they might have on the service and organisation, and what steps can be taken to ameliorate them.
Offsite backups should be validated against the latest production version of the product (e.g. ideally spun up on a blank server from sources via an automated deployment process, or a Cloud-hosted instance of the product which you spin up specifically for backup testing). Ideally testing and reporting of backup system should be automated, with an appropriate monitoring/reporting system on top, and periodic review by humans. In practice, these types of things don’t happen as much as they should because they take expertise, time and money to setup, and a shared understanding between service provider and client around the importance of having them in place. You get what you measure (and what you invest in), just because there are a few clauses in the SLA does not mean you are covered…
~ Rodney