Update on UC Davis LMS Fiasco: Finishing the term with two partial systems

After the LMS outage that started May 20th – covered here, here, and here at e-Literate – UC Davis has finished its spring academic term as of June 9th using two partial systems, one for faculty and one for students and neither of which is fully functional. In other words, UC Davis never fully recovered its LMS (Sakai system branded as SmartSite) functionality from the outage. UC Davis staff have indicated they will provide more information to us by interviews and public records, but they have not said when they will be ready to talk. Scriba (the Sakai commercial affiliate and hosting provider that caused the outage) has not replied to requests for an interview or statement. When and if these occur, I’ll post updates.

For a short timeline:

• May 19th: Scriba notifies UC Davis of an emergency maintenance planned for May 20 – 23 as they changed data centers. Regarding the data center move, CEO Michael Sanders stated “The failures are as of a result of a third party and are outside of our control.” (irony alert inserted here)
• May 20th: The outage begins on Friday evening with plans to have system available Monday morning. This outage affected all Scriba customers, also including UC Santa Cruz, Limerick University, and Providence College. UC Davis backed up course data locally before the outage began.
• May 23rd: Monday morning rolls around with no recovery of UC Davis system.
• May 24th: With no information from Scriba on when and if they can recover SmartSite, UC Davis begins plans to build its own locally-hosted instance of SmartSite using the May 20th data backup. Note that Sakai is open source and that UC Davis has full access to their own source code.
• May 27th: UC Davis gets close to re-building SmartSite but notes that full functionality would not be ready before the end of the term on June 10th. In particular, the campus-hosted system would not be able to support student access, presumably due to the scale of 30,000 students. After our first post on the outage Friday morning, a source from Scriba notifies me by email Friday afternoon that they have their system recovered.
• May 28th: UC Davis proceeds with faculty-only testing of the campus-hosted system but also notifies students that the Scriba-hosted system is also available. The campus-hosted system becomes the “system of record” and will be used to send grades to the student record system. There is no integration between the campus-hosted system that faculty and TAs can use and the Scriba-hosted system that students can use. Faculty are instructed on how to export grades from one system and manually upload into the other system. This is how UC Davis finishes its term – with two partial systems.
• June 6th: UC Davis notifies faculty that the campus-hosted system will be a “a fully-functional system available to both faculty and students in time for Summer Session 1, which begins on June 20”. UC Davis had decided earlier this year to move to Canvas as its LMS, and Canvas is also available for Summer classes.
• June 9th: Spring term ends.

In addition to the confusion of having two partial systems, both SmartSite instances continue to have availability problems. Based on the UC Davis notification system (and again I would like to commend UC Davis staff for such timely and descriptive information throughout this crisis):

• June 7th: Campus-hosted SmartSite has degraded service from 8am – 2pm.
• June 9th – 10th: Scriba-hosted SmartSite has several outages and degraded service periods.
• June 14th (today): Campus-hosted SmartSite has degraded service starting at Noon.

The Scriba-hosted system has a reported uptime of approximately 72% during the past month.

I plan to add some more analysis to this whole episode, including commentary on the open source angle, but I am giving one more try this week to get UC Davis staff and Scriba management time to give their insights first.