Instructure has had a very interesting reaction to the news and blogs about security vulnerabilities with Blackboard’s Learn LMS several months ago. They have decided to engage Securus Global, the same firm that did the ethical hacking for the Australian universities in the Blackboard investigation, to test Instructure’s Canvas LMS product. They have also invited me to be essentially an embedded reporter – participating in the process and independently reporting on the testing and Instructure’s response to any vulnerabilities identified.
While doing research for a my post analyzing Blackboard’s response to the reports of security vulnerabilities, I had the opportunity to interview several LMS vendors to get background on their philosophies and practices around security. I think it is important to understand how the broader LMS market is handling security concerns, especially as the LMS has become such a central part of the an institutions’ academic operations.
In the post I argued that we need more transparency in the LMS market.
We need more transparency in the LMS market, and clients should have access to objective measurements of the security of a solution. To paraphrase Michael Feldstein’s suggestions from a 2009 post:
- There is no guarantee that any LMS is more secure just because they say they are more secure
- Customers should ask for, and LMS vendors should supply, detailed information on how the vendor or open source community has handled security issues in practice
- LMS providers should make public a summary of vulnerabilities, including resolution time
I would add to this call for transparency that LMS vendors and open source communities should share information from their third-party security audits and tests. All of the vendors that I talked to have some form of third-party penetration testing and security audits; however, how does this help the customer unless this information is transparent and available. Of course this transparency should not include details that would advertise vulnerabilities to hackers, but there should be some manner to be open and transparent on what the audits are saying.
Securus Global is a global security company that helps clients interested in “quality security advisory, assessment and assurance services”. Although they perform the full range of security consulting that includes strategy and management, Securus is best known for their testing engagements. In this context, they can test applications and systems for security vulnerabilities and advise clients on how to improve security. Typically ~90% of their work is for clients of an application (e.g. the Australian universities wanting to verify if their LMS has vulnerabilities), rather than the 10% where the work is for the application vendor themselves. It is preferable for the vendor to do their own testing up front, but at least for Securus Global customers it is usually the application client who requests the service.
To Instructure’s credit, they are asking for the testing before clients have reason to request these tests. Furthermore, with this embedded reporter concept, Instructure is taking the additional step and risk of not controlling the message. Securus indicated that this was the first time they had a vendor include an independent party in this manner. They congratulated Instructure for their bold approach, although using more colorful language.
Let’s be clear – we do not know what the results of this testing will be and whether it will look good, bad or indifferent for Instructure. Securus has indicated that the vast majority of testing results in some level of critical vulnerabilities. As noted in my original post:
In the interviews, all LMS vendors acknowledged that no web-based software is perfect and you should always expect some vulnerabilities. The issue should not just be on whether there are vulnerabilities, but perhaps more importantly, on how a company or organization responds to a security vulnerability or incident.
Based on my call for more transparency, how could I refuse the offer? Here is a vendor who is willing to have information shared from a third-party security test and to not even control the reporting of this information.
I am receiving nothing for this offer other than the chance to practice what I preach about transparency. While I assume that Instructure will have their own description of the process and results, I will keep my writing independent. I will get input from Securus to ensure that the testing and reporting do not jeopardize any of Instructure’s clients, which could affect the timing of my posts.
The result will be a series of posts reporting on this process over the next few weeks – both the results of the testing and analysis of Instructure’s response to the results. We have already had a conference call introducing me to the Securus team doing the testing, and I will be allowed to review reports coming from Securus, participate in phone calls where Securus shares results of testing, and follow-up directly with Securus without Instructure’s involvement where appropriate.
I have deleted some content-free snark by a comment troll. The rules here at e-Literate are (1) criticism is fine but must have substance, and (2) you must use a real email address.
What assurance are you going to provide?
Peter, I’m not sure I fully understand your question, so let me know if this answer works.
The arrangement for the actual testing is between Instructure and Securus Global, and they are taking the necessary steps to ensure the testing itself does not cause any client problems. This is standard fare for security testing, so this should not be a problem.
My role is to report and analyze the results. The primary situation I have to avoid is describing the specifics of any vulnerabilities prior to the issues being fixed. This is why I mentioned that the timing of my posts might be delayed. I plan to describe the high-level results, without specifics, but timed such that no vulnerabilities are prematurely exposed.
we’re pretty sure securus global will find some kind of vulnerability, possibly even a serious one. this, in and of itself, is not terribly interesting or surprising – all software systems have security flaws. the key is identifying it and taking responsibility for it and then fixing it – fast. as the security expert bruce schneier said about security flaws: “You have to make the entity in the position to solve the problem, responsible for the problem. Otherwise, it doesn’t get solved.”
a native cloud system like canvas has a fundamental advantage over on premise and/or hosted software systems in that we are able to continuously update our software – there’s no such thing as an “old version”. legacy architectures have version numbers associated with them, which is a sure sign that they require fork-lift upgrades or emergency fire-drill patches to keep up to date. waiting six months for major bug fixes can be irritating, and waiting that long for fixes is putting the integrity of your entire educational institution at risk.
as far as i’m aware, this is the first time that an “embedded reporter” has been involved in a real security audit of a software system. here at instructure, we’re optimistic that it will be an interesting, educational experience for everyone.
I’m interested in the “embedded reporter” model in itself, Phil (and Josh). I can understand why people might be wary, but I also think what you’re doing is a good way forward, in building this more open partnership that several of us are now carrying on about. It makes sense to have the commentary written by someone else, just as it makes sense for Phil to have mapped out this clear explanation.
But the terminology doesn’t seem quite right to me, though, given the very awkward history of embedded journalism. No matter how much you ironise it, it’s a term that introduces a whole lot of baggage that doesn’t really need to be there.
Why not just say “independent observer”? Because if that’s not what this is about, then I don’t think it would be happening, because Phil would also then have something to lose.
I’d love to see this model used in a whole lot of other edtech scenarios. Having just staggered through an LMS evaluation that became positively Homeric in its duration, I can’t help feeling that if vendors were more less risk-averse about these evaluations and allowed some of the testing outcomes to be shared, educational institutions (who are actually very used to doing a whole lot of stuff in public, including institutional audit) would in turn be impressed by the goodwill, and there’d be much less duplication in the process.
I could understand present practices of industrial secretiveness if anyone genuinely had anything to hide, but the major LMS features are now fairly generic, in the same way that cars have four wheels; and none of them have major flaws, unless you count teeth-grindingly conservative design as a flaw. Do more in the open, and we can all have a bit more time to do laundry.
Most of the RFP’s I’ve seen for LMS systems include requests for security testing practices and reviews. The claim that they are doing this as a pro-active measure before clients even asked is not credible. It also doesn’t seem like they’ve chosen their security vendor other than because that company found an issue in Blackboard and then issued a press release to drum up business.
Blackboard does ongoing security testing and number of 3rd party vendors who audit and test its software products and services. It has dedicated staff focused in secure development, testing and delivery of services. The Blackboard Learn app includes a number of direct countermeasures to web application vulnerabilities.
More info here.
http://www.blackboard.com/footer/security-policy.aspx
Kate, the reason I used the phrase ’embedded reporter’ is to capture the concept that I have been invited in to the process as it happens, rather than digging up information after the fact. I do realize that the phrase may have some baggage, but I didn’t have another phrase to capture this aspect. Independent observer does capture the rest of the concept and could also be sued. You should thank my colleague Katrine Maguire, who convinced me not to use ’embedded reporting’ in the post title (or perhaps I should thank her).
Good point on need for more transparency in other edtech evaluation processes.
John, you are correct that most RFPs have requests for security information, and I did not mean to imply that no client or potential client of Instructure had previously asked for security testing. What I’m calling out is that Instructure is doing this somewhat public testing without any known security issues to trigger the investigation, and they are doing so in a manner where they do not control the message.
The real point of my call for more transparency, and why I’m interested in this independent observer role (see, I’m learning, Kate) is that all LMS providers I talked to have ongoing security testing. What has been missing is transparency in the process, with independent parties describing the results rather than the LMS providers themselves. I have found this to be true even in most RFP processes.
There are some situations where the institutions pay for their own security testing of LMS products during an evaluation, but even in this case, the information is kept private to that specific evaluation and not available for other institutions.
The transparency that I’m suggesting is “that LMS vendors and open source communities should share information from their third-party security audits and tests” to the market in general.
I love the non-traditional approach. It speaks to Instructure’s culture of thinking and doing things differently. As others have, I’d also caution to hold back on providing any details of security-related issues until after they are patched. Because Canvas is delivered SaaS, it will be very easy to update everyone together.
Blackboard differs in that some clients are hosted, others are on-premise, and every client chooses to upgrade/patch on a timeline that each individually sees fit. As such, Blackboard controls less of the complete end-to-end experience, except for hosted clients where it is possible to address issues by patching en-masse or via an Intrusion Detection System or firewall rule.
I, personally, am in favor of working towards some sort of industry best practices document. Is there an organization that everyone can partner with to facilitate such an effort? It seems we’d want everyone involved – D2L, Moodle, and Sakai too.
@John,
Your comment below is wrong. Where did you get this information from? All the media/journalists got from us was a “No Comment”.
“It also doesn’t seem like they’ve chosen their security vendor other than because that company found an issue in Blackboard and then issued a press release to drum up business.”
How Instructure got to us to is more than likely through the press around the Blackboard vulnerabilities. We do this type of testing for companies all around the world.
If you check the media, you’ll see we haven’t commented anywhere. We don’t need to resort to scare tactics and what you insinuate to get business. Our reputation and work speaks for itself.
Personally I think this is a brave move from Instructure and we to are interested in seeing how it all pans out. We’re not interested in any BB v Canvas arguments. If more companies, more proactively approached the security of their systems, how is this a bad thing?
Drazen Drazic
CEO
Securus Global
George, one of the reasons that I am enthusiastic about the Sakai/Jasig merger is that, in the new organization, we’ll be able to undertake instructional technology projects that are more ecumenical and not Sakai-specific or Sakai-only or even open source-only. I think an LMS security best practices group, which could both document best practices and maybe even work toward increasing transparency and comparability across platforms, would be a great project to host in the new organization.
To clarify my earlier comment. I did not intent to imply anything with regard to Securus Global. My comment is merely regarding Instructures securty by press release action. Sorry for the poor sentence strucuture.
@john,
instructure never issued a press release related to security. what press release are you referring to?
@John
Not long ago Blackboard tried to “drum up business” by trying to sue D2L out of business. Please get off your moral high chair and start thinking about how you might serve you existing clients better.