IMS Report on Learning Technology Satisfaction and Trends

The IMS has issued a new report with the somewhat imposing title, “IMS GLC Learning Technology Satisfaction and Trends, North American Higher Education, 05 February 2007 Update.” You can get it here, but first you’ll have to register with the site (if you haven’t done so already). It’s important to read Section 2 of the report (and particularly the “Survey Approach and Disclaimers” subhead) carefully; it is very difficult to get definitive data for this kind of research, and the authors of the report were very careful to spell out the limitations of their methods.

Nevertheless, there’s lots of interesting stuff here. Here are some of my own observations, in no particular order:

• The amount of money spent on content and content authoring tools dwarfs everything else. Institutions in the survey appear to be spending nearly twice as much money on pre-packaged content as they are on their learning management systems. If you add to that the investments in authoring tools and content management systems, it becomes crystal clear that making high-quality, free, re-usable content available (and findable) is absolutely imperative from an institutional cost perspective. This is one area where the bean counters and the ed tech geeks have interests that are aligned.
• The authors of the report express surprise at the low level of investment in assessment software, given all the talk about accountability in higher education lately. But there are a number of factors that can account for this difference. First, the intensity of focus on accountability has jumped quite a bit in the last year and software sales/adoption cycles in higher education are long. Uptake of assessment software is likely to be a lagging indicator, not a leading indicator. Second, every major LMS on the market today has some built-in assessment capabilities, and these may be enough for most institutions. Most importantly though, is that today’s assessment software is at best one small and weak component in an accountability strategy.
• Unsurprisingly, LMS’s didn’t rate particularly highly on the range of learning technology tools in terms of stakeholder satisfaction. Within the category, though, eCollege, ANGEL, and Moodle received the top three rankings (in that order). According to the survey authors, these three products “scored significantly ahead of the other products rated.” Now, there may be an issue with sample size, so I wouldn’t read those results too finely. Nevertheless, the broad-brush conclusions are probably pretty valid.
• ANGEL kicked butt. They scored #1 in learning object repositories and #2 in learning management systems, with very, very high customer satisfaction ratings overall. These guys have always impressed me as being extremely customer-focused; I’m not surprised that they did so well.
• While Wikipedia and iPods were on the radar, most of the Web 2.0-ish tools that all the cool kids talk about did not get on the collective radar of the respondents. This reinforces my own belief that, contrary to the criticisms it received, NMC’s 2007 Horizon Report is not overly conservative. If anything, it’s optimistic.
• The authors anticipate that up to 25% of responding institutions could switch learning management systems in the next 12 months. Particularly given the fact that 12 months is incredibly short-term for this sort of migration, that number is staggering. When you couple that with low levels of satisfaction with the product category, it indicates to me that institutions will quickly jump to non-LMS alternatives for learning environments (e.g., an LMOS) as soon as they can find something that they can feasibly support institution-wide. In connection with this trend, I note also that the report indicates portals are finally taking off. So that key piece of enabling technology may be more widely available. (More on critical developments in the evolution of the portal in a future post.)
• The section on diffusion of best practices indicates that…well…there isn’t much of it. To me, the data strongly underlines the need for something like the e-Learning Maturity Model (eMM) that Stephen Marshall advocates.

There’s lots more. Go get the report and read it for yourself.