As Phil noted in his analysisof the SJSU report, one of the main messages of the report seems to be that some of what we already know about performance and critical success factors for more traditional online courses also seem to apply to xMOOCs. But how good is the ed tech industry at taking advantage of what we already know?
Not very good, as far as I can tell.
By reading the SJSU research report (download actual report here), one item that really hits me is that however different the scaling model is for MOOCs, they are still online courses and have similar success factors. I am not trying to minimize the value of the report by the title of this post, because there is real value in letting objective data lead you to conclusions, even if hunches and assumptions would have led to similar results.
First, however, it would be useful to understand the data and key terms – AOLE is Augmented Online Learning Environment, which represents the additional support structures and for-credit status for MOOCs.
Out of a total of 274 students enrolled in AOLE courses, 249 students remained in the sample after data cleaning, which included removal of students enrolled in multiple courses and of those with no course activity. In addition, 36 students were removed who withdrew from the course or received a final grade of Incomplete. This left 213 students for the deeper analysis. This data set will be referenced throughout this report as a “research data file”.
So this data does not include no-shows, withdraws or incompletes, making it more akin to analyzing a traditional college course after the add / drop period.
Back in late July we found out that San Jose State University was pausing their SJSU Plus pilot program using Udacity for-credit MOOCs due to low passing rates. While there was fairly extensive media coverage of the story broken by Inside Higher Ed, there was the promise of a National Science Foundation (NSF) funded research report based on the spring 2013 courses. I described why the research report was important, despite indications that the lessons learned aligned with well-known practices for online education.
With this many known issues, which have also been documented in several excellent blog posts, why is it valuable for SJSU to use the external NSF-funded report?
In my opinion, there will still be value in A) releasing the full data set and analysis, and B) getting others to understand key points.
Many specialists might understand why the program failed, but many others do not. Campus leaders and policy makers need to learn many of the lessons that are already known in ed tech / higher ed community, and SJSU Plus will help in this area. Plus, I wouldn’t discount some unexpected findings (e.g. need for self-pacing in math courses as mentioned in student surveys). [snip]
It is important for some of the new participants (and I would add foundations, state governments, university presidents, etc) to fully understand what worked and what didn’t work in this application of online education. The official report from SJSU will provide a valuable service to help with this learning process.
There is also value in this very public case establishing a precedent of pausing pilots that don’t work, evaluating results early in the process, adjusting the course and support design based on findings, and focusing on student learning outcomes as the primary measure of success.
After several delays, SJSU has now released the full NSF-funded research report on Elaine Collins’ official web page (she was the principal investigator). The full report can be downloaded here.
This is going to be a more personal blog post than I typically make here at e-Literate.
The open letter from San José State University’s philosophy department in protest of the edX JusticeX course being taught at SJSU is getting a lot of attention, as is the follow-up statement from the SJSU faculty senate. I have some concerns with both of these letters—particularly the one from the philosophy department—but before I get into them, I’d like to emphasize my points of agreement and solidarity with the department:
Despite all this common ground on values that are dear to me, I find aspects of the department’s letter to be deeply problematic.
As Phil mentioned in his last post, he and I had the privilege of participating in a two-day ELI webinar on MOOCs. A majority of the speakers had been involved in implementing MOOCs at their institutions in one way or another. And an interesting thing happened. Over the course of the two days, almost none of the presenters—with the exception of the ACE representative, who has a vested interest—expressed the belief that MOOCs provide equivalent learning experiences to traditional college courses. Keep in mind, these folks were believers. They were enthusiastic about MOOCs in general. But they tended to describe the value of MOOCs as reaching a different audience than the traditional matriculated college student and provide a different value. They talked about it extending the university mission. By and large, they did not talk about it as being an improvement on, or even equal to, a traditional class. Now, there were well over 400 participants, so it wouldn’t be fair of me to say that there was unanimity, about this point or any other. But the level of agreement was remarkable.
On the other hand, there was widespread enthusiasm for using MOOCs as essentially substitutions for textbooks in classes that included instructors from the local campus. Vanderbilt created what they called a course “wrapper” around a Coursera MOOC on machine learning. Folks from Stanford talked about the notion of a “distributed flip,” i.e., a group of flipped classrooms participating together in a MOOC. And SJSU talked about using an edX course in a blended course environment on one hand, and a Udacity course with Udacity-provided “course mentors” on the other.
The obvious conclusion is that MOOCs are more of a threat to textbook companies than they are to universities. I think that’s true, but I also think it’s an oversimplification. There is a deeper (and older) trend to boil down a course into a set of digital artifacts that can be “played” by the student at will. It’s worth taking a deeper look at that trend, where it’s going, what’s useful about it, and what’s pernicious about it. (more…)
No, this isn’t just an attempt to cram as many sexy keywords into one post title as possible. xMOOCs and adaptive analytics share an ambition: They both are at least partially motivated by a desire to teach at scale. With MOOCs, the goal is obvious. With adaptive analytics, less so, partly because there are multiple motivations and maybe because the desire for scale is not something that is polite to talk about due to a certain amount of discomfort with it. But the motivation is definitely there if you look closely, as I’ll get into in a bit.
The problem is that both of these approaches, in their current incarnations, miss one absolutely critical element of the teaching process. As you can probably guess from all the qualifiers I am using in my language, I don’t think it’s an inherent or permanent failing. But I worry that it’s a failing due to a deep cultural blind spot that we have about what education is, and that it therefore will be challenging to address.
