How many times have you heard the statement that ‘MOOCs have a completion rate of 10%’ or ‘MOOCs have a completion rate of less than 10%’? The meme seems to have developed a life of its own, but try to research the original claim and you might find a bunch of circular references or anecdotes of one or two courses. Will the 10% meme hold up once we get more data?
While researching this question for an upcoming post, I found an excellent resource put together by Katy Jordan, a graduate student at The Open University of the UK. In a blog post from Feb 13, 2013, Katy described a new effort of hers to synthesize MOOC completion rate data – from xMOOCs in particular and mostly from Coursera.
Via a combination of thinking about ‘what makes a successful MOOC?’, and looking for a topic for my final project on the Infographics MOOC, I decided to try to pull together the various statistics floating around online about MOOC completion rates. I’m trying to see if any differences emerge on the basis of platform or the assessment methods used.
My draft graph synthesising everything I’ve found so far can be found here: http://www.katyjordan.com/MOOCproject.html
What Katy has done so far is compile completion rate data from 24 MOOCs – 19 from Coursera, 3 from edX, 1 from Udacity and 1 from MITx (a precursor to edX). The data is available in a table format or in a graph.
As you can see on the right side, you can filter by MOOC provider, University or Assessment Type (auto grading, peer grading or some combination).
In the graph view, you can see completion rates vs. total enrollment of each MOOC (the colors represent different Assessment Types).
By clicking on each dot, it highlights the MOOC, basic data and link to data source.
This is the most thorough summary of MOOC completion rate data that I have seen – for xMOOCs. Kudos to Katy for putting together this data, and please comment on her blog post to let her know of new data available.
Some notes based on the data available:
- The average completion rate of xMOOCs is 7.6%, with a minimum of 0.67% and a maximum of 19.2%. The 19.2% appears to be an outlier from Ecole Polytechnique Fédérale de Lausanne, although it may be worth figuring out how they got their rate so high.
- Does it bother anyone that we get this data from a graduate student from The Open University but not from any of the xMOOC providers who claim the power of data analytics in their platforms?
- In the end, the meme of 10% MOOC completion rates is not too far off, at least for Coursera courses.
The problem I see is that scalar metrics of # of students and completion rates are insufficient for open courses, where there are different student types. Only a subset of students actually plan to complete a course, whereas the majority of students enrolled are there only to sample and explore. Measuring completion rate off the total number of students is misleading, but so is touting the total number of students enrolled (billions served) as a key metric. This is not a problem with Katy’s data, but with the common discussion of MOOCs in popular media and even higher education circles. I’ll address a different model in my next post.
Discuss here or at Google+ post.
It’s the backend of the donkey! http://cogdogblog.com/2013/02/25/back-end-of-the-donkey
Percentages of finishers out of total entering open signups are the weakest, yet only (partial) measure we got. Completion needs to be tied to the intended goal of the learner is hopefully where you are going.
I love it when worlds come together – Katy is my phd student. Thanks for bigging up her work, she’ll be chuffed.
Just because the MOOC providers don’t release this data doesn’t mean they don’t have it. They are commercial concerns and this is commercially sensitive data.
Alan, yes, I plan to go more on different learner types within open education, each having different intended goals. By the way, nice post if disturbing image.
Martin, after some research I discovered that Katy would be pleased:] some great work there.
AJ, I’m sure the providers have this data, but I question the value of not releasing it. Open education is a different beast, and hoarding all data can be dangerous, especially if you make your living off of media narratives and momentum.
At the very least, there should be some release from providers on how they are using such data to benefit learners. My $0.02.
I am forced to wonder why we are focusing on “completion rate.” Do we know this construct aligns with “participant met his or her goals?”
I am forced to wonder what the true “completion rate” of my undergraduate colleagues was those decades ago… did more than 10% (or so) complete? (I expect so) achieve his or her goals in every course? (I have my suspicions).
Today, I reviewed a project completed by a participant in #edcmooc She posed to herself the question “Am I really learning in MOOC’s?” (my paraphrase). She answered her own question, “who cares.” She found her own utopia.
Cipher with that 🙂
Hi Phil, thanks for featuring my work and helping to bring it to a wider audience. I’m really pleased to see all the discussions that it has sparked, both about openness of data, and meanings of success in the context of MOOCs 🙂
A lot of people have asked whether completion rates are the right way of framing the success of a MOOC; I agree that there is much more to the potential positive impacts of MOOCs for students than completion rate but, at the moment, completion rate is what the providers are measuring most consistently.
Katy, thanks for your work. I agree with you that completion rate is the data available at the moment and therefore worth considering / analyzing.
Thank you Katy for your great work
Thanks to you too Phil for indexing it.
Thanks Kathy
I just asked that yesterday and I found your message today .
1.- Enrollment are high.
Because
1.1. Registration means just one click. I had done it 15 times so f ar .
1.2. Students even do noıt bother to view the first lecture since they have no obligation. They did not invest any money, they will not get any credit. Just they will learn somehow something they wondered just for a moment during clicking for registration.
1.3 Some, few, really wanted to learn something just go on .
Then that is the result.
From this application MOOCs are a failure : But only application is a failure .
If a MOOC has credit and degree at the end
People would be willing
1.- To pay a reasonable amount , say $ 100 or so
If they were going to be provided with a credit and a degree at the end .
Only a 1,000 enrollment for 10 semesters at $ 100 would make any MOOCs
effective. Then income in 10 semester is 1,000 x 10 semesters x $ 100 = $ 1,000,000 that is enough to cover any cost of that online .
To continue MOOCs as it is waste of money and resources + waste of time of the sstudents too .
Schools must
Start charging something like $ 10 to $ 100 per course
Start providing credits and degrees
Satisfy themselves with 1,000 enrollment per semester and get better teacher students contacts
Also makes money for themselves .
Another model is
Market MOOC courses among the 4,400 colleges in the USA at a small fee let
the local college provide exams + credits and degrees
Second model is easier to apply and a quick one too . Already there are some colleges doing that . Antioch + Colorado State Uni + San Jose State + some more .
MOOCs are great if they are applied in the right way as I have explained above . It is wibn win case . Let us do not miss the boat .
Let us be careful.
There are good MOOCs and there are bad MOOCs .
Investigate, ask your friends, then decide to get a MOOCs course .
There are some very very good ones and also there are many bad ones .
I usually go for the elite schools for security. Brand means good .
Kathy
Can you set up some organisation to get a complete data regularly covering all courses . I am willing to pay $ 100 per month subscription for such information . There may be many too . Thanks billion again .
If Coursera will not decide on their business model to finance the initiative, I am afraid they will bankrupt soon .
Edx non profit solved their problems .
They will collect from their member providing schools $ 50,000 and or $ 10,000 plus 50 % of what they collect . If they help developing online course too then they ask for $ 235,000 per course . Not bad . Non profits
became very profitable now . To me it is just not good . That should not be
done to member schools .
Katy
Please what is the latest data . Thanks billion.