Update (3/10): Patterns and descriptions have been updated based on feedback in a new post. Added links to Astronomy, AI Planning courses.
Thanks to feedback from my last post, I have modified the proposed description of patterns for students engaged in MOOCs. I also want to introduce a graphic to visually represent these patterns.
- I have removed the language comparing passive participants to traditional students based on the idea that they expect others to define academic goals for them and ‘expect to be taught’ (thanks to Colin Milligan for description). While this distinction between passive and active participants is important, I have removed the direct reference to traditional students – the reader can apply their own comparisons.
- I have removed the usage of the term archetype. As Satia Renee put it so well on Google+, archetype implies “more an internal personality type expressing itself in patterns of behavior” when I am trying to capture the patterns of behavior. Thus I’m sticking with the less loaded term of patterns.
- I have added language, thanks to Kevin Kelly, that captures the growing case of Drop-Ins as students focused on a particular topic within a MOOC for usage outside of that MOOC.
- Finally, I have moved Drop-Ins right after Lurkers based on Colin’s comments and to help with the graphical view below.
As a recap, I believe we are seeing the following four patterns of student behavior within MOOCs:
- Lurkers – These students are the majority of xMOOC participants, where people enroll but just observe or sample a few items at the most. Many of these students do not even get beyond registering for the MOOC or maybe watching part of a video.
- Drop-Ins – These are students who become partially or fully active participants for a select topic within the course, but do not attempt to complete the entire course. Some of these students are focused participants who use MOOCs informally to find content that help them meet course goals elsewhere.
- Passive Participants – These are students who view a course as content to consume and expect to be taught. These students typically watch videos, perhaps take quizzes, but tend to not participate in activities or class discussions.
- Active Participants – These are the students who fully intend to participate in the MOOC, including consuming content, taking quizzes and exams, taking part in activities such as writing assignments and peer grading, and actively participate in discussions via discussion forums, blogs, twitter, Google+, or other forms of social media.
An important point is that some students change between patterns – such as a passive participant deciding to fully jump in and become an active participant, or even an active participant becoming frustrated and becoming a lurker. From what I’ve seen, this type of change occurs once per course at the most for any individual student.
There are still some good comments coming into the original post, so I will probably refine the definitions over time.
Additional Notes on Behavior
I have already described the data captured by Katy Jordan on MOOC completion rates. Note that this data compares the ratio of students completing a course to total number of students registered.
There are several courses *, typically on Coursera, where we now have a deeper description of the student behavior based on information shared by the professors. I suspect this view will be different between xMOOCs and cMOOCs, and even between different MOOC providers. For now, treat these observations as primarily based on Coursera-style MOOCs.
The majority of students (60 – 80%) reported as registered in a course are lurkers who tend to leave the course completely by the second week and may not even engage with the material in any significant way.
- IHTS: 46k registered, by week 2 there were 11.6k who “completed week 1”
- EDC MOOC: 46k registered, by week 2 there were 7.4k logging in
- Bioelectricity: 12k registered, 8k watched any videos, 3k watched week 2 intro
- Microeconomics: 37k registered, 25% watching video during week 2
The courses seem to stabilize by week 2 or 3 in numbers of students still in course. While this observation is mostly based on anecdotes from blog posts, there are two charts capturing the data for Bioelectricity and EDC:
Graphical View
How would these student patterns appear over time, at least for those courses similar to the Coursera MOOCs with intermediate data? I believe the following graphic captures the basic shape and topology of student patterns. Note the graphic is a generalization, and ideally we would have this type of diagram for each course.
* Links to information on specific MOOCs with intermediate data provided by professors:
Phil, I like how you are characterizing this. I think that you need to separately consider, define, document and count the no-shows. Right now in my IHTS couse, I have 27,752 enrolled and at the end of the first week only 10,202 have even logged in. With no-shows likely around 50%, it is a mistake to (a) lump them in with lurkers and (b) characterizing them as dropping off in numbers as the course progresses. The data you are looking at is important – but I think that you should separately label no-shows.
Charles, that is a very good point – particularly a) about needing to separate no-shows from lurkers. The majority of the drop off in week 1 are no-shows.
I did, however, intend to show lurkers dropping off during the course, as active participants tend to have the highest retention week-to-week (anecdotal). Do you have data that shows lurkers past week 2 (i.e. those logging in but not watching majority of videos) and whether they drop off over time or not?
Our study recruited participants in Week 13 of Change11 (a cMOOC rather than an xMOOC) so I can provide no hard data/evidence about the initial tail off.
In my ‘lurker’ group (and the discussion on your previous post shows that the term is problematic), lurkers were highly engaged with the course (albeit with the content rather than the community) and would not be significantly more likely to drop out than ‘active participants’.
I were to speculate on which group would be most likely to tail-off in a cMOOC then it would be the ‘passive participants’ group – as it is this group whose expectations would be most ‘at odds’ with the course experience.
Phil – I also *really* like your approach here – brings out one of the most salient points of success to consider for MOOCs.
Chuck’s suggestion was my main reason to comment. I’d suggest that we don’t even need to *count* the people who register but don’t log in – I think counting those numbers is a residual artifact of conventional courses, where there’s a strong institutional incentive (e.g. limited seats), and student implications for not showing up (e.g. grades, cost).
Second – a word beyond “pattern” seems appropriate for these groupings – typology? ontology? To me, the patterns are related to deeper issues of student motivations and commitment; which is more than a collection of actions.
Thanks for the good work!
John, if I were designing the MOOC phenomenon, I would not count the “no-shows” at all. Given reality, where the MOOC providers not only share the total registrations but tout them, I think we have to count no-shows to help understand the data in the public realm.
Good point – although I’ve heard some advocates (well, founders) speak both ways: a) mentioning the total registrations, then b) saying the no-shows are insignificant, because you can register in 5 seconds while drinking a beer (that’s a quote, not my slant). So counting them is probably the safest bet, until they stop mentioning them – but in our writeups about efficiency/effectiveness about MOOCs, I think that I’ll choose to ignore them if *I’m* the one writing.
If there was a qualitative study done with individuals from these archetypes, which would be the one that would be most interesting? What questions would be most illuminating to ask?
Yes
I think noshows are 90 % or so .
Can we collect data from every one of the courses finished.
Registrations
No shows
1st week drop outs
2nd week drop outs
…
…
15 weeks drop outs
Exam takers
Exam passers
Such a data must be very useful.
Any body can do that .
I also wish Coursera never talks about total number of registration. That is a false information . But they like to do that .
Nancy, the best qualitative research is from Colin, but that was for a cMOOC.
http://worklearn.wordpress.com/2012/12/19/change-11-srl-mooc-study-initial-findings/
There is also an article in today’s IHE on the subject (not directly to your question, but worth reading):
http://www.insidehighered.com/news/2013/03/08/researchers-explore-who-taking-moocs-and-why-so-many-drop-out
As for illuminating questions, I would suggest these three:
– For all types, what were your goals entering MOOC?
– Did you achieve your goals?
– For those dropping out or becoming a less active type, what were reasons?
Mgozaydin, Anecdotally, the no-shows appear to be 60 – 80%. “We” can’t collect data on every course, but the providers (Coursera, Udacity and edX in particular) can and do. I do not know if they are analyzing the same issues internally.
Agree on Coursera total number of registration usage.
I am in fact glad to read this webpage posts which contains good information related to education thanks to providing such information.
Regards
Abhimanyu Poddar
One thing that should be taken into account is that many of the coursera courses are being given for the first time, so when they are announced people sign up without knowing, the commencement date or details of the course. When the course finally starts, those signed up have to decide whether the course is what they want to do based on other commitments in their lives or other coursera courses that may be running concurrently.
I have now completed 3 coursera courses, but have dropped out of at least half a dozen because they didn’t fit in at that time. With some the workload was just too high (Probabilistic Graphical Models by Daphne Koller), some clashed with other courses, some just didn’t look as interesting as they did when first announced. So just because lots of people drop out of a course doesn’t mean that hey aren’t active participants in some other course.
As the Coursera offerings mature and are scheduled more predictably, the dropout rate may reduce somewhat as people are able to plan ahead more effectively.
Andrew, interesting points.Have you seen the updated model, which adds a new category of No-Shows?
https://eliterate.us/emerging-student-patterns-in-moocs-a-revised-graphical-view/
I certainly see people taking on different roles in different courses. That is why I changed the language of the student types. Originally I called them ‘student archetypes’, but that implied that intrinsic identities for students that would span courses rather than students choosing to take different roles based on how the courses fit within their lives.
Beyond the two issues you mentioned on future reduction of dropout rates (offerings mature and schedules become predictable), I also wonder if the novelty will wear off, leading to fewer people signing up just to participate in the shiny new thing. Over time I could see a reduction in the number of No-Shows.
I can only speak for myself, but I don’t think the novelty will wear off in the short term because there are so many interesting courses being added, far more than I have time to do. Like many people I only have the spare time to do one coursera course at a time (although having said that I am doing both “Synapses, Neurons and Brains” and “Computational Neuroscience” at the moment, simply because they were scheduled at the same time) so the only viable strategy is to enrolled for a course when it is announced, effectively bookmarking it, and making a decision whether to proceed with the course when it commences.
I think rather than focussing on the dropout rate of MOOCs, it is far more important to highlight the impressive number of completions. My first Coursera course was Andrew Ng’s Machine Learning, which I believe had about 35,000 enrolments and about 5,000 completions. It was en excellent course with difficult subject matter and programming assignments that required a high level of understanding. The fact that 5,000 people completed the course with a passing grade, many in countries where they would have limited access to higher education, is a significant achievement.