Robert McGuire wrote an article for Campus Technology, Building a Sense of Community in MOOCs, that touches on an important topic – is the centralized discussion forum a barrier to student engagement?
But more students can also mean more isolation within the crowd. “Online classes can be really lonely places for students if they don’t feel like there’s a community,” notes Maria Andersen, director of learning and research at Instructure, which runs Canvas Network, an open repository where participating schools can deliver their own MOOCs.
Ironically, the biggest obstacle preventing MOOC students from forming relationships is the feature most relied on to encourage them. Discussion forums are the number one complaint by readers and contributors of MOOC News and Reviews, an online publication devoted to critiquing individual MOOC courses and the evolving MOOC landscape. Most MOOC discussion forums have dozens of indistinguishable threads and offer no way to link between related topics or to other discussions outside the platform. Often, they can’t easily be sorted by topic, keyword, or author. As a result, conversations have little chance of picking up steam, and community is more often stifled than encouraged.
There are several studies that appear to show that MOOC discussion forums have few students participating and that the forums are dominated by a small number of students.
edX
From a study on the first edX MOOC:
However, we know that, on average, only 3% of all students participated in the discussion forum. Figure 10 below illustrates the small number of posts the vast majority of students actually made. But we know that certificate earners used the forum at a much higher rate than other students: 27.7% asked a question, 40.6% answered a question, and 36% made a comment. In total, 52% of the certificate earners were active on the forum. We are analyzing the number of comments individual students posted to see if it is predictive of that individual’s level of achievement or persistence.
Stanford
More recently, there is a study from Stanford looking at discussion forum activity across 23 separate MOOCs on the Coursera platform. Across all registered students, no MOOC had more than 10% of students posting on a forum, and most were below 5%. Note that they measured students having only one forum post (typically the introduction forum) and those with more than one.
The team then excluded all students getting less than a 10% as a grade, which removed 86% of registered students. The rate of students posting to the forums rose significantly.
The Stanford study also found a reverse correlation between the course size and the percentage of students posting.
Several people whom we discussed this data with asked whether there might be an inverse relationship between the size of a class and the % of students who post. For instance, if students mainly use the forum to answer questions and check for existing answers before posting, they may find that in a larger class there’s less need to post, since their questions are already addressed.
Excluding two outlier classes, and looking again only at students who scored at least 10% in a class, and only at students who posted more than once (so that introductions are excluded), we see that there’s only a very small inverse correlation between size of class and % of posters (correlation value is -0.4):
Edinburgh
A third source on the low engagement of MOOC students in discussion forums comes from the University of Edinburgh and their study of 6 MOOCs. Here we see that students are far less likely to engage in discussion forums than in videos or assessments.
What we are seeing here matches the lessons from the early cMOOCs, as described by Stephen Downes.
It’s interesting that this article [a post based on the Campus Tech article] addresses a lesson we learned in the first few weeks of our MOOC in 2008 – the centralized discussion forum is not a good tool for a course of thousands of people.
Indeed, Robert McGuire also commented on this same post with similar conclusions:
I’m surprised at how many classes rely uncritically on discussion forums when ten minutes of experience reveals how inadequate they can be, at least without more thoughtful management of them.
While the emergence of MOOCs is still quite young, I think it is becoming quite clear that certain elements can scale quite effective (videos, quizzes), but that centralized discussion forums do not scale. For MOOCs to be more effective, we need to see different approaches to student engagement.
Outlier at Duke
I should mention that the Duke report on the Bioelectricity MOOC could be an outlier in the positive reviews of students on the discussion forums. This data is from a voluntary survey at the end of the course, so there is obviously a self-selection bias, but it is worth noting these results.
In addition to overall course satisfaction, students reported that they were satisfied with the forums and
the instructor (1=strongly disagree to 5 strongly agree):
• Forum discussions with my peers enhanced my understanding of the material (m=4.16)
• The forums were a safe, supportive place to post (m=4.19)
• The organization of the forum was conducive to communicating with my peers (m=4.07)
• The instructor enhanced my understanding of the material (m=4.38)
• I would take another course from this instructor (m=4.25)
Updates 9/17: Corrected Campus Technology reference.
Also, here is related information from Vanderbilt based on their MOOC reporting (this snippet from their first MOOC):
Of those 23,313 active students, 20,933 of them (90%) watched at least one lecture video, 5,702 (24%) took at least one quiz, 2,072 (9%) submitted at least one assignment for peer grading, and 942 (4%) posted at least once in the discussion forums. [snip]
Across their other three MOOCs, the forum participation of active students was 9%, 22% and 6%. Some relevant commentary from Derek Bruff:
Why so much participation in the LSIO forums [the one with 22%] compared with the other two courses? David Owens, LSIO instructor, encouraged forum participation, building it into the completion criteria and seeding the forums each week with a question that permitted multiple perspectives.
Use blackboard for your mooc and you CAN control thread creation
Amy Collier, Sherif Halawa, and I have an article coming out in the EDUCAUSE Review in a couple of weeks on forum use in MOOCs when used as part of a blended course, and it’s even more dismal. (On the plus side, we did find that unaffiliated MOOC completers were making frequent visits to the forums, at least as viewers.)
Carl, care to explain more of what you mean? I’ve take a Bb-based MOOC and saw the same problems in that course, but interested in counter-examples.
Mike, I’m definitely looking forward to the ER article. As long as Teddy Diggs isn’t reading this comment thread, I won’t complain if you leak an early copy :} By the way, which platforms were the basis for study, and was it easy to gather data on the subject?
I’m not surprised by the lack of participation in MOOC discussion forums. It is completely in-line with behavior that exists in most other online communities – where 90% of participants are lurkers, 9% participate minimally and 1% of the participants account for 90% of the activity. http://www.nngroup.com/articles/participation-inequality/ If MOOC instructors want to improve these stats they’re up against a lot of inertia.
I’ve experienced the swamping of the plenary forum in a MOOC, particularly by participants talking *at* others rather than *with* them.
However I found that if I were to reply to someone’s post rather than post a new one of my own, it was likely to attract other replies (including from the OP) and a robust conversation could emerge.
I also found the participants tended to self-organise into their social media forums of choice (Facebook, Twitter, Google+ etc), which segregated the conversations and made them much easier to handle.
Phil, you make a useful distinction I hadn’t been able to put my finger on until you mentioned scaling and discussion forums.The forums are very useful for many students and, while forums are the number one complaint I hear, they are also frequently mentioned as a place where exciting things are happening and some students find useful support from one another. But the later group, as the research you’re citing indicates, is a fraction of the students enrolled over all, probably students with a certain level of engagement and comfort already. And that doesn’t scale, as you say.
Hi Phil, is there any data for dividing up MOOC students into small groups (instead of the thousands hanging out in the equivalent of an airplane hangar discussion forum) ? It seems to me, you’d have better luck with engagement, if the students were in smaller sub-groups inside a MOOC, and had to post a few intelligent posts, before the next content got released. If they stayed in the same groups throughout the duration of the course, they would then see the same familiar names post again and again, building up relationships and encouraging further connection and engagement in discussions. If you can then perhaps also tie in with the peer assignments, you’d have another way to increase familiarity within a sub group of students. I suppose certain forums would be useful centralized and scaled (like Q and A, useful crowd sourced course resources), whereas the smaller group forums should be more intimate, to allow connections and relationships to be cultivated. I think this is also one of the reasons why most students still prefer blended learning, instead of solely online.
Jan, great resource – thanks for sharing. Plus, I have to respect any research starting from someone named Will Hill.
I particularly liked this comment:
“The first step to dealing with participation inequality is to recognize that it will always be with us. It’s existed in every online community and multi-user service that has ever been studied.
Your only real choice here is in how you shape the inequality curve’s angle. Are you going to have the “usual” 90-9-1 distribution, or the more radical 99-1-0.1 distribution common in some social websites? Can you achieve a more equitable distribution of, say, 80-16-4? (That is, only 80% lurkers, with 16% contributing some and 4% contributing the most.)”
In that light, I would say that over-reliance on centralized discussion forums will tend to push the curve closer to 99-1-0.1 distribution, whereas other approaches can push closer to 80-16-4. There are a few isolated examples where grading-based forums can lead to higher participation (see update on Vandy in post), but for the majority of cases this is not true.
Again, thanks.
Ryan, that is an interesting point that MOOC participants “tended to self-organise into their social media forums of choice”. This tendency leads to the somewhat obvious question of ‘why not encourage this behavior and enhance it (i.e. make it easier to start, discover and join these self-organizing communities)’. One counter-argument is that you might lose the ability to grade forum participation, but I would argue that force-fitting forums to enable grading is self-defeating in the long run.
Robert, thanks for the original article, and I’m glad this post is contributing to the discussion.
Sicco, I think there are interesting anecdotes where small-group discussions and network building are working, but I don’t there is a lot of data, per se. Martin Mawksey does have a great visualization of DS106 communities over time:
http://ds106.us/2013/02/15/snowflake-visualization/
One challenge as it relates to MOOCs, however, is the significant turn-over – both with drop-outs and drop-ins. This makes it difficult to pre-define groups that have enough stability to allow network building over time.
The University of Phoenix is spending a lot of time & money on defining cohorts for their courses, for the reasons that you call out. But, they have not attacked the MOOC problem yet.
Great post, Phil, thanks so much.
Do you know of any research on community participation in NovoEd (VentureLab) MOOCs? This is the one platform I’ve experienced that makes it significantly easier for small groups to form and support each other. I also found the forum experience more effective than in other platforms. In terms of my own ongoing autoethnographic MOOC participation research study, where n=1, this was the only MOOC I’ve managed to complete. I think these two facts are related somehow, because it certainly wasn’t the content, the tasks/activities or the instructor that kept me hanging on — it was the ease of connection to other learners.
Heh, I like that – “autoethnographic MOOC participation research study”. I assume this was funded by the Gates Foundation?
No, I have not seen any reports yet out of NovoEd but would be very interested in finding out if there is one. It’s good to hear your positive experience. Just sent a tweet request and brazenly cc’d the first VentureLab prof. Let’s see if he takes the bait.