A couple of weeks ago I was interviewed for an article in the NY Times about recent pushback against MOOCs.
Jonathan Rees, a history professor at Colorado State University at Pueblo, who has written critically about MOOCs, said their spread is likely to lead to a three-tiered world, with a few high-status “super professors” for whom the courses provide both status and royalties; a larger pool of tenured professors who continue to teach their regular in-person classes until they retire; and “a huge army of adjuncts and teaching assistants,” whose jobs will be vulnerable to online competition.
“The problem with this MOOC-as-labor-issue argument is that it has no place for students and learning,” said Phil Hill, an education technology consultant. “Our starting point ought to be what students need and whether this is an effective form of learning.”
Perhaps to the dismay of Jonathan’s Aunt Nancy, my comments were positioned as refuting his comments. In the context of the NY Times article, that positioning was accurate; however, in Jonathan’s follow-up blog post, he misinterprets my meaning.
MOOCification is about power too. If people with access to power want to convince recalcitrant professors to accept MOOCs, arguing that they must be because this must be is a terrible idea. Yet this is precisely what the two quotes following mine in that New York Times article from last week try to do. [snip]
Whenever Phil makes that infuriating argument, I usually respond with some variation of “profs gotta eat.” Not surprisingly that argument works really well on other profs, but not so well on Phil. However, I want to point out to Phil’s clients that they can’t just wish the class divide away. Since profs gotta eat, they’re going to worry about eating whether you tell them they’re allowed to or not. Moreover, plenty of us believe that by looking out for our interests we are looking out for students and learning. Therefore, making this kind of argument simply serves as a way to shut down all discussion of the subject, which will only breed resentment. [emphasis added]
While I enjoy reading Jonathan’s blog and appreciate his real concern with student learning and the profession of teaching, he is incorrect in his assertion that I want to “convince recalcitrant professors to accept MOOCs” or that I don’t believe that “by looking out for [professors] interests we are looking out for students and learning”. I am not making “this kind of argument”.
Focus on Students and Learning First
What I meant to say is, ironically, what I said: “Our starting point ought to be what students need and whether [MOOC] is an effective form of learning”. The lens of MOOC-as-labor-issue tends to create an us-versus-them, you’re either pro-MOOC or anti-MOOC, mentality that leads to, well, nowhere except arguments. The lens of MOOC-as-teaching-method to be explored should lead us to critically look at learning outcomes and exploration of how to apply or refine the teaching method.
Michael made this same point in a more elegant fashion in his commentary on the SJSU Philosophy Department rejection of edX:
Rather than thinking of MOOCs as products to be bought or rejected, they could have approached them as experiments in teaching methods that can be validated, refuted, or refined through the collective efforts of a scholarly community.
SJSU Plus and Udacity
And this is where the story comes to San Jose State University and the recent announcement that they were “pausing” the Udacity MOOC-for-credit pilot program (SJSU Plus), as described at Inside Higher Education:
After six months of high-profile experimentation, San Jose State University plans to “pause” its work with Udacity, a company that promises to deliver low-cost, high-quality online education to the masses. [snip]
Preliminary findings from the spring semester suggest students in the online Udacity courses, which were developed jointly with San Jose State faculty, do not fare as well as students who attended normal classes — though Junn cautioned against reading too much into the comparison, given the significant differences in the student populations.
Preston Rudy, vice president of the San Jose State chapter of the California Faculty Association, said the university must be cautious with what he called its experiments on students.
“It’s wise to reevaluate and pursue something based on the evidence rather than the advertisement,” Rudy said.
The Chronicle has added more details on the actual results:
News of the break coincided with the leaking of a slide show containing preliminary data on the spring trials, which included three mathematics courses that San Jose State instructors built with Udacity. The courses were offered to a mix of students, some who were enrolled at the university and others who were not, including some high-school students.
The pass rates for the San Jose State students in those courses ranged from 29 percent to 51 percent. For nonenrolled students, the range was 12 percent to 45 percent.[emphasis added]
It turns out that the low end of the scores occurred in the developmental math course suggested by the Gates Foundation. From IHE:
The spring courses – a remedial math course, a college algebra course and an introductory statistics course – were chosen in part because of the wishes of Bill Gates, whose foundation gave the effort a grant, Junn said. Gates, the founder of Microsoft, is a fan of math and science education and wanted the offerings to include remedial math, Junn said.
And the Chronicle:
The [remedial math] instructors played to two audiences: 3,500 learners who had signed up for the course as a MOOC through Udacity, and just under 100 others who were taking the course for credit, about half of whom were enrolled at San Jose State. The two groups watched the same video demonstrations and completed the same routine assignments. The only difference was that the students taking the course for credit also were required to complete three online examinations, with monitoring by ProctorU, the online proctoring company.
Only 29 percent of the San Jose State students in the credit-bearing section passed the course, along with 12 percent of the non-matriculated students, according to Ms. Junn’s slide show.[emphasis added]
Proper Student Support
This situation should surprise no one who follows higher education and online education – remedial students need additional support, even more so if they are in an online course. The fact that the MOOCs were rushed made the situation even worse, as there was not time to design a support system for these students, per IHE:
But, because of the haste, faculty were building the courses on the fly. Not only was this a “recipe for insanity,” Junn said, but faculty did not have a lot of time to watch how students were doing in the courses because the faculty were busy trying to finish them. It took about 400 hours to build a course, though the courses are designed to be reused.
As long as I’m being self-referential in this post, I should note that the point about student support corresponds to one Michael and I made in our position paper for The 20 Million Minds Foundation regarding California online education:
It is important to remember the real goal of using online education to address bottleneck courses here. It is not to offer students seats in courses. It is to get students to complete those courses successfully so that they can complete their programs more quickly. While California cannot guarantee student success, the state can put in place provisions that guarantee students access to the kinds of support that are known to increase the likelihood of student success. This includes taking care to preserve existing campus support networks when bringing in new solutions—particularly solutions implemented by third parties—as well as taking care to provide students with extra support when it is needed. These considerations are important for locally developed solutions, but they are especially important for safety valve solutions where some of the traditional campus support and quality control mechanisms may be circumvented to achieve greater accessibility.
Online education classes typically require more self-discipline, better reading skills, and better awareness of when to seek help than traditional classes do. Offering an online class to a student who otherwise would be shut out altogether is often better than nothing. But we need to recognize that we are already starting with a solution that has its challenges for achieving a goal of high completion rates, even if everything else is equal. Not all students are equally well-prepared for online learning, and pushing students who are likely to fail into an online course may, in fact, be worse than the status quo. Online courses are not a panacea. Students will need help in evaluating whether online is appropriate for them. And if it is not, those students should be given priority access to the traditional on-campus or blended courses.
To me, this is the lens to use when evaluating new teaching methods, including online education in general and MOOCs in particular: evaluating what students need and evaluating whether the teaching method in question effective or not. Clearly for the SJSU Plus program with Udacity as currently designed, the answers are a lot more support and no.
As Currently Designed
But the story does not end in July 2013. To their credit, SJSU is launching an independent research effort to evaluate the results-to-date of the Udacity experiment, with the goal of learning from best practices and improving student outcomes. In a surprisingly good editorial from the LA Times (with the one exception of conflating MOOC and online courses):
The disappointing results from San Jose State’s experiment with online courses shouldn’t be interpreted to mean that such courses can’t help students. But the classes the university offered in collaboration with online provider Udacity were practically a model of how to do online education badly: rushed into existence and sloppily overseen. No one was even aware that some students who had signed up for the classes lacked reliable access to computers. The one thing the college did well was monitor the results of the three pilot courses and call a timeout when failure rates proved unacceptably high. [snip]
Even pilot programs must be carried out with more care. Online courses should be developed thoughtfully, from within the colleges, not as a result of top-down directives from the governor. The subjects that are offered should be based on student demand and faculty analysis of which would work best online. The preferences of even the best-intentioned billionaires should not be part of the equation. Nor should online courses be viewed as major money-savers, as Brown has pitched them. It still takes well-educated people, interacting with those who need an education, to provide high-quality courses, whether that’s via the Internet or in a classroom.
I would add that it is quite impressive that SJSU (as described in the official blog post) is “currently awaiting a more comprehensive National Science Foundation data analysis and report that will be available in August” and that they “welcome vigorous public discussion of our pilot efforts and assessments of their effectiveness”.
How often do we get the chance to review the results of traditional college courses and see institutions publicly study the learning outcomes in order to improve the course effectiveness? This open review is one innovation from the SJSU Plus program that should be extended to other courses.
Conclusion
And this gets back to my original point in the NY Times article. I don’t believe in pushing MOOCs on recalcitrant faculty, but I do believe that student needs and learning outcomes are the primary measures of whether teaching models work or not.
Phil,
OK, I’ll take back the “shut down all discussion” part, but the resentment issue remains. Suppose assessment results show SJSU students learn like gangbusters, Faculty will pick apart what those assessments actually assessed because a) They want to keep their jobs and b) They probably have a more complex definition of learning than Udacity does. These two points are inseperable. It makes no sense for either side of this debate to deny both.
That last word should be “either,” not both. To expect anyone to gently acept their own obsolesence is poor personnel management.
I agree with the comment you made in the NYT article, the focus should be on students, and consideration of learning effectiveness, not on the MOOC itself. And, though Michael has the right idea as well, that MOOCs should not be looked as consumable products to be bought and sold, but more as “experiments in teaching methods that can be validated, refuted, or refined through the collective efforts of a scholarly community”, I do not see this as applicable to the Udacity and San Jose State project. Far from it. This project was a very costly experiment, not only in terms of [taxpayer] resources, but in the context of student learning, which was comprised [pass rates were dismal, between 29% and 51% of students passed]. It was a rash, and irresponsible undertaking by the CA state public university system, and it was doomed from the start.
California state leaders were looking a quick fix, a solution to the costs and access challenges that CA public institutions are facing, evidenced by Governor Jerry Brown call to Sebastian Thrun back last July, asking for help. There appeared to be little analysis or exploration of solutions to the complex problem[s], and the though the problems were identified somewhat in the SB 520, there appeared to be little consideration given to alternatives. In fact the bill appeared to be written for the MOOC model.
Yet, on what premise was this decision made to invest significant resources in partnership agreement with Udacity [apparently the contract negotiations between SJSU and Udacity was an intensive process]. There appeared to be more time spent on hashing out the details for the contract between the SJSU and Udacity than on developing the courses.
Furthermore, there is no data to validate that the MOOC model produced successful learning outcomes for entry-level college students. In fact preliminary data on MOOC completion rates suggests it’s educated learners with at least one degree that participate in MOOCs, are in their mid thirties, are more likely to live outside of the United States (http://www.insidehighered.com/news/2012/06/05/early-demographic-data-hints-what-type-student-takes-mooc). The demographics do not fit the profile of a high school student requiring remedial work, or for entry-level college courses. Furthermore, given that it is a well known fact that 2/3 of US students are unprepared for college level course work, it is completely unrealistic to think that first year college students will be successful in a MOOC environment, even more so for students enrolling in the remedial math course.
There are several examples of programs implemented at Universities across the United States that addressed similar challenges that CA state universities face. Blended learning programs, for example have produced remarkable results, not only in terms of costs savings but also in student success rates, and student and faculty satisfaction with the learning process and outcomes. I’ve written about some of these programs before on e-literate in comments, so I won’t repeat myself.
Thank you Phil for this post; I always respect your opinions and insights :). I do respectfully disagree with you however, that it is “impressive that SJSU is “currently awaiting a more comprehensive National Science Foundation data analysis and report that will be available in August”. I see this as a waste of more money and time. It’s most obvious why the program didn’t work. Time to move on and focus on developing programs that work for students, involving faculty in the process, and setting-up formative and summative assessments to evaluate learning effectiveness that can be analyzed to improve and refine learning outcomes.
Now that both Phil and Jonathan are in the same space, it may be possible to develop a larger exchange. The one piece missing is a representative such as Stephen Downes for the alternative philosophy which does take students into consideration, the c, or connectivist, MOOC.
There are a number of tracks, or themes, with two being the idea of “experiment” and “obsolescence of the professor”. Both of these really hang on nuances and the issue of “risk”.
As a former tenured professor, administrator and international consultant on HEI futures and watching what comes across my “desk” as editor of an education foresight journal, On the Horizon, http://www.emeraldinsight.com/oth.htm, it is interesting that this argument on education delivery parallels shifts in or emergence of paradigm shifts in the content as well.
The most interesting of these is in economics with the current controversy about the rise of heterodox economic theory vs conventional neo-classical economics, which, in part, started with a “revolt” by French students (post autistic economics). The second is an insightful article in the July 13, 2013 NYT’s by Nicholas Christakis, “Let’s Shake Up the Social Sciences. Christakis points out that in the science/tech areas, many departments have disappeared to be replaced by innovative alternatives: anatomy and histology of the old and stem-cell biology and molecular neurobiology representing the new.
Here it is interesting that one of the most over-regulated education systems in the US is the P-12 school systems in both content and process, yet it has at many points been willing to risk the introduction and possible failure of a variety of methods and content (not without controversy) including technology. That system is littered with failures and still there are points of light with ranges from charter schools teaching classics to self-directed models.
There was, during the 60’s and 70’s a glimmer of this in the HEI’s with even governments creating entire campuses with new technology and interdisciplinary content. Others were more cautious with similar experiments cocooned in their larger curriculum. Some have survived and even thrived. Therefore:
a) The idea that MOOC’s, e-learning, and even new brick-space alternatives as “experiments” is problematic; and particularly in the HEI’s it is misdirection or worse. It is problematic because it misdirects and does not deal with the complicated social milieu including the role of faculty and the shifting student population.
b) Smart individuals with Ph.D.’s may be excellent researchers, but with only experience as a student, may not be the most appropriate as educators and/or administrators. There is a big difference between OJT or “self-learning” vs “self-directed” learning as the above article points out for students taking MOOC’s or even conventional classes. Industry can spend in HR between 5-10% of their income in providing support and education to their employees from housekeeping to the glassed-in corner office at the top. It is not just the students that need support.
It is interesting that the faculty resist as do admins but for different reasons. It is also interesting that most faculty know of changes in industry which has lead to the need for job “retraining”, particularly in the trades yet have not yet dealt with this in their own “industry”. This is more than shifting roles as educators and researchers but the radically shifted roles of faculty in the governance of the institution in which they are employed. (Think Luddite, flight engineers and railroad brakemen)
c) MOOC’s are just a surrogate for the larger issues facing education, now global. The self-contained Ivory Tower, a point of light in a dark sea, is no longer a safe haven any more than the castle in Poe’s Mask of the Red Death. Knowledge flows across geo-political boundaries as does money, goods and services.
We may be at an educational “tipping point” as so eloquently pointed out in an eponymous SantaFe Institute working paper and MOOC’s may be at a similar point in the Gartner defined Hype Cycle. It is hard to see MOOC’s as disruptive in that they are just a new manifestation of several older efforts. It is time to place them within the context of a larger manifestation.
To create a different perspective, one can look at the future of the academic journal. A narrow cast history pointing to its future has been addressed by Bill Cope. Both the Royal Society which started the first, Philosophical Transactions, and On the Horizon plan to address this issue. Cope touches on the problems which stem in part from the use of the journal and its editorial board as a way for institutions to abrogate their responsibility of evaluation of faculty research for promotion and tenure which has significant consequences on what gets published, where, and its ultimate value.
More importantly, Cope points out that the journal, in a large part created what now gets presented as text.
In part it controls what content is acceptable and thus what appears in the “class-room”, whose theories are acceptable and the merit. Thus it is not just how knowledge is delivered but what is presented in the classroom. This, as mentioned in the above post, is what started the revolt of the French economic students, because of the fact that departments hired faculty of certain philosophies and dogma which did not make theoretical sense. There are many “edge” ideas which may or may not get support at the research level and which then move down into the “texts”.
These issues, as noted above, may be of greater concern than the methodology of delivery and ultimately how graduates from post secondary institutions contribute to society.
The problem with HEI’s transcends the methodology. Form and function may be the lesser of many worries.
Jonathan, I’m not denying that there is faculty resentment and also agree that a) fac want to keep jobs and b) have more complex definition of learning. But I find it more useful to view this through student / learning lens. In the SJSU Plus example, the courses failed, and part of the reason was how it rushed faculty and ignored well-known faculty advice on remedial education. Nevertheless, we’ll continue our media wrestling tour :]
Debbie, you raise a great point about using university resources in an effort that has no data to back up that it might work (MOOCs as remedial). Personally, I think it is good in a distributed system that a school tries out a method even before it is safe; it would have been far better to do so in a careful, planned manner.
On the area of disagreement (value of independent study), I agree that many of the lessons are already known. However, there will still be value in A) releasing the full data set and analysis, and B) getting others to understand key points. You (and many others) might understand why the program failed, but many others do not. Campus leaders and policy makers need to learn many of the lessons that are already known in ed tech / higher ed community, and SJSU Plus will help in this area. Plus, I wouldn’t discount some unexpected findings (e.g. need for self-pacing in math courses as mentioned in student surveys). I might develop this idea further in another blog post, but I welcome the dialog.
I think the best thing to come from the MOOC debates is the increased attention on the complexities of institutionalized teaching and learning. We had an open forum for candidates for VP of Instruction the other day, and I asked how to prioritize resources and support of faculty with low technology literacy, given the competing priority of moving from 11 to 10 week term, and the increasing faculty work load. The candidate did not have an answer. I was glad to see it recognized as a very hard problem.
In order for any of our instruction to be successful, MOOC or not, we have many issues we need to resolve. Stakeholders don’t even agree on the definition of success. There’s often disagreement on expected outcomes. It’s possible most instructors don’t understand assessment. If we can’t agree on what students should learn, and how we know they’ve learned, it’s pretty difficult to determine the best ways to help them learn.
The paragraph before you conclusion is the most important point. We can’t ethically shun MOOCs without applying scrutiny to our current instructional models and evaluation practices. While efficiency should not be the primary focus of investigation, we must be aware that individual instructors have a finite amount of time to perform all the expected duties of their roles. We likely all agree they do not get nearly enough time for professional development, but I’m not sure anyone’s found a viable solution.
Phil
I agree with you that often times taking risk and experimenting with new teaching methods is necessary, even before there is solid data to support its effectiveness, though SJSU would have done themselves a big favor had they taken a measured and thoughtful approach as you said.
You are right, I had not considered this — there may be value in analyzing the report on the project from the perspective of using it as a tool to inform campus leaders and policy makers. More importantly there may be value IF the report delves into the pedagogical approaches, feedback from students and data from LMS. LMS data analyzed properly can reveal student patterns and provide insight when results are examined holistically. To be useful for educators, all of this data will need to be synthesized carefully to draw out helpful recommendations.
Thanks Phil for your post. I look forward to following the developments.
Phil, I think your comments are to the point, if we started to look at factors which have been shoved into what economists call “externalities” but which loom large like the monsters under the bed:
a) education parallels health care. Up to grade 12 the families and students do not directly pay for their education as with full coverage medical. When families have to directly pay for education services as with medical care, the fiscally conservatives in government believe that they, as we are seeing, become more concerned as to what they are getting for their “investment” and, hence, there is more public oversight.
b) Up to recent times, there was a large segment of the population which did not go beyond grade 12 or to some more applied post secondary programs. The college degree is now considered the brass ring. Yet, because of the issues of “a” many students who enter universities are in need of academic upgrading. These programs are weak and cannot correct in one semester or one year the cumulative effect of a number of years. Even foreign students find they may need a year just to master English to attend English speaking universities though their content capabilities are high. Again, the health care parallel is telling. Individuals who engage in risky practices or neglect their health may never be able, even with masterful intervention to correct cumulative punishment to their health.
c) I agree with you regarding the issue of faculty. The entire post secondary system has problems. Among these are how faculty gain tenure and promotion and their upward mobility by being able to move to higher ranked institutions. Those who are concerned about retaining their teaching jobs in their current embodiment have made career choices the same as those who commit to the pub/perish route which has them more focused on their disciplinary areas of expertise. Both of these make radical changes in delivery of content problematic for many reasons which need to be unpacked. This would be problematic in the past and is more so today when Baumol’s disease looms like a large shadow.
d) As MIT has noted, their free content for classes and their xMIT were, in part, probes to improve their delivery, even in brick space. MIT is a selective admissions institution and they have not made it a point to actively expand their market. They have not sought to build a “business” model, at least as publicly, as the other xMOOC’s nor has the c-MOOC community which attracts, selectively because of their model. There are different needs and different expectations
e) As to experimentation, another of the monsters under the bed is “competencies”. As that gets puzzled through, the burden will shift to the student and shift expectations away from the idea of “butts in seats” wither physical or virtual. There are many issues here which many of those critical of the MOOC will have to reconsider since the burden created by mapping old wine into new bottles, both content and process shifts responsibility, much as the health care system now rewards “wellness” instead of the expectation that the system must treat persons regardless of how they take responsibility.
tom
Tom, as usual I’m having to hit the library for in-depth research and vocabulary-setting before replying to your comments :}
Jen and Deb, great points. I could add to it that I hope there is analysis on the course planning / decision-making process itself and not just student patterns & pedagogy. There are some talented people on the SJSU campus who could have guided this program and avoided many problems. How did they get left out?
Agreed — we want to know where the decision making went wrong, and why talented people were left out – however the focus should be on the learning, and students. The purpose of this program was to reach students and provide a learning method using a new delivery mechanism – it did not work – we need to focus on why? What worked, what didn’t? Where was the breakdown? etc.
Focusing on the poor decision making and planning that happened hopefully will prevent further mistakes and should be part of the discussion, however the focus of the project was on student learning and improving access through alternate delivery methods – we need to make not lose site of the overall purpose of the project.
I want to take on Jonathan’s “Professor’s gotta eat” argument ( if that’s what it is).
Professors have to eat, sure enough, but the state, and the citizens of the state, have no obligation to feed them.
When Gutenberg printing was introduced in Europe, the first printed books were, not surprisingly, Bibles. The professional scribes argued that mechanically produced books would be dead and hence without value. We need scribes to breath life into the word.
Not surprisingly, that argument didn’t wash.
How much better would have been this one: “Scribes have to eat.”
The conflation of Professor’s culinary needs and student’s educational needs is pretty specious. In my almost fifty years as professor and department chair, I can hardly remember a single moment when anyone gave a damn about student learning. And why should they, when tenure and merit pay were almost entirely based on publication. Not being noted for stupidity, professors (including me) heeded these words to the wise and left their students in the dust.
We are now drowning in floods of mediocre scholarship (possibly including my own, but one can always hope or hide in denial) while students are drowning in debt. MOOCs offer one avenue out of this flood, and the discussion in this strand completely ignores this.
Leonard Waks
Prof. Emeritus of Educational Leadership, Temple University
Author: Education 2.0 (Paradigm: 2013).
Tom, I agree on the points of externalities and strongly agree that “MOOC’s are just a surrogate for the larger issues facing education, now global” and point c) about faculty upward mobility / career choices. To Jonathan’s point, it is worth noting that this is not so much about individual choices of faculty who didn’t create the system, but rather about systemic issues and cross-institutional choices. The HE system has evolved or become one that makes it difficult to adapt.
As the SJSU example shows us, we need faculty and other HE expertise to push new models forward in ways that do not forget the lessons we have learned with current system – shouldn’t throw out the baby with the bath water.
Hooray for the geezer.
Leonard,
If “Profs gotta eat” is not a viable argument, then there will be very few professors. While this might seem like a good outcome to someone who’s had a career like yours, I can assure you that the rich will continue to pay for the privilege of acessing the many dedicated instructors who can pay their bills.
The rich get an education. The poor get MOOCs and robots. Is that fair to students? Perhaps the “learning outcomes” will be the same, but the quality of the experience won’t be and nobody will go into debt up to their eyeballs to be treated like a number.