e-Literate

Present is Prologue

Tag: HarvardX

  • Miami, Harvard and MIT: Disability discrimination lawsuits focused on schools as content providers

    In the discussions at Google+ based on last week’s post about the Miami University of Ohio disability discrimination lawsuit ((Insert joke here about G+ and its hundreds of active users.)), George Station made two important points that deserve more visibility.

    It’s been a-coming for several years now. Cal State has some pretty strong rules in place for compliance with ADA and state-level disability laws. Still, [Universal Design for Learning] UDL is a little-known acronym on any campus you care to visit, and staff support is probably one person in an office, except for Miami of Ohio as of this week, I guess…

    Add the recent edX settlement with the US Department of Justice, and the whole direction of edtech changes…

    Put another way, it should come as no surprise that the US Department of Justice is ramping up its enforcement of disability discrimination regulations in the education world. Captioning service provider CaptionSync has an excellent summary of the field, written before the DOJ intervention at Miami. (more…)

  • The Quotable Justin Reich: MOOC research needs to reboot

    Thanks to Audrey Watters I just read a new article in Science Magazine and publicly posted here by Justin Reich, the lead researcher for HarvardX (Harvard’s implementation of edX and associated research team) ((Note that Science Magazine access requires a subscription or purchase or individual article.)). Justin calls out the limitations of current MOOC research that focuses on A/B testing and engagement instead of learning, single-course context, and post hoc analysis with proper course design. While praising the field for making available cleansed data for any type of analysis, his core argument is that we need new approaches that cannot be solved just by research teams.
    Update: Added link to publicly-available DOCX article.

    While the whole article is worth reading, there are quite a few insightful quotes should get past the journal paywall.

    • Big data sets do not, by virtue of their size, inherently possess answers to interesting questions.
    • We have terabytes of data about what students clicked and very little understanding of what changed in their heads.
    • It does not require trillions of event logs to demonstrate that effort is correlated with achievement.

    (more…)

  • Learner-Centered Analytics: Example from UW La Crosse MOOC research

    Last week I wrote a post What Harvard and MIT could learn from the University of Phoenix about analytics. As a recap, my argument was:

    Beyond data aggregated over the entire course, the Harvard and MIT edX data provides no insight into learner patterns of behavior over time. Did the discussion forum posts increase or decrease over time, did video access change over time, etc? We don’t know. There is some insight we could obtain by looking at the last transaction event and number of chapters accessed, but the insight would be limited. But learner patterns of behavior can provide real insights, and it is here where the University of Phoenix (UoP) could teach Harvard and MIT some lessons on analytics.

    Beyond the University of Phoenix, there other examples of learner-centered analytics exploring usage patterns over time. While I was at a summit at the University of Wisconsin at La Crosse last week, Bob Hoar showed me some early results of their “UW-System College Readiness Math MOOC” research that is part of the MOOC Research Initiative. I interviewed Bob Hoar and Natalie Solverson as part of e-Literate TV, where they described their research project:

    The results to date focus on capturing and visualizing the student patterns, and progress can be tracked at this project site (go to the site to see interactive graphics). (more…)

  • No, I don’t believe that Harvard or MIT are hiding edX data

    Since my Sunday post What Harvard and MIT could learn from the University of Phoenix about analytics, there have been a few comments with a common theme about Harvard and MIT perhaps withholding any learner-centered analytics data. As a recap, my argument was:

    Beyond data aggregated over the entire course, the Harvard and MIT edX data provides no insight into learner patterns of behavior over time. Did the discussion forum posts increase or decrease over time, did video access change over time, etc? We don’t know. There is some insight we could obtain by looking at the last transaction event and number of chapters accessed, but the insight would be limited. But learner patterns of behavior can provide real insights, and it is here where the University of Phoenix (UoP) could teach Harvard and MIT some lessons on analytics.

    Some of the comments that are worth addressing:

    “Non-aggregated microdata (or a “person-click” dataset, see http://blogs.edweek.org/edweek/edtechresearcher/2013/06/the_person-click_dataset.html ) are much harder (impossible?) to de-identify. So you are being unfair in comparing this public release of data with internal data analytic efforts.”

    “Agreed. The part I don’t understand is how they still don’t realize how useless this all is. Unless they are collecting better data, but just not sharing it openly, hogging it to themselves until it ‘looks good enough for marketing’ or something.”

    “The edX initiative likely has event-level data to analyze. I don’t blame them for not wanting to share that with the world for free though. That would be a very valuable dataset.”

    The common theme seems to be that there must be learner-centered data over time, but Harvard and MIT chose not to release this data either due to privacy or selfish reasons. This is a valid question to raise, but I see no evidence to back up these suppositions.

    (more…)

  • What Harvard and MIT could learn from the University of Phoenix about analytics

    Last week Harvard and MIT released de-identified data from their edX-based MOOCs. Rather than just produce a summary report, the intent of this release was to open up the data and share it publicly. While it is good to see this approach to Open Data, unfortunately the data set is of limited value, and it actually illustrates a key problem with analytics in higher ed. From MIT News description:

    A research team from Harvard University and MIT has released its third and final promised deliverable — the de-identified learning data — relating to an initial study of online learning based on each institution’s first-year courses on the edX platform.

    Specifically, the dataset contains the original learning data from the 16 HarvardX and MITx courses offered in 2012-13 that formed the basis of the first HarvardX and MITx working papers (released in January) and underpin a suite of powerful open-source interactive visualization tools (released in February).

    At first I was eager to explore the data, but I am not sure how much useful insight is possible due to how the data was collected. The data is structured with one student per row for each course they took (taking multiple courses would lead to multiple rows of data). The data columns (pulled from the Person Course Documentation file) are shown below:

    (more…)

  • Harvard Faculty Request Faculty Oversight of HarvardX (Their Usage of edX)

    Yesterday, 58 faculty members from the Faculty of Arts and Sciences at Harvard wrote an open letter to the dean requesting faculty oversight of HarvardX. When schools sign up for edX, their implementations tend to be called SchoolX, thus HarvardX specifically refers to their usage of the MOOC platform, not to the overall edX organization. This distinction is important, given Harvard’s founding role in creating the edX organization and $30m pledge of support.

    The letter is short, so I’ll quote it in full (the signatures are much longer than the letter itself).

    As the university marks the first anniversary of edX and HarvardX, some faculty are tremendously excited about the potential of HarvardX; others are deeply concerned about the program’s costs and consequences. We appreciate the meetings, town halls, and other arenas in which faculty have been able to discuss HarvardX. But we believe that many critical questions about the relationship of the FAS to HarvardX, and to edX, have not yet been addressed. These questions (which fall outside the remit of the two existing HarvardX faculty committees, most of whose members are not from FAS) range from faculty oversight of HarvardX to the impact online courses will have on the higher education system as a whole.

    The Faculty of Arts and Sciences is directly responsible for the teaching of Harvard undergraduates and Ph.D. students. It is our responsibility to ensure that HarvardX is consistent with our commitment to our students on campus, and with our academic mission. Given the rapid pace of development of HarvardX, we believe it is essential to have a formal, sustained, and structured faculty discussion on these issues as soon as possible. We write to request that you appoint a committee of FAS ladder faculty to draft a set of ethical and educational principles that will govern FAS involvement in HarvardX, to be brought before the FAS for a vote in the coming academic year.

    Note that they request FAS ladder faculty, which means tenure and tenure track faculty and specifically not adjuncts and lecturers. It is possible, however, that the requested committee of ladder faculty could choose to involve adjuncts in the process.

    (more…)