e-Literate

Present is Prologue

Tag: SJSU Plus

  • SJSU research report confirms MOOCs are online courses

    By reading the SJSU research report (download actual report here), one item that really hits me is that however different the scaling model is for MOOCs, they are still online courses and have similar success factors. I am not trying to minimize the value of the report by the title of this post, because there is real value in letting objective data lead you to conclusions, even if hunches and assumptions would have led to similar results.

    First, however, it would be useful to understand the data and key terms – AOLE is Augmented Online Learning Environment, which represents the additional support structures and for-credit status for MOOCs.

    Out of a total of 274 students enrolled in AOLE courses, 249 students remained in the sample after data cleaning, which included removal of students enrolled in multiple courses and of those with no course activity. In addition, 36 students were removed who withdrew from the course or received a final grade of Incomplete. This left 213 students for the deeper analysis. This data set will be referenced throughout this report as a “research data file”.

    So this data does not include no-shows, withdraws or incompletes, making it more akin to analyzing a traditional college course after the add / drop period.

    (more…)

  • SJSU releases NSF-funded research report on Udacity pilot

    Back in late July we found out that San Jose State University was pausing their SJSU Plus pilot program using Udacity for-credit MOOCs due to low passing rates. While there was fairly extensive media coverage of the story broken by Inside Higher Ed, there was the promise of a National Science Foundation (NSF) funded research report based on the spring 2013 courses. I described why the research report was important, despite indications that the lessons learned aligned with well-known practices for online education.

    With this many known issues, which have also been documented in several excellent blog posts, why is it valuable for SJSU to use the external NSF-funded report?

    In my opinion, there will still be value in A) releasing the full data set and analysis, and B) getting others to understand key points.

    Many specialists might understand why the program failed, but many others do not. Campus leaders and policy makers need to learn many of the lessons that are already known in ed tech / higher ed community, and SJSU Plus will help in this area. Plus, I wouldn’t discount some unexpected findings (e.g. need for self-pacing in math courses as mentioned in student surveys). [snip]

    It is important for some of the new participants (and I would add foundations, state governments, university presidents, etc) to fully understand what worked and what didn’t work in this application of online education. The official report from SJSU will provide a valuable service to help with this learning process.

    There is also value in this very public case establishing a precedent of pausing pilots that don’t work, evaluating results early in the process, adjusting the course and support design based on findings, and focusing on student learning outcomes as the primary measure of success.

    After several delays, SJSU has now released the full NSF-funded research report on Elaine Collins’ official web page (she was the principal investigator). The full report can be downloaded here.

    (more…)

  • SJSU Plus Udacity Pilots: Lack of transparency in describing data

    Alternate Headline: “Our Long National Nightmare is Over – SJSU and Udacity solve problem of college graduates being able to pass remedial math”

    The more I read on SJSU’s announcement on the pilot program, the more troubled I am with the lack of clear description of student population change (I wrote briefly about the change in student populations yesterday). In a nutshell, the spring 2013 pilot was completely different in the major demographic variables than the summer 2013 pilot. That’s good, right, showing that SJSU and Udacity are learning their lessons? It would be good if SJSU clearly described the student differences and avoided any implications that the numbers could be compared. Further, it would be good to avoid misleading comparisons to face-to-face courses at SJSU.

    But that is not what is going on. SJSU, in particular, is going out of its way to compare spring, summer pilots alongside SJSU on-campus courses in its media blitz. And the strategy is working, based on the articles that came directly from SJSU / Udacity interviews and information releases.

    (more…)

  • SJSU Plus / Udacity Update: Different student populations

    San Jose State University (SJSU) and Udacity have announced the results of their summer pilot, and the headlines cover the big improvements (text from IHE article, table from Udacity blog).

    Thrun recently hinted that the summer pilot’s results would be more positive, and that Udacity was getting close to finding the “magic formula” to deliver high-quality, low-cost education.

    The lone holdout among the SJSU Plus courses is entry-level math, which saw the smallest increase in students who received a passing grade, from 23.8 to 29.8 percent. That places the pass rate almost 40 percentage points below the closest SJSU Plus course, and about 15 percentage points below the pass rate of the on-campus course.
    Thrun Blog

    I am not trying to throw cold water here, but it is very important to look at the student populations. To his credit, Sebastian Thrun describes the big differences in his blog post.

    This summer, we ran the second instance of our pilot. While in the Spring, we actively sought out underserved high schools from low-income areas in California, this time we simply opened up enrollment to anyone. As predicted, with 2,091 students who enrolled, we mainly reached students who would not ordinarily attend college. Only 11% of the summer students who took the for-credit courses from SJSU were matriculated students in one of the California State Universities. 71% of our students came from out of state or foreign countries. And while the total number of high school students went up, their proportion in the total student body went down. [snip]

    One key difference between Spring and Summer was that we opened our Summer session to everyone. This led to a substantial difference in student body. Among the student body, 53% reported that they already hold a post-secondary degree (5% Associate, 28% Bachelor’s, 16% Master’s, and 4% Doctorate). Only 12% of the students had a high school graduate diploma or equivalent, and 15% were active high school students. This is very different from the Spring Pilot, in which approximately 50% of the student body were active high school students, and the other 50% were matriculated SJSU students. [emphasis added]

    One of the key success factors of online education is to target student populations who can succeed in this environment. If you target groups (remedial students, underserved students, high school students), then there must be tremendous student support.

    Pay attention to the student populations – this might be the real story. If you ignore the difference in student populations, you might draw the wrong conclusions.