e-Literate

Present is Prologue

Tag: John-Campbell

  • Blackboard’s New Early Warning Analytics Product

    A couple of weeks ago in my post about the different types of learning analytics, I described retention early warning systems thusly:

    Most people don’t think about early warning systems as being in the same category as adaptive analytics, but if you consider that “adaptive” really just means “adjusting to your personal needs,” then a system like Purdue’s Course Signals is, in fact, adaptive. It sees when a student is in danger of failing or dropping out and sends increasingly urgent and specific suggestions to that student. It does that without “knowing” anything about the content that the student is learning. Rather, it’s looking at things like recency of course login (Are you showing up for class?), discussion board posts (Are you participating in class?), on-time assignment delivery (Are you turning in your work?), and grade book scores (Is your work passing?), as well as longitudinal information that might indicate whether a student is at-risk coming into the class. What Purdue has found is that such a system can teach students metacognitive awareness of their progress and productive help-seeking behavior. It won’t help them learn the content better, but it will help them develop better learning skills.

    Well, last week, Ray Henderson announced Blackboard’s new Retention Center and described it as follows:

    The Retention Center gives critical insight on learning and activity gaps to instructors, within the LMS, that helps them quickly diagnose students that are falling behind. Pre-configured and automatic so they don’t have to hunt for it. No set-up: it automatically calls out students that are at risk while instructors still have time and space to do something about it. With the feature instructors can see:

    • Who’s logging in: this is a simple but powerful predictor of student success. Instructors see how long it’s been since students have logged in to the course and how many have been away for five days or more. And not by fishing through student profiles or reports but in an automatic view complete with red flags where they’re needed.
    • Whether they’re engaged: which students have had low levels of course activity, at 20 percent or below the average in the last week.
    • Whose grades are suffering: which students are currently trending at 25 percent or more below the course average so they can target extra help to where it’s most needed – even when it isn’t asked for.
    • Who has missed deadlines: instructors might know this anecdotally or on a case-by-case basis, but now they can get a real-time view of all students that have missed one or more deadline.

    Eerily similar, no? A number of years back, when I pressed Course Signals inventor John Campbell on which factors in the LMS are most highly predictive of student success across different courses, he named exactly these four. The only surprise here is that this isn’t a common analytics feature of every LMS and courseware platform on the market yet. Purdue proved that their value in helping at-risk students is high. I’m glad Blackboard is stepping up.

    The one piece that’s missing is a simple standard where an SIS or other longitudinal data system could pass an at-risk “credit score” to the early warning system to modify its sensitivity. If a student on the honor roll drops off the radar for a week, it’s less of a cause for concern that a student on academic probation (for example). I tried to push this idea for a standard at the IMS a few years back but got nowhere with it at the time. I hope that Blackboard will push for something like it now that they have a system to take the data.

  • Oracle's New Academic Enterprise White Paper

    The product group I’m in at Oracle (Academic Enterprise Solutions, or AES) has a new white paper out on the company’s vision of the future of the academic enterprise. A lot of this is aspirational, but it does give you a sense of the general direction that the company would like to take in terms of product development. Also, being Oracle’s vision, it focuses on Oracle’s view of academic IT and how Oracle products fit in. If you don’t like enterprise-y approaches or you don’t like Oracle products, then this document probably won’t be of much use to you.

    I’d like to acknowledge the influence of a number of outside folks on the thinking behind this paper:

    • My former colleagues at the SUNY Learning Network, especially Patrick Masson and Bernie Durfee, were co-creators of the Learning Management Operating System (LMOS) concept that the white paper specifically invokes.
    • Much of Oracle’s specific vision of the future of learning environments is heavily derivative of the excellent work begin done in the Sakai community around Sakai 3. 
    • Oracle’s thinking about academic analytics has been enlightened by the work of John Campbell’s team at Purdue University.

    I apologize to any others I may be missing; there are a lot of smart people working on these problems, and the AES team has tried to listen closely to as many of them as possible.

  • Why the Retention Early Warning Critics Are Wrong

    One criticism I consistently hear when talking about retention early warning systems is that they may provide value for the university but mostly don’t for the student. The university benefits by retaining the student because it gets more tuition. But, the argument goes, the student may have all kinds of valid reasons for dropping out of a course or a program. Furthermore, retention and learning have no necessary relationship, they argue. You can stay in school and still not get anything of value out of it. The (usually implicit) conclusion from these arguments is that retention systems are nothing more Big Brother tools for squeezing more money out of hapless students.

    There’s something a little odd about this argument even on the face of it. No early warning system is forcing students to stay in school. The students must somehow be complicit in any impact on retention. Even so, I have never been one to dismiss it out-of-hand. However, after spending a half day in an EDUCAUSE seminar by John Campbell and Kim Arnold about Purdue’s early warning system, I can say with great confidence that the critics are missing the boat—at least with respect to Purdue’s approach, and probably in general.

    (more…)

  • Must-Read: Campbell and Oblinger on Academic Analytics

    John Campbell and Diana Oblinger have co-authored an EDUCAUSE paper on academic analytics that anyone with a practical interest in the topic should read. To begin with, it is a model of how to write a paper that addresses multiple institutional stakeholders across very different domains of expertise. It starts with a clear overview of the goals, breaks down the technical and logistical challenges into terms that non-experts can easily understand, lists out the likely questions, benefits, and risks for each stakeholder group, and presents high-level steps to prepare organizations that are looking to take on such an academic analytics project. And on the substance, it presents a balanced and comprehensive picture of the pros and cons of applying data mining techniques to student information in the service of improving educational outcomes.

    I particularly appricate the practical goals that the the authors set: increasing retention and graduation rates. I understand these may seem like pedestrian measures that don’t tell us how much students have learned and don’t necessarily improve the quality of the teaching either. But Campbell and Oblinger make a persuasive case that achieving these goals correlates with better outcomes for students, universities and society as a whole. Students who graduate tend to get better pay and better benefits and are more engaged in civic activities than their peers who don’t graduate, particularly in certain minority communities. Universities with higher percentages of students who graduate save more money per-student on recruiting, leaving more money to invest in improving the quality of education. And the society as a whole benefits from having more people who are not dependent on public support programs, who pay more in taxes, and who are more engaged in politics and other community-focused activities. On top of all this, we actually know how to measure retention and graduation rates. I’m concerned that many of the frantic efforts we on college campuses now to quantify student learning will turn out to be wastes of time and money because we don’t really know how to quantify learning in meaningful ways. Worse, if we delude ourselves about how much we’re able to measure, we may end up distorting the system of educational incentives in ways that actually harm students. This is exactly what has happened in K-12 in the United States with No Child Left Behind.

    The Campbell/Oblinger approach may not be sexy, but it’s practically and ethically sound while still managing to be ambitious.  This paper deserves your attention.