Following the IHE piece on Essex County College’s struggles to get good outcomes from their personalized learning program in developmental math, and following my blog post on the topic, Phil and I had an interesting exchange about the topic in email with ECC’s Vice President for Planning, Research, and Assessment Doug Walercz. With his permission, I’d like to share some of his observations with you. One of the big takeaways from the conversation, for me, is that our cultural notion of the pedagogical work that happens in a good lecture is pretty impoverished relative to the reality. We don’t have a clear understanding of all the things that a good lecture accomplishes, and therefore we often lose valuable elements of student support when we try to replace it. This has pretty serious implications for MOOCs, flipped classrooms, personalized learning, and a wide array of pedagogical approaches that replace a traditional in-person lecture with something else.
Tag: Essex County College
-
Personalized Learning is Hard
Paul Fain has written a really good, nuanced article at IHE covering the update that Essex County College gave of their developmental math adaptive learning pilot at a recent conference in Washington, DC. We did a two–part case study on ECC in our e-Literate TV series). The headline results are as follows:
- In the first year, the pass rate was worse than in the traditional classes. (The first semester was “disastrous.”)
- This year—the second year—the pass rate is coming closer to the traditional class but is still underperforming.
- The article seems to imply that students who earn a C in the personalized learning class do better than students who earn a C in the traditional class, but the article is not explicit about that.
There is no magic pill. As Phil and I have been saying all along—most recently in my last post, which mentioned ECC’s use of adaptive learning—the software is, at best, an enabler. It’s the work that the students and teachers do around the software that makes the difference. Or not. In ECC’s case, they are trying to implement a pretty radical change in pedagogy with an at-risk population. It’s worth digging into the details.
-
The Fraught Interaction Design of Personalized Learning Products
David Wiley has a really interesting post up about Lumen Learning’s new personalized learning platform. Here’s an excerpt:
A typical high-level approach to personalization might include:
- building up an internal model of what a student knows and can do,
- algorithmically interrogating that model, and
- providing the learner with a unique set of learning experiences based on the system’s analysis of the student model
Our thinking about personalization started here. But as we spoke to faculty and students, and pondered what we heard from them and what we have read in the literature, we began to see several problems with this approach. One in particular stood out:
There is no active role for the learner in this “personalized” experience. These systems reduce all the richness and complexity of deciding what a learner should be doing to – sometimes literally – a “Next” button. As these systems painstakingly work to learn how each student learns, the individual students lose out on the opportunity to learn this for themselves. Continued use of a system like this seems likely to create dependency in learners, as they stop stretching their metacognitive muscles and defer all decisions about what, when, and how long to study to The Machine.
Instructure’s Jared Stein really likes Lumen’s approach, writing,
So much work in predictive analytics and adaptive learning seeks to relieve people from the time-consuming work of individual diagnosis and remediation — that’s a two-edged sword: Using technology to increase efficiency can too easily sacrifice humanness — if you’re not deliberate in the design and usage of the technology. This topic came up quickly amongst the #DigPedNetwork group when Jim Groom and I chatted about closed/open learning environments earlier this month, suggesting that we haven’t fully explored this dilemma as educators or educational technologist.
I would add that I have seen very little evidence that either instructors or students place a high value on the adaptivity of these products. Phil and I have talked to a wide range of folks using these products, both in our work on the e-Literate TV case studies and in our general work as analysts. There is a lot of interest in the kind of meta-cognitive dashboarding that David is describing. There is little interest in, and in some cases active hostility toward, adaptivity. For example, Essex County College is using McGraw Hill’s ALEKS, which has one of the more sophisticated adaptive learning approaches on the market. But when we talked to faculty and staff there, the aspects of the program that they highlighted as most useful were a lot more mundane, e.g.,
It’s important for students to spend the time, right? I mean learning takes time, and it’s hard work. Asking students to keep time diaries is a very difficult ask, but when they’re working in an online platform, the platform keeps track of their time. So, on the first class day of the week, that’s goal-setting day. How many hours are you going to spend working on your math? How many topics are you planning to master? How many classes are you not going to be absent from?
I mean these are pretty simple goals, and then we give them a couple goals that they can just write whatever they feel like. And I’ve had students write, “I want to come to class with more energy,” and other such goals. And then, because we’ve got technology as our content delivery system, at the end of the week I can tell them, in a very efficient fashion that doesn’t take up a lot of my time, “You met your time goal, you met your topic goal,” or, “You approached it,” or, “You didn’t.”
So one of the most valuable functions of this system in this context is to reflect back to the students what they have done in terms that make sense to them and are relevant to the students’ self-selected learning goals. The measures are fairly crude—time on task, number of topics covered, and so on—and there is no adaptivity necessary at all.
But I also think that David’s post hints at some of the complexity of the design challenges with these products.
-
How Student and Faculty Interviews Were Chosen For e-Literate TV Series
As part of our e-Literate TV set of case studies on personalized learning, Michael and I were fully aware that Arizona State University (ASU) was likely to generate the most controversy due to ASU’s aggressive changes to the concept of a modern research university. As we described in this introductory blog post:
Which is one reason why we’re pretty excited about the release of the first two case studies in our new e-Literate TV series on the trend of so-called “personalized learning.” We see the series as primarily an exercise in journalism. We tried not to hold onto any hypothesis too tightly going in, and we committed to reporting on whatever we found, good or bad. We did look for schools that were being thoughtful about what they were trying to do and worked with them cooperatively, so it was not the kind of journalism that was likely to result in an exposé. We went in search of the current state of the art as practiced in real classrooms, whatever that turned out to be and however well it is working.
As part of the back-and-forth discussions with the ASU case study release, John Warner brought up a good point in response to my description that our goal was “Basically to expose, let you form own opinions”. (more…)
-
Worth Considering: Faculty perspective on student-centered pacing
Over the weekend I wrote a post based on the comment thread at Friday’s Chronicle article on e-Literate TV.
One key theme coming through from comments at the Chronicle is what I perceive as an unhealthy cynicism that prevents many people from listening to students and faculty on the front lines (the ones taking redesigned courses) on their own merits.
Sunday’s post highlighted two segments of students describing their experiences with re-designed courses, but we also need to hear directly from faculty. Too often the public discussion of technology-enabled initiatives focus on the technology itself, often assuming that the faculty involved are bystanders or technophiles. But what about the perspectives of faculty members – you know, those who are in the classrooms working with real students – on what challenges they face and what changes are needed from an educational perspective? There is no single perspective from faculty, but we could learn a great deal through their unique, hands on experiences. (more…)
-
Worth Considering: Students can have their own perspectives on edtech initiatives
Triggered by Friday’s article on e-Literate TV, there have been some very interesting conversations both in the Chronicle comment thread and on the e-Literate TV site. The most, um, intense conversations have centered on the application of self-regulated learning (SRL) in combination with adaptive software (ALEKS) to redesign a remedial math course at Essex County College. Michael has been wading in very deep waters in the comment threads, trying emphasize variations of the following point.
But that debate should be in the context of what’s actually happening in real classrooms with real students, what the educational results are, and what the teachers and students involved think of their experiences.
Right now, the “sides” are having a fight–it’s not really a debate because the sides aren’t really talking to each other–in near total absence of any rational, educator-evaluated, evidence-based conversation about what these approaches are good for. One side says they will “fix” a “broken” education system, while the other side says they will “destroy” the education system. Well, what are the students saying?
-
The ETV Personalized Learning Series: What We Hope It Contributes
It seems like there has been an avalanche of high-profile books about the future of education lately—Kevin Carey’s The End of College, Jeff Selingo’s College Unbound, Anya Kamenetz’s The Test, Michael Crow’s Designing the New American University, and Fareed Zacharia’s In Defense of a Liberal Education, to name a few. The fact that so many of these books are being written now by high-profile authors and are getting so much attention indicates that there is a sense in the public consciousness that we may be at some sort of inflection point. But it’s possible to read a bunch of these books—whatever their virtues may be—and still have only a cloudy notion of what is actually happening on the ground in classrooms right now, for several reasons. First, many of these books are written by non-educators. Second, it’s hard to paint a complete picture while also marshalling examples to support a thesis about tectonic cultural change. But maybe most importantly, it’s just hard to convey a visceral sense of what’s going on in the day-to-day educational lives of teachers and students with the written word.
Which is one reason why we’re pretty excited about the release of the first two case studies in our new e-Literate TV series on the trend of so-called “personalized learning.” We see the series as primarily an exercise in journalism. We tried not to hold onto any hypothesis too tightly going in, and we committed to reporting on whatever we found, good or bad. We did look for schools that were being thoughtful about what they were trying to do and worked with them cooperatively, so it was not the kind of journalism that was likely to result in an exposé. We went in search of the current state of the art as practiced in real classrooms, whatever that turned out to be and however well it is working.
These first case studies set the tone for the series. One is on Middlebury College in Middlebury, Vermont, and the other is on Essex County College in Newark, New Jersey. The wide differences between these two schools is indicative of the kind of breadth that we have aspired to achieve for the whole series. Each case study is 30 minutes of video, broken into 10- to 15-minute segments, very little of which is us talking. (We will be releasing some analysis episodes that bookend the series, but those were filmed last and will come out after all the case studies.) And as we had hoped, each case study yielded some lessons and surprises for us.