Personalized Learning Changes: Effect on instructors and coaches

Kate Bowles left an interesting comment at my previous post about an ASU episode on e-Literate TV, where I argued that there is a profound change in the instructor role. Her comment:

Phil, I’m interested to know if you found anything out about the pay rates for coaches v TAs. I’m also interested in what coaches were actually paid to do — how the parameters of their employable hours fit what they ended up doing. Academics are rarely encouraged to think of their work in terms of billable increments, because this would sink the ship. But still I’m curious. Did ASU really just hike up their staffing costs in moving to personalised learning, or was there some other cost efficiency here? If the overall increase in students paid off, how did this happen? I’m grappling with how this worked for ASU in budgetary terms, as the pedagogical gain is so clear.

This comment happened to coincide with my participation in WCET’s Leadership Summit on Adaptive Learning, where similar subjects were being discussed. For the purposes of this blog post, we’ll use the “personalized learning” language, which includes use of adaptive software as a subset. Let’s first address the ASU-specific questions.

ASU

The instructor in the ASU episode was Sue McClure who was kind enough to help answer these questions by email. Sue is a lecturer at ASU Online, which is a full-time salaried position with a teaching load of four courses per semester. Typical loads include 350 – 400 students over those four courses, and the MAT 110 personalized learning course (using Khan Academy) did not change this ratio. Sue added these observations:

During the Fall Semester of 2014 we offered our first MAT 110 courses using Khan. There was a great deal of work in the planning of the course, managing the work, working with students, hiring and managing the coaches, tracking student progress, and more. Of course, our main responsibility to help our students to be successful in our course overshadowed all of this. The work load during the first semester of our pilot was very much increased compared to previous semesters teaching MAT 110.

By the time that we reached Spring Semester of 2015 we had learned much more about methods that work best for student success, our coaches were more experienced, and our technology to track student progress and work was improved. During the second semester my work load was very much more in line with teaching MAT 110 before the pilot was begun.

For the TAs (coaches), they also had the same contracts as before the personalized learning approach, but they are paid on an hourly basis. I do not know if they ended up working more hours than expected in this course, but I did already note that there were many more coaches in the new course than is typical. Unfortunately, I cannot answer Kate’s follow-up question about TA / coach hourly pay issues in more detail, at least for now.

Putting it together, ASU is clearly investing in personalized learning – including investing in instructional resources – rather than trying to find cost efficiencies up front. Adrian Sannier in episode 1 described the “payoff” or goal for ASU.

Adrian Sannier:So, we very much view our mission as helping those students to find their way past the pastiche of holes that they might have and then to be able to realize their potential.

So, take math as an example. Math is, I think, a very easy place for most people to understand because I think almost everybody in the country has math deficits that they’re unaware of because you get a B in third-grade math. What that means is there were a couple of things you didn’t understand. Nobody tells you what those things are—you don’t have a very clear idea—but for the rest of your life, all the things that depend on those things that you missed you will have a rocky understanding of.

So, year over year you accumulate these holes. Then finally, somebody in an admissions exam or on the SAT or the ACT faces you with a comprehensive survey of your math knowledge, and you suddenly realize, “Wow, I’m under-prepared. I might even have gotten pretty good grades, but there are places where I have holes.”

We very much view our mission as trying to figure how it is that we can serve the student body. Even though our standards haven’t changed, our students certainly have because the demographics of the country have changed, the character of the country has changed, and the things we’re preparing students for have changed.

We heard several times in episode 1 that ASU wants to scale the number of students served (with same standards) without increasing faculty at the same rate, and to do this they need to help more of today’s students succeed in math. The payoff is retention, which is how the budget will work if they succeed (remember this is a new program).

WCET Adaptive Learning Summit

The WCET summit allowed for a more generalized response. In one panel moderated by Tom Cavanaugh from University of Central Florida (UCF), panelists were asked about the Return on Investment (ROI) of personalized learning1. Some annoying person in the audience2 further pressed the panel during Q&A time to more directly address the issue raised by Kate. All the panelists view personalized / adaptive learning as an investment, where the human costs in instructors / faculty / TAs / coaches actually go up, at least in early years. They do not see this as cost efficiency, at least for the foreseeable future.

(My photos from inside the conference stunk, so I’ll use a better one from dinner instead.)

David Pinkus from Western Governors University answered that the return was three words: retention, retention, retention. Tom Cavanaugh added that UCF invested in additional staff for their personalized / adaptive learning program, specifically as a method to reduce the “friction” of faculty time investment.

I should point out that e-Literate TV case studies are not exhaustive. As Michael and I described:

We did look for schools that were being thoughtful about what they were trying to do and worked with them cooperatively, so it was not the kind of journalism that was likely to result in an exposé. We went in search of the current state of the art as practiced in real classrooms, whatever that turned out to be and however well it is working.

Furthermore, the panelists at the WCET Summit tended to be from schools that were leading the pack in thoughtful personalized learning implementations. In other words, the perspective I’m sharing in this post is for generally well-run programs that consciously considered student and faculty support as the key drivers.3 When these programs have developed enough to allow independent reviews of effectiveness, student retention – both with the course and ideally within a program – should be one of the key metrics to evaluate.

Investment vs. Sustainability

There is another side to this coin, however, as pointed out by someone at the WCET Summit4. With so many personalized learning programs funded by foundations and even institutional investments above normal operations, there is a question of sustainability. It’s all well and good to demonstrate that a school is investing in new programs, including investments in faculty and TA support, but I do not think that many programs have considered the sustainability of these initiatives. If the TA quoted in the previous blog is accurate, ASU went from 2 to 11 TAs for the MAT 110 course. Essex County College invested $1.2 million in an emporium remedial math program. Even if the payoff is “retention”, will there be enough improvement in retention to justify an ongoing expenditure to support a program? Sustainability should be another key metric as groups evaluate the effectiveness of personalized learning approaches.