I was delighted that we are able to publish Mike Caulfield’s post on how ed tech gets personalization backwards, partly because Mike is such a unique and inventive thinker, but also because he provided such a great example of how “personalized learning” teaching techniques are different than adaptive content and other product capabilities.
The heart of his post is two stories about teachable moments he had with his daughters. In one, he helped his middle school-aged daughter understand why an Iranian author was worried that people in the Western world have harmful stereotypes of Iranians. In the other, he helped his high school aged-daughter see how her knowledge of the history of rocket science could be useful in answering a question she was asked about Churchill’s Iron Curtain speech. Mike’s stories show truly significant learning of the kind that changes students’ perspectives and, if we’re lucky, their lives. It is not just personalized but deeply personal. He was able to reach his daughters because he understood them as humans, well beyond the boundaries of a list of competencies they had or had not mastered within the disciplines they were studying.
For now and the foreseeable future, no robot tutor in the sky is going to be able to take Mike’s place in those conversations. This is the kind of personal teaching that humans are good at and robots are not. But neither are the tools we have today useless for this sort of teaching. Vendors, administrators, and faculty alike have broadly misunderstood their role and potential. In this post, I’m going to talk about both how these tools are useful for the kind of education that Mike cares about (and I care about) as well as, perhaps more importantly, why we are so prone to getting that role wrong.
Robot Tutors in the Weeds
One fact that shines through about Mike’s daughters is that they are both pretty smart. His middle schooler clearly read and understood the book by the Iranian author. She was stuck on not a “what” question but a “why” question. His high schooler knew a lot about the space race—not just the what, but also the why. She just hadn’t yet seen the relevance of what she knew to a “why” question she was being asked in a different domain. Both girls were working on higher-order thinking skills. One of the reasons that Mike could teach them is that they already had a lot of foundational knowledge.
Not all students do. They don’t all have strong reading comprehension or study study skills. Not everybody is good at remembering facts or distinguishing causal connections from co-occurring but irrelevant details. For example, some readers could benefit from being stopped every few paragraphs or pages and asked questions to help them check themselves to see if they’ve understood what they read (and become more self-aware about their reading comprehension in general). This is the sort of thing that computers are good at.
What happens in classrooms where students don’t have this sort of comprehension tutoring help (robot or otherwise)? Sometimes the students who need that help don’t get it, and they fail. They are never able to answer the “why” questions because they don’t know the “what.” Other times, the teacher slows down to cover the “what” in class in order to help the students who are struggling. This teaching strategy has a few side effects. First, it takes a lot of class time, which means that there is little or no time to discuss the “why.” This leaves kids like Mike’s daughters, who are ready and hungry for the “why,” bored. Second, all the sudents quickly learn that they don’t have to read the book because the teacher will go over the important parts in class. I hear complaints from teachers all the time that they have trouble getting “kids today” to read. I believe them. But I’m skeptical of the explanations that I hear for why this is so. I don’t think it’s primarily because of TV or YouTube or mobile phones. All of those factors fall under the larger umbrella cause that students don’t have to read anyore. Nowhere is that more true than in the classroom. If you were asked to read something in advance of a meeting, and you knew the person running the meeting would take almost all the meeting time reviewing the aspects of the reading that she thought were important for you to know, would you read in advance? Or would you find a better use for that time?
How Homework Got Broken and How Not to Fix It
Most teachers—especially middle school and above—have a passion for their subject. (Elementary school teachers, who are generalists, more often have a passion for the students, although the two are not mutually exclusive.) They love the “why” and want to talk about it. But they end up spending most of their time talking about the “what” because if they don’t they will leave some students behind. As we have seen, a side effect of this understandable behavior is that students learn not to do the homework, which means that they increasingly come into class not knowing the what. And the viscious cycle continues.
So teachers grade the homework, hoping that the grades will force the students to come to class prepared to discuss the “why.” To be clear, there are different reasons why teachers might want to count homework toward a course grade. One is when the homework is carefully constructed to incrementally build skills, so each homework assignment is essentially a summative assessment of the next small step on the hill the teacher is trying to get the students to climb. We see this most often in math, engineering, or other subjects where there is a strong emphasis on increasingly sophisticated application of procedural knowledge. But more often than not, teachers count day-to-day homework toward a course grade primarily because they are trying to motivate students to learn the “what” at home.
This approach has side effects of its own. Students are motivated by grades, but only to a point. They quickly become quite sophisticated at calculating how much of the homework they have to do in order to get the minimum grade that they want to achieve. Excellent students and weak ones alike make this calculation. Unfortunately, weak students often miscalculate, undershoot, and fail. Meanwhile, good students may be getting good grades, but they are not necessarily learning all that they could be. And, of course, the more the homework counts toward the course grade, the more incentive students have to cheat.
Part of my day job as a consultant is to help companies who design educational technology products understand teachers and students better. In the course of doing that work over the past few years, I have spoken to a lot of students and teachers about the homework problem. Many of the best teachers either don’t count homework toward the course grade or count it just a little—enough to communicate to the students that the homework matters, but not enough to trigger the what’s-the-minimum-I-have-to-do calculation. They use the grade as just one tool in an overall strategy designed to help students see that the “what” questions they are learning to answer in their homework are relevant to the far more interesting “why” questions about which the teachers are passionate and would like their students to become passionate about too. They pose mysteries at the end of class that the students can only solve with the knowledge they gain from doing the homework. Or they have little verbal in-class quizzes to keep the students on their toes, in the context of a discussion of how the tidbit in the verbal quiz matters to the larger topic being discussed.
Interestingly, a number of students have told me that they like in-class verbal quizzes. Well, “like” probably isn’t quite the right word. Appreciate. Value. Are grateful for. But they only will tell you this if you ask the right question, which is the following:
“How can you tell that your teacher cares about you?”
The story that we often tell ourselves about other people’s children is that they are lazy. They don’t like to work or to learn. But the first question that most students are trying to answer for themselves when they start a new class, particularly if that class is about something they don’t already care about, is “Does this teacher care if I learn?” If the answer is “no,” if the relationship is purely transactional, then most students will try to figure out the minimum cost they have to pay in order to get a satisfactory grade. Think back to your own school days. Didn’t you do that sometimes? I did. The less I thought the teacher respected me or cared about me, the harder I played the I’m-going-to-do-almost-nothing-and-still-ace-your-stupid-class-you-arrogant-ass game. And the more I could get away with it, the more I was convinced that the teacher didn’t care about me. “After all,” I reasoned, “nobody who really cares about me as a student would let me get away with being so damned lazy.”
This is a problem that grading homework won’t fix, robo-grading won’t fix, and adaptive robo-grading won’t fix. In fact, those strategies often make the situation worse. Two things enrage students almost more than anything:
- Making them buy expensive books that they never actually need and that the teacher never even mentions in class discussion
- Making them do hundreds of stupid homework problems that seem to have no obvious connection to anything on the tests (or, really, anything in the world) and that the teacher never even bothers to talk about in class
Luckily, these selfsame robo-homework tools actually can help avoid the cascade of course design failures I have traced in this post, if only they are designed and deployed a little differently.
Personalized Learning
Let’s review some of the things that educational software can do well:
- Check students’ mastery of low-level cognitive skills such as memory of facts and application of procedural knowledge
- Provide feedback to students and teachers on the students’ progress toward mastery
- Provide feedback to the teachers on the students’ progress
- In some (but not all) cases, help the students when they get stuck on low-level mastery skills
When I say “low-level,” that is not a value judgment. Mike’s daughter had to know basic facts about the U.S. and Soviet space programs in order to make inferences about their consequences for the broader political climate. The teacher clearly wanted to spend time with the students discussing the “why” question, and Mike’s story illustrated how humans are much better suited than robots for helping students learn to answer those sorts of questions. But we also know that teachers get stuck spending all their time on the “what” because some students get stuck there. Students don’t want to spend any more time on the “what” than their teachers do. It is in everybody’s interest to get students to learn as much of the basic facts and procedural knowledge as possible outside of class so that the teacher can spend class time on the real intellectually challenging aspects of the subject.
Software can help solve this problem by giving both students and teachers feedback on how the students are doing with the “what.” Some students, like Mike’s daughters, won’t need help, but it’s not an all-or-nothing thing. Most people are better at grasping the basics of some subjects than others, or better at some low-level cognitive tasks than others. Personally, my reading comprehension is very good, but I am horrible at memorizing. I was interested in science but ended up dropping every college science course I registered for because the memorization killed me every time. Tutoring software might have helped me. In high school, I scored very well on the physics Achievement test because I could derive most of the physics needed to answer the questions based on the “why” I had absorbed in class. But I did poorly in the class itself because I was bad at remembering and applying simple formulas.
I could have become good at physics. I could have learned to love it. When I was a kid, I used to write letters to NASA to request pictures from their telescopes. My high school teacher knew that about me, because I was in a small class, and because he was the kind of teacher who made a point of knowing that sort of thing about his students. But my college professors had no way of knowing given the contact that they had with me in their large lectures. If they could have seen my results on formative assessments, and if they had more time in class to help students like me with the sticking points rather than repeating the reading that almost nobody did because they knew the professor would repeat it, then I might have had a different relationship with the subject. I took every philosophy of science course I could but avoided the actual science classes because I was afraid of them.
Tools that can help students like me exist today. But more often than not, two common mistakes currently get in the way of them being used in ways that actually would have helped me get through physics (or biology, or art history). The first is grading. The minute the homework becomes high-stakes, it breaks the ability to help students who are stuck. Rather than reducing student anxiety about the course, it raises it. Rather than motivating students to do the best they can for the teacher and themselves, it motivates them to calculate the impact of each assignment on their grade. Students need to believe that mastering the “what” matters, but this is not the way to convince them.
This brings me to the second and related mistake, which is failing to make explicit connections between the “what” and the “why” for students. They need to understand the point of learning all that low-level stuff. I didn’t care about solving physics problems, but I did care about understanding physics. I might have been more motivated to take on the scary work that was hard for me if I had seen a clearer connection between the two. This is all about course design. It’s about using the homework tool in context. It’s about reclaiming classroom time to have discussions like the ones that Mike had with his daughters, and maybe sometimes to review the specific “what” problems that students are getting stuck on.
Putting all this together, fixing the problem of broken homework requires the three personalized learning strategies that Phil and I have been writing about:
- Moving content broadcast—especially lectures about the “what”— out of the classroom to make room for discussions about the why
- Making homework time contact time, so that students can get help from the teacher when they are stuck with the “what” and also see that the teacher cares about whether they are learning
- Providing a tutor in cases where the software can help the student with the “what,” or maybe a human tutor by enabling the teacher to see where students are stuck and focus class time on getting them unstuck
The term of art for using homework this way is”continuous formative assessment:”
You don’t need technology to do this. It’s just a feedback loop that could be accomplished by manually marking up students’ work or otherwise guiding them as they work. Technology just provides the ability to amplify that feedback loop and make it less labor-intensive to implement. But most vendors aren’t optimizing their homework products for this kind of use. Instead, they spend all their time adding gradebook features and increasingly complex ways for instructors to customize problem sets and reduce cheating. And they do this, more often than not, because their customers ask them to. (Of course, they don’t often hear from non-customers who aren’t interested in graded homework but might be interested in continuous formative assessment.)
The fundamental problem isn’t the tool or the vendor. It’s the cascade of unintended consequences caused by students who come into the class with different levels of skill and motivation, and the coping mechanisms teachers have employed to deal with the challenges of teaching a heterogenous group. Right now, we are mostly getting products that are designed to minimize the pain caused by that cascade or tools that are designed to replicate the failures in a more automated and therefore cheaper way. But we could easily be getting products that help teachers to create that positive feedback loop between themselves and their students. If we want that to happen, then we have to start asking a different question:
How can the capabilities afforded by educational technologies empower teachers to learn and implement teaching strategies that work better for them and their students?
[…] By Michael FeldsteinMore Posts (1071) I was delighted that we are able to publish Mike Caulfield’s post on how ed tech gets personalization backwards, partly because Mike is such a unique and inventive thinker, but also because he provided such a great … Continue reading → […]