The “Ed Tech” category includes posts about educational technology products themselves, including LMSs and other learning platforms, adaptive learning and other digital curricular materials products, learning analytics, and educational apps of all types. It also includes technical aspects of ed tech products, especially interoperability.
Doug Lederman at Inside Higher Ed wrote an excellent article describing the trajectory and demise of the “failed $75 million experiment” of the University of Texas System’s Institute for Transformational Learning (ITL).
Starting in 2011, the Texas system invested nearly $100 million in the institute ($23 million remains unspent) to try to drive digital technologies into the approaches its campuses use to reach, educate and graduate students. Over five years, the institute helped several UT campuses launch distinctive new academic programs and developed three core pieces of technology that, among other things, deliver online learning and students’ transcript information via the blockchain. But its total revenue over the five years: $1 million, The Texas Tribune reported.
Critics have derided that output as paltry given the investment and questioned whether the money could have been better spent. Advocates for the institute say it was the sort of necessary and worthwhile experiment that higher education institutions undertake all too rarely, even as they acknowledge a set of miscalculations that — along with political and cultural circumstances in part beyond its control — contributed to its undoing.
Among those missteps, according to numerous people who worked closely with the institute, were a tendency to approach campus faculty members with arrogance rather than as partners, and a lack of clarity from the start about the endeavor’s goals and business plan. Those were compounded by a problem that afflicts many educational technology efforts, whether funded by Silicon Valley investors or campus leaders: too little patience for what is almost always a long, drawn-out process of adoption and proof.
It’s worth reading the whole article, and sprinkled throughout are quotes from faculty and staff from the University of Texas system – ITL’s natural customer base – about overly optimistic claims, inability to understand or listen to its stakeholders, and real or perceived arrogance. I observed similar reactions when on the UT Austin campus for a video case study in Fall 2016 about their attempts to reinvent the large lecture class. When I asked about the overlap or collaboration with ITL on what seemed to be a perfect fit for that group’s objectives, I got eye rolls and comments about ITL being irrelevant for campus-based innovation efforts. Regardless of how that situation arose, the point is the same – there seems to have been a real breakdown of communication and trust between ITL and UT campus educators, despite the fact that “some of the work done by the institute has been divested to the various UT campuses.”
ITL was not alone as a failed, or at least abandoned, initiative to transform education based on innovative ed tech, and there is a theme in general of focusing more on the supposed tech-based innovation rather than the issue of cultural change and understanding social barriers (and this includes a culture’s view on whether the innovation is worth adopting). This is not a new subject, and similar challenges from a vendor standpoint led to the 2014 post “Pilots: Too many ed tech innovations stuck in purgatory”, an adaptation of which forms the second half of this post.
The general interest on dealing with the social change issues around real adoption of innovative education approaches – as well as the associated culture required to test theories, honestly assess results, and make continuous improvements in teaching practices – is what has led us at e-Literate to work on the Empirical Educator Project that Michael described a few weeks ago.
Despite the billions of dollars invested and provided in grants over the past several years, the vast majority of ed tech is used in only a small percentage of courses at most campuses. ((I’m not arguing against faculty prerogative in technology adoption and for a centralized, mandatory approach, but noting the disconnect.)) Most ed tech applications or devices have failed to cross the barriers into mainstream adoption within an institution. This could be due to the technology not really addressing problems that faculty or students face, a lack of awareness and support for the technology, or even faculty or student resistance to the innovation. Whatever the barrier, the situation we see far too often is a breakdown in technology helping the majority of faculty or courses.
Diffusion of Innovations – Back to the basics
Everett Rogers wrote the book on the spread of innovations within an organization or cultural group in his book Diffusions of Innovations. Rogers’ work led to many concepts that we seem to take for granted, such as the S-curve of adoption:
Source: The Diffusion of Innovations, 5th ed, p. 11
leading to the categorization of adopters (innovators, early adopters, early majority, late majority, laggards), and the combined technology adoption curve.
Source: The Diffusion of Innovations, 5th ed., p. 281
But Rogers did not set out to describe the diffusion of innovations as an automatic process following a pre-defined path. The real origin of his work was trying to understand why some innovations end up spreading throughout a social group while others do not, somewhat independent of whether the innovation could be thought of as a “good idea”. From the first paragraph of the 5th edition:
Getting a new idea adopted, even when it has obvious advantages, is difficult. Many innovations require a lengthy period of many years from the time when they become available to the time when they are widely adopted. Therefore, a common problem for many individuals and organizations is how to speed up the rate of diffusion of an innovation.
Rogers defined diffusion as “a special type of communication in which the messages are about a new idea” (p. 6), and he focused much of the book on the Innovation-Decision Process. This gets to the key point that availability of a new idea is not enough; rather, diffusion is more dependent on the communication and decision-process about whether and how to adopt the new idea. This process is shown below (p. 170):
Source: The Diffusion of Innovations, 5th ed., p. 170
What we are seeing in ed tech transformation initiatives in most cases, I would argue, is that the new ideas (applications, products, services) are stuck the Persuasion stage. There is knowledge and application amongst some early adopters in small-scale pilots, and there are good motivations in general from transformation groups like ITL and vendors, but majority of educators either have no knowledge of the innovation or are not persuaded that the idea is to their advantage, and there is little support or structure to get the organization at large (i.e. the majority of faculty for a traditional institution, or perhaps for central academic technology organization) to make a considered decision. It’s important to note that in many cases, the innovation should not be spread to the majority, either due to being a poor solution or even due to organizational dynamics based on how the innovation is introduced.
The Purgatory of Transformation
This stuck process ends up as an ed tech purgatory – with promises and potential of the heaven of full institutional adoption with meaningful results to follow, but also with the peril of either never getting out of purgatory or outright rejection over time.
Ed tech innovators can often be too susceptible to being persuaded by simple adoption numbers such as 1,100 institutions or total number of end users (millions served), or by new the promise of ideas that look great on paper, but meaningful adoption within an institution – actually affecting the majority of faculty or courses – is necessary in most cases before there can be any meaningful results beyond anecdotes or marketing stories. The reason for the extended purgatory is most often related to people issues and communications, and the ed tech market (and here I’m including transformation initiatives, vendors, consultants, and campus support staff and faculty) has been very ineffective in dealing with real people at real institutions beyond the initial audience for an idea.
As Larry Cuban stated in his commentary about Larry Berger’s “confession” that the engineeriomy model of personalized learning is flawed (also covered here at e-Literate):
First, those confessing their errors about solving school problems seldom looked at previous generations of reformers seeking major changes in schools.They were ahistorical. They thought that they knew better than other very smart people who had earlier sought to solve problems in schooling.
Second, the confessions seldom go beyond blaming their own flawed thinking (or others who failed to carry out their instructions) and coming to realize the obvious: schooling is far more complex a human institution than they had ever considered.
Finally, few of these confessions take a step back to not only consider the complexity of schooling and its many moving parts but also the political, social, and economic structures that keep it in place (see Audrey Watters here). As I and many others have said often, schools are political institutions deeply entangled in American society, culture, and democracy. Keeping the macro and micro-perspectives in sight is a must for those seeking major changes in how teachers teach or how schools educate. Were that to occur the incidence of after-the-reform regret might decrease.
Cover image credit: Ludovico Carracci [Public domain], via Wikimedia Commons
Let’s be honest: as an academic term of art, personalized learning is horrible. It has almost no descriptive value. What does it mean to “personalize” learning? Isn’t learning, which is done by individual learners, inherently personal? What would it mean to personalize learning? And who would want unpersonalized learning? Because the term carries so little semantic weight, it is a natural for marketing purposes: “Our personalized learning is new, improved, and 99.44% pure!” Unfortunately, this also sets it up perfectly for the inevitable War of Definitions. Remember the Great MOOC War a few years ago? Were MOOCs the creation of the Canadian Constructivists or of the Stanford professor who invented a self-driving car? Are we talking about an xMOOC or a cMOOC? Which one is the good one, and which one is the bad one? Now that the furor has died down, there is relatively little debating over the definition of the term MOOC and much more focus on how the family of approaches that are collected under that term can best serve different educational purposes.
Let’s just skip to the end this time, shall we?
The two of us spent the past three years visiting colleges and universities that have undertaken so-called personalized learning projects, and we talked to the students, teachers, and administrators about what they are actually doing and why they are doing it. We visited a wide range of institutions and talked to a wide range of stakeholders, based on our daily work as consultants to colleges and universities and as analysts of the educational technology industry and our work on a grant funded by the Bill & Melinda Gates Foundation. Through these observations, we have been looking for the ground truth underneath the hype of personalized learning. As a result of this process, we observed a family of technology-enabled educational practices that are potentially useful for a range of educational challenges. We would like to share our framework, which we hope will be useful for thinking about (1) the circumstances under which personalized learning can help students and (2) the best way to evaluate the real educational value for products that are marketed under the personalized learning banner.
Personalized Learning as Practice
Imagine for a moment that personalized learning is not already a term in the ed tech lexicon and, further, that there is no need for any new term to be “catchy” or “sticky.” The most descriptive label we could come up with for the practices that the two of us have observed in our school visits might be undepersonalized teaching. If the ideal, most personal teaching modality is one-to-one tutoring, there are many reasons why we fall short of this ideal in real-world classrooms. The most stereotypical depersonalized teaching experience is the large lecture class, but there are many other situations in which teachers do not connect with individual students and/or meet the students’ specific needs. For example, even a small class might contain students with a wide-enough range of skills, aptitudes, and needs that the teacher cannot possibly serve them all equally well. Or a student may have needs (or aptitudes) that the teacher simply doesn’t get an opportunity to see within the amount of contact time that the class allows. The truth is that students fall through the cracks all the time, even in the best classes taught by the best teachers. Failing a course is the most visible evidence, but more often students drift through the class and earn a passing grade—maybe even a good grade—without getting any lasting educational benefit.
If we choose to think of personalized learning as a practice rather than a product, we can start by taking a hard look at course designs and identifying those areas that fail to make meaningful individual contact with students. These gaps will be different from course to course, subject to subject, student population to student population, and teacher to teacher. Although there is no generic answer to the question of where students are most likely to fall through the cracks in a course, there are some patterns to look for (as we will discuss later in this article).
Technology then becomes an enabler for increasing meaningful personal contact. In our observations, we have seen three main technology-enabled strategies for lowering classroom barriers to one-on-one teacher/student (and student/student) interactions:
Moving content broadcast out of the classroom: Even in relatively small classes, a lot of class time can be taken up with content broadcast such as lectures and announcements. Personalized learning strategies often try to move as much broadcast out of class time as possible in order to make room for more conversation. This strategy is sometimes called “flipping” because it is commonly accomplished by having the teacher record the lectures they would normally give in class and assign the lecture videos as homework, but it can be accomplished in other ways as well, for example with reading-based or problem-based course designs.
Turning homework time into contact time: In a traditional class, much of the work that the students do is invisible to the teacher. For some aspects, such as homework problems, teachers can observe the results but are often severely limited by time constraints. In other cases, such as comprehension of assigned readings, the students’ work is invisible to the teacher and can be observed only indirectly and with significant effort. Personalized learning approaches often allow the teacher to observe the students’ work in digital products, so that there is more opportunity to coach students. Further, personalized learning often identifies meaningful trends in a student’s work and calls the attention of both teacher and student to those trends through analytics.
Providing tutoring: Sometimes students get stuck in problem areas that don’t require help from a skilled human instructor. Although software isn’t good at teaching everything, it can be good at teaching some things. Personalized learning approaches can offload the tutoring for those topics to adaptive learning software that gives students interactive feedback while also turning the students’ work into contact time by making it observable to the teacher at a glance through analytics.
None of these techniques, by themselves, undepersonalize the teaching. They generally need to be designed and implemented by skilled educators as part of a larger course design that is intended to address the particular problems of particular students. In the business world, an analogous initiative might be called “business process redesign.” Emphasis is on process. The primary question being asked is, “What is the most effective way to accomplish the goal?” The redesigned process may well need software, but it is the process itself that matters. In personalized learning, the process we are redesigning is that of teaching individual students what they need to learn from a class as effectively as possible (though we can easily imagine applying the same kind of exercise to improving advising, course registration, or any other important function).
We saw a noteworthy example of this educational design work in action at Essex County College (ECC) in Newark, New Jersey. The majority of ECC students need to take developmental math in order to complete their degrees, and the majority of those do not pass developmental math. Of those who do, the majority do not pass the college-level math course that follows. ECC leadership believed that this educational failure could be attributed to two main factors. First, students came into developmental math with an enormous range of prior knowledge: some had the equivalent of a fourth-grade math education; others needed to learn only a few concepts. Students on one end of the spectrum typically got lost because they were not receiving the individual help they needed, whereas those on the other end often got bored and eventually failed or dropped out because they were being forced to spend a lot of their time on skills that they had already mastered. Second, many ECC students had never been taught good study skills, and faculty did not have the class time needed to teach those skills. So to address the two personalization gaps in this particular course, the college redesigned developmental math using personalized learning techniques.
ECC used an overall pedagogical framework called Self-Regulated Learning. Students in the course spend part of their class time in a computer lab, working at their own pace through an adaptive learning math program. Students who already know much of the content can move through it quickly, giving them more time to master the concepts that they have yet to learn. Students who have more to learn can take their time and get tutoring and reinforcement from the software. Teachers, now freed from the task of lecturing, roam the room and give individual attention to those students who need it. They can also see how students are doing, individually and as a class, through the software’s analytics. But the course has another critical component that takes place outside the computer lab, separate from the technology. Every week, the teachers meet with the students to discuss learning goals and strategies. Students review the goals they set the previous week, discuss their progress toward those goals, evaluate whether the strategies they used helped them, and develop new goals for the next week.
Note the role of the software in this design. In the lab, it primarily takes on the role of tutor, helping most of the students most of the time with routine skill coaching and practice so that the teacher is freed up to give individual attention to those students who really need it. In the goal-setting sessions, the software acts mainly as a record keeper. It helps students track their time on task, number of problems solved, and so on. The teacher then helps students figure out what to do with that information. In both cases, the software is an important enabler of the new teaching practices. But the value that it adds is quite different from the way personalized learning software products are often characterized by sales reps, marketing materials, and many news stories. It is thus worth taking a minor detour from our exploration of personalized learning as a practice to examine the significant gap between the ground truth of the practice and the popular characterizations of the products.
Why Such Hype?
So far, we have deliberately and explicitly set aside the various policy, political, and business pressures that have brought the term personalized learning into broader use so that we could focus on the educational value that lurks underneath the hype. But it is also important to understand these pressures so that we can be on guard for the ways in which they might deform the discussion and distract us from the real value that we should be talking about. None of the three approaches that we identified above are particularly new; nor do they require fancy algorithms and expensive products to achieve. There are two specific reasons why these approaches are being attached to heavily marketed products right now.
First, on the policy side, there has been a shift in emphasis from access to degree completion. President Barack Obama set the tone by announcing a goal that the United States be number one in the world in the proportion of college graduates by 2020. Since then, state and federal policy makers have followed suit. Colleges and universities now have to account for gainful employment metrics and track their institutional scorecard results. Base funding, particularly for public institutions, is increasingly tied to performance against these and similar metrics. Grant funding is also increasingly outcomes-driven. As higher education institutions have narrowed their focus on these metrics, the students who fall through the cracks and fail out of the standard educational model have come into sharper relief.
With these policy changes and the funding that follows them, being “student-centric” is no longer a nice-to-have goal. Rather, it is a critical success factor for improving measurable student outcomes and therefore getting funding and being seen as a successful institution. For example, whereas in the past a few forward-thinking community college administrators might have thought to take on a project like ECC’s personalized learning developmental math redesign out of a sense of mission, now every community college administrator must be looking for ways to improve degree completion by eliminating failure traps such as developmental math. The security of institutional funding depends on it.
The second big change has been the widespread commercialization of the adaptive learning techniques that have existed in educational research laboratories for over fifty years. As the term adaptive learning suggests, these products provide students with a certain amount of one-on-one tutoring (although the methods that these systems use for analyzing students’ progress and providing useful feedback vary widely by product and discipline). This change in the market has been enabled by technological advances that are increasingly resulting in one networked computer for every student in a class and affordable developer access to machine learning tools.
Market forces are playing a big role in publishing too. Textbook publishers have found that their traditional business model is collapsing as more students find ways to avoid buying new textbooks. Cengage, for example, was forced to go through bankruptcy. McGraw-Hill Education was sold to a private equity firm. Pearson’s stock is near historic lows. All of these companies have had multiple rounds of major layoffs. They are in desperate need of a new product, and they are increasingly latching onto the personalized learning trend as the cure for what ails them. Meanwhile, according to Ambient Insight, U.S. ed tech companies received $3.6 billion of angel and venture capital funding in 2015. (This doesn’t even include mergers and acquisitions, which have also been huge.) Startup founders find themselves in an increasingly crowded field and are under strong pressure to promise and produce big results. Every vendor of a developmental math product, whether that vendor is an established textbook publisher or a young startup, is aware that campus presidents and provosts need to solve their degree-completion problem and that developmental math is very likely to be a big part of that problem. The vendors therefore market their products as a solution to degree completion. Every textbook vendor and aspiring textbook disruptor knows that stories about improving pass rates through technology sell. But what to call these products? Personalized learning is a term that sounds good without the inconvenience of having any obviously specific pedagogical meaning, so it becomes the flag that all vendors fly, even though different products do very different things and even though undepersonalization is rarely accomplished through software alone.
Thus, through policy and commercialization, the personalized learning marketing juggernaut was born. Unfortunately, the combination of the marketing shortcuts and the funding pressures created a strong temptation for magical thinking. Campus leaders are being asked to believe that they can solve their degree-completion and other accountability metric problems by buying software that will somehow magically provide personalized learning (in a way that faculty members, by implication, do not). For obvious reasons, faculty are likely to reject this stunted conception of personalized learning. Leaders who want to see their campus communities benefit from personalized learning approaches need to guard against product-centric characterizations and should suggest that discussions of vended solutions take place in the context of course and curricular designs that undepersonalize teaching. Otherwise, the baby will probably get thrown out with the bathwater.
Good Candidate Opportunities
One of the benefits of reframing personalized learning as undepersonalized teaching and focusing on the three techniques we outlined earlier is that faculty can readily translate this framework into their own contexts and start identifying opportunities that are good candidates for undepersonalization (not all of which will require vended products). In contrast to a product-centric conception of personalized learning, a practice-centered conception is something that faculty can own. That said, we have seen areas of opportunity where personalized learning is often a good fit, and not all of those areas are always obvious.
To begin with, any course that students enter with a wide range of prior knowledge and ability is a good candidate. ECC’s developmental math is a prototypical example. Another example is Austin Community College’s ACCelerator lab, used heavily for developmental math courses. But the course doesn’t have to be remedial and the institution doesn’t have to be access-oriented in order for personalized learning to be helpful. For example, at Middlebury College, a geography professor realized that some students in his course on geographical information systems (GIS) were struggling. For context, this is a general education course that is taken by students in a wide range of majors and specialties. In an elite college like Middlebury, nobody worries too much about completion rates, particularly at the institutional level. The professor simply observed that some students were working very hard. In fact, the course had such a strong reputation for difficulty that taking and passing it was considered a badge of honor. Students were sleeping in the labs. But the professor didn’t see this as a sign that he was inspiring his students to work hard. He saw it, rather, as a course-design problem. Some students were working harder than they should because he wasn’t reaching them in the way that they needed to be reached.
As it turns out, one critical skill that the course was teaching—spatial reasoning—is rarely taught in high school. A handful of the students in the class came in with either prior training or natural talent. They did well. The others were the ones who were sleeping in the labs. They needed more time and more help. The professor decided to make videos of his lectures and assign the videos as homework. With this change, struggling students could watch the videos as often as they needed in the (relative) comfort of their own dorm rooms while the professor’s class time was also freed up for more interactive work. (After he had done this, a colleague told the professor that this technique is called “flipping the classroom”—a term he had never heard before.) He is also thinking about developing tutorial software that can help students work through the homework problems in ways that best suit their needs.
Another obvious opportunity to undepersonalize teaching is in large lecture courses. For example, administrators at the University of California, Davis, became interested in redesigning their survey biology and chemistry courses because they recognized that they were losing a high percentage of first-year students—the ones who typically take these large lecture courses. It is very easy for a student to become passive in this broadcast-heavy course design. The team involved in the course-redesign projects wanted students to both get more individual attention and take more individual responsibility for their learning. To accomplish these goals, the team employed personalized learning practices as a way of making room for more active learning in the classroom. Students used software-based homework to experience much of the content that had previously been delivered in lectures. Faculty redesigned their lecture periods to become interactive discussions. Meanwhile, the teaching assistants who ran the discussion sections used analytics from the homework software to identify the areas where students were struggling; as a result, they could better focus their class time on those areas. Importantly, teaching assistants received additional training in how to employ active learning principles in their teaching techniques. Once again, in contrast to marketing pitches and popular narratives, the software played only a supporting role, albeit an important one, in undepersonalizing the large lecture.
A less obvious opportunity for personalized learning is in the design of problem-based learning courses. The previous two examples fit within the common understanding of personalized learning being used to help students work through traditional, didactic courses with more support. But Arizona State University incorporated the “flipping” aspect of the technology into an online STEM lab course for non–science majors. In this course students, for their final project, are asked to evaluate the likelihood that there are other intelligent civilizations in a randomly assigned field of stars. The teaching philosophy of the faculty member who was the lead designer of the course is that he should be a coach or a guide, helping students navigate difficult problems. In his view, both content delivery and assessment are activities that take time away from that core function. He and his colleagues framed the course, Habitable Worlds, as a series of challenges. Overcoming each challenge requires the students to learn new knowledge and skills. The course design is based on mastery learning: students must demonstrate that they have learned one skill before moving on to the next. The course is also difficult. Students often get stuck, which is by design. Faculty and teaching assistants, freed up from both content delivery and assignment grading, spend most of their time responding to students’ questions. And because the coursework is all software-based, they can see exactly what students are doing, how far students have progressed, and where students are struggling. Students can proceed at their own pace, moving quickly where they can and getting help where they need it. Yet despite the self-paced nature of the course, there is also a strong social component. Students can and often do seek out each other’s help. Because personalized learning practices make space for more interactivity, these practices often go hand-in-hand with active learning. And active learning is often social.
We suspect there are many opportunities in addition to the ones we have identified here. If faculty are given a commonsense framework and a chance to experiment, refine, and share, they will find novel and exciting ways to better support their students’ individual educational needs.
Doing It Right
Because personalized learning is a family of educational practices that support good course designs, implementing those practices well is not as simple as buying a product. To begin with, course design is always a time-consuming process when done correctly. Second, in many cases faculty will be trying techniques they have not used before, requiring them to teach in ways that are very different from how they have taught before, that are far removed from their experience (and therefore instincts) of what works and what doesn’t, and that may have ripple effects they don’t anticipate. On top of all this, the vast majority of faculty are neither trained in course design/research nor compensated for any time they invest in it. They will need time and support. In many cases, implementing personalized learning well can require an institutional effort analogous to the one required to implement an online learning program well.
Looking across a range of personalized learning projects that have had varying degrees of success, both at the schools we visited and elsewhere, we can identify six steps for a successful strategy. It should be noted that these themes could be applied to any number of pedagogical innovations.
Identify the student need that is to be addressed. The various personalized learning approaches are just one set of tools in the toolbox. Successful programs generally start by identifying a significant educational problem that faculty and program staff believe can be corrected with a change in course design.
Design the pedagogical structure. If the problem that is identified can be addressed through personalization, then how will the course support different students differently? The answer has to be more than just “adaptive learning.” Successful programs identify opportunities in the course design to improve individual support for students.
Pick the products or technologies. The details of different products or technological approaches are most meaningful when they impact what can be done with the course design. Successful programs pick the right tool based on the job at hand rather than on who has the best marketing pitch.
Don’t forget faculty training. Because personalized learning, done properly, generally means implementing new pedagogical approaches, faculty may need to learn to teach in ways that they haven’t taught before. Successful programs provide faculty with training and pedagogical support.
Don’t forget technology support. Software helps with learning only when it works, and Murphy’s Law can hit with a vengeance when technology is mixed with teaching. Successful programs make sure that faculty have the technology training, equipment, and support staff that they need in order to be successful.
Be prepared to measure, fail, and iterate. Because personalized learning approaches often require new software, new teaching techniques for faculty, new responsibilities for students, and in some cases new scheduling challenges, institutions will almost inevitably get some things wrong in the first couple of iterations, and those mistakes may have real impact on outcomes. Successful programs approach implementation empirically but with patience.
On the bright side, the fact that personalized learning is now being attached to funding-related metrics such as degree-completion rates means that attaching institutional support costs to a funding stream will also be easier. In many cases, schools can build personalized learning “muscle mass” by focusing on metric-relevant projects first and then expanding the initiative once the critical institutional knowledge and support mechanisms have been put in place.
Final Thoughts: Ed Tech Groundhog Day
There is a lesson to be learned here, and it is broader than personalized learning. Every popular ed tech trend, going at least as far back as the original online asynchronous distance learning courses in the 1990s, has brought with it a food fight, with proponents hyping the trend as revolutionary and opponents attacking it as harmful. And in every case, a policy or other institutional driver has resulted in a rush of companies responding to the market opportunity created by that driver. Together, these forces generate hype and magical thinking, which in turn provoke an equal and opposite reaction.
In this article, we have tried to identify specific teaching practices being used by educators, and we have tried to describe them in commonsense terms that should make intuitive sense to experienced teachers. These practices always exist at the beginning. Some teacher somewhere comes up with a specific approach to a specific problem. External forces then make that problem a more institutionally consequential one, and companies rush in to name, market, and sell solutions. In the process, we lose track of the original educational idea. It’s like playing a game of telephone in a noisy airport. Except in this case, the message in the game is an actual plan for how we are going to help our students, and when it gets to the end of the telephone line, we will act as if we received the message with perfect fidelity. And then fight over it. Endlessly. Much of the Gartner hype cycle can be attributed to this process.
We can break out of this hype cycle with a fairly simple (though not necessarily easy) approach. Whenever a new ed tech trend gets named—whether it is distance learning, adaptive learning, personalized learning, competency-based education, MOOCs, or something else—we should start trying to understand that trend by looking for the best examples of what teachers and students are doing when they are doing the thing we just named. We should ask them what they are doing, and why. We should ask how their practice is working and what they are learning and what they don’t yet know. We should attach the name of the new trend to those educational practices and those reasons—rather than to any products, technologies, or services. We should not waste time debating whether the name we came up with for those practices is the perfect name or exactly what it includes or excludes. Instead, we should spend our time trying to understand the practices themselves and their applicability to the educational problems we are trying to solve.
Yes, personalized learning is a lousy term, but it is attached to legitimate educational practices that have the potential to improve the lives of many students. It is also a term that is trapped in the early stages of its hype cycle. So let’s just skip to the end and break personalized learning out of the hype cycle by doing our best to understand—and explain—what it really is and why it really matters.
In an article Michael and I wrote for EDUCAUSE Review in 2016, we described our view of personalized learning as “a family of teaching practices that are intended to help reach students in the metaphorical back row”. One of the key practices focused on gaining increased visibility into student coursework.
These same automated homework tools can also give teachers an easy view into how their students are doing and create opportunities to engage with those students. “Analytics” in these tools are roughly analogous to your ability to scan the classroom visually and see, at a glance, who is paying attention, who looks confused, who has a question.
With the usage of digital courseware to provide the homework tools, this focus on visibility into the learning process can apply across the entire course. What are different schools learning in this area?
As part of our e-Literate TV series of video case studies, we had a chance this fall to interview several institutions that are focusing on the benefit of increased visibility into student learning by usage of digital courseware based on interviews at the Realizeit users conference. ((This post is not meant to endorse Realizeit’s platform over other companies’ platforms. We are focusing on institutional perspectives and lessons to be learned.))
In the first episode we explored the challenge of going beyond pilots and deploying systems at scale. In this second episode I interview representatives from the University of Mississippi and the University of Central Florida, asking them to describe their experiences and focus on the issue of increased visibility.
We’ll share one more set of interviews from this conference in the coming weeks.
This post is part of our e-Literate TV series, which is funded in part by the Bill & Melinda Gates Foundation. The findings and conclusions (or views) contained within are those of the authors and do not necessarily reflect positions or policies of the Bill & Melinda Gates Foundation.
There is an article in EdWeek that quotes Larry Berger, CEO of Amplify, in his “confession” about personalized learning. The focus is on K-12 education but applies directly to higher ed as well.
Until a few years ago, I was a great believer in what might be called the “engineering” model of personalized learning, which is still what most people mean by personalized learning. The model works as follows:
You start with a map of all the things that kids need to learn.
Then you measure the kids so that you can place each kid on the map in just the spot where they know everything behind them, and in front of them is what they should learn next.
Then you assemble a vast library of learning objects and ask an algorithm to sort through it to find the optimal learning object for each kid at that particular moment.
Then you make each kid use the learning object.
Then you measure the kids again. If they have learned what you wanted them to learn, you move them to the next place on the map. If they didn’t learn it, you try something simpler.
If the map, the assessments, and the library were used by millions of kids, then the algorithms would get smarter and smarter, and make better, more personalized choices about which things to put in front of which kids.
I spent a decade believing in this model—the map, the measure, and the library, all powered by big data algorithms.
Here’s the problem: The map doesn’t exist, the measurement is impossible, and we have, collectively, built only 5% of the library. [snip]
So we need to move beyond this engineering model. Once we do, we find that many more compelling and more realistic frontiers of personalized learning opening up.
Larry is exactly right that there is a fundamental problem with the assumptions behind what he calls the engineering model of personalized learning. But there are alternate models that offer “more compelling and more realistic frontiers”. We have described this contrast in models at e-Literate, most directly in Michael’s post The Battle for “Personalized Learning”.
Phil and I have decided to claim this prime piece of linguistic real estate. We are asserting squatters’ rights.
We hereby decree, by the power vested in us by nobody at all, that “personalized learning” shall henceforth refer to a family of teaching practices that are intended to help reach students in the metaphorical back row. The ones who are bored, or confused, or tuned out, or feeling stupid. Personalized learning practices are almost always ones that teachers have been using for a very long time but that digital tools can support or enhance. Here are a few that we have identified so far:
Move content broadcast out of the classroom: In many disciplines, the ideal teaching format is a seminar, in which students spend class time engaged in conversation with a professor. In others, it is a lab. Both models have students actively engaged in academic practice during class time, when the professor, as the expert practitioner, is present to coach them. Every class spent lecturing is a wasted coaching opportunity.
Many disciplines have traditionally used assigned readings to move content broadcast out of the classroom, and some still do. But it is not always possible to find readings that capture what you want to cover, and in any case, it is becoming harder to persuade students to read. Luckily, there are tools that can help with this problem. You can record and post your lectures as videos, which students can watch as many times as they need to absorb what you’re trying to tell them. You can assign podcasts that they can listen to on the go, or find interactive content that keeps them more engaged.
Make homework time contact time: Good teachers help students see the direct connection between the work they do at home and the overall purpose of the class. They do this in a variety of ways. Sometimes they mark up and comment on the student work. Sometimes they ask the students questions in class that require them to build on the work they did at home. For a variety of reasons, which often boil down to professors’ having less available time per student, this has become harder to do. The great crutch that is now being used to limp along without actually solving this problem is robo-graded homework assignments. By itself, automated practice might help some students drag themselves through to the end of the semester. But it doesn’t often inspire them to think that maybe they are not destined to be the student in the back row forever. (There are important exceptions to this rule, which I address below.)
On the other hand, these same automated homework tools can also give teachers an easy view into how their students are doing and create opportunities to engage with those students. “Analytics” in these tools are roughly analogous to your ability to scan the classroom visually and see, at a glance, who is paying attention, who looks confused, who has a question. Nor are these the only tools available for making homework time feel less isolated and pointless. Any homework activity that is done electronically can be socially connected. Group work done on a discussion board can be read over by the professor when she has time. Highlights and margin notes on readings can be shared and discussed in class. This sort of effort on the professor’s part doesn’t have to be exhaustive (or exhausting). Sometimes a small gesture to show a student that you see her is all it takes.
Hire a tutor: You know what tutors are typically good for in your particular discipline. You also know that there generally aren’t enough good ones available, and that even when there are, it’s tough to get students to come into the tutoring center. One of the best uses of machine-graded homework systems, especially when they are “adaptive,” is to treat them as personal tutors that are available to students whenever they need them and wherever they are. They aren’t perfect, but what tutors are? Sometimes getting students out of the back row means helping them to believe that they are capable of learning. And sometimes students are willing to pose a question to a computer that they would be embarrassed to ask in person. In those cases, a little extra practice and feedback on the basics, without judgment, can make all the difference — even if the feedback comes from a machine. And if adaptive learning robo-tutors don’t fit the needs of your students and your discipline, technology also makes it possible to connect students with actual human tutors, who are available online to help them get through the rough spots.
There is a battle for personalized learning, and the description of the engineering model is useful for understanding one approach (unfortunately the one most often used in marketing and by ed reformers). But there is an alternative and it is more compelling.
On Friday I reported about SB2328, a bill that passed (with amendments) the Hawai’i Senate Committee on Higher Education and would have mandated open educational resources (OER) for all courses at all 10 University of Hawai’i campuses. And if there were no adequate OER materials for a course? “. . . the faculty member or lecturer responsible with providing instruction for the course shall create the instructional materials and offer those materials free of charge to students through open educational resources.”
This bill was a disaster in the making. Not only would it have been unworkable in terms of funding and intellectual property ownership, it would also have set back the OER movement by associating OER with unfunded faculty mandates and reduction of academic freedom. All this from good intentions but apparently shallow understanding.
As mentioned in an update to Friday’s posts, the amendments that resulted from committee hearings removed the mandates. We now have the amended language, and it is a completely different bill.
(1) Establish the University of Hawai‘i open educational resources task force to conduct a comprehensive analysis and evaluation on all general education courses and high attendance courses taught at the University of Hawai‘i system to identify open educational resources for those courses;
(2) Establish and appropriate funds for an open educational resources pilot project grant program to incentivize faculty that adopt, develop, and implement open educational resources; and
(3) Require the University of Hawai‘i open educational resources task force to report its findings and recommendation initiatives for supporting and expanding the use of open educational resources at the University of Hawai‘i to the Legislature prior to the Regular Session of 2019.
During the hearings there were several dozen testimonies shared, and all but two opposed the bill (and one of those two changed positions to oppose). Leading the opposition was the University of Hawai’i Professional Assembly, the local faculty union.
In short, the bill amendments removed mandates, creates a task force charged with a one-year evaluation of high-enrollment and general ed courses, and creates a $50,000 grant fund to incentivize faculty adoption.
It is not clear whether the bill will make it through remaining hurdles to become state law, but if it does, we will have a fairly significant move in the state dealing with the costs of curricular materials and OER adoption.
Billy Meinke, OER Technologist and UH Manoa and a key player within the system (let’s call him Kane OER), was unaware of SB2328 before it came out. This gets to the heart of the problem – the original bill appears to have been written without any input from the people already working on OER adoption within the University of Hawai’i.
I still have a problem with the preamble of the bill that uses the misleading claim that the “average cost for books and supplies for the same academic year at public colleges averaged $1,250.” Students actually pay about half this amount, and this false setup will lead to erroneous estimates of how much any such bill could save for its students. But for now, crisis averted.
Thanks for update from Brent Auernheimer, I found out that the Hawai’i Senate Committee on Higher Education recently debated a bill regarding Open Educational Resources (OER) usage at the University of Hawai’i system of 10 campuses. Introduced on January 19th, SB2328 states:
Beginning with the 2020-2021 school year, all courses at all campuses within the University of Hawai‘i system that require the use of instructional materials, including textbooks, shall use instructional materials from the open educational resources at the University of Hawai‘i; provided that the use of instructional materials, including textbooks, that requires a student to purchase or pay a subscription for the materials shall be prohibited; provided further that if open educational resources does not have relevant instructional materials available for a course, the faculty member or lecturer responsible with providing instruction for the course shall create the instructional materials and offer those materials free of charge to students through open educational resources.
Read that carefully – OER for all courses, no commercial services around OER allowed, and if appropriate OER does not exist, the faculty member must create the material themselves, all mandated from the state legislature.
When I first saw this news, I assumed it was a either a misguided effort that would quietly be killed in committee or a political statement. Predictably, and appropriately, the University of Hawai’i Professional Assembly actively opposed this bill, calling it “legislative overreach” and “infringement on academic judgement”, while also calling out the costs and support needed for faculty to create such materials.
On January 30 hearings, the vast majority of testimony – much of it from faculty members – opposed the measure with only two statements supporting. Yet on February 6, the Senate Committee unanimously passed the bill on to the full Senate.
The committee(s) on HRE recommend(s) that the measure be PASSED, WITH AMENDMENTS. The votes in HRE were as follows: 5 Aye(s): Senator(s) K. Kahele, Kim, S. Chang, Keith-Agaran, Kidani; Aye(s) with reservations: none ; 0 No(es): none; and 0 Excused: none.
I have not been able to determine what the amendments are for the bill (or if that refers to future amendments coming from floor debate), and I also do not know how likely it is to pass the full senate or to become state law. I’ll keep looking for more information.
Unless I’m missing something, this could be a jump-the-shark moment for portions of the OER movement. Comments appreciated.
Update: From Twitter stream (sounds like some good changes):
Several years ago I wrote a post titled “Pilots: Too many ed tech innovations stuck in purgatory”, where I used Everett Rogers’ Diffusion of Innovations framework to explore why we have plenty of pilots but not very many large-scale adoptions of ed tech innovations.
What we are seeing in ed tech in most cases, I would argue, is that for institutions the new ideas (applications, products, services) are stuck the Persuasion stage. There is knowledge and application amongst some early adopters in small-scale pilots, but majority of faculty members either have no knowledge of the pilot or are not persuaded that the idea is to their advantage, and there is little support or structure to get the organization at large (i.e. the majority of faculty for a traditional institution, or perhaps for central academic technology organization) to make a considered decision. It’s important to note that in many cases, the innovation should not be spread to the majority, either due to being a poor solution or even due to organizational dynamics based on how the innovation is introduced.
This stuck process ends up as an ed tech purgatory – with promises and potential of the heaven of full institutional adoption with meaningful results to follow, but also with the peril of either never getting out of purgatory or outright rejection over time.
Accordingly, we have more information about institutions with quite a few pilots around digital courseware, but there is not much information about colleges or universities implementing at scale. What are the problems to be solved for large-scale adoption, and what lessons can be learned (both positive and negative)?
As part of our e-Literate TV series of video case studies, we had a chance this fall to interview several institutions that are dealing with this challenge – deploying courseware at scale – based on interviews at the Realizeit users conference. ((This post is not meant to endorse Realizeit’s platform over other companies’ platforms. We are focusing on institutional perspectives and lessons to be learned.)) In a future episode we’ll describe more directly what Realizeit’s platform is and is not, but to start, let’s get a sense of the institutional perspective.
In this first episode I interview representatives from American Public University System (APUS) and Bay Path University, asking them to describe their programs. APUS has redesigned more than 1,600 courses based on active learning, as well as new competency-based programs, and Bay Path is applying courseware broadly across the entire institution.
We’ll share additional interviews from this conference in the coming weeks.
This post is part of our e-Literate TV series, which is funded in part by the Bill & Melinda Gates Foundation. The findings and conclusions (or views) contained within are those of the authors and do not necessarily reflect positions or policies of the Bill & Melinda Gates Foundation.
