e-Literate

Present is Prologue

Category: Recommended Reading

  • Recommended Reading: Article in Chronicle on Measuring Learning

    Dan Barrett’s piece over the weekend in The Chronicle, “The Next Great Hope for Measuring Learning,” deserves a close read. He describes in some detail a ground up effort by faculty and administrators across several institutions to define and measure what it is that students are learning and why it’s important. In doing so, these faculty and administrators are moving beyond looking simply at content mastery and focusing on broader skills of quantitative reasoning, writing and critical thinking. It’s forcing them to develop new approaches to reviewing student work, moving away from the idea of “grading” and toward “scoring” against a rubric that looks more sophisticated learning outcomes.

    The effort, directed by the Association of American Colleges and Universities (AAC&U) – an active advocate in this space (see Michael’s earlier blog post, “Outcomes-Based Education and the Conservative Radicalism of the AAC&U”), is not without its skeptics, including many participants in the actual project. That said, the approach is being watched closely as a way for higher education to define and measure appropriate criteria that lead to improvements in teaching and learning while simultaneously speaking to accountability issues around which there is so much attention.

    Like many promising initiatives in higher education recently highlighted by e-Literate, well thought out, collaboratively developed homegrown solutions usually trump those imposed by third parties working in isolation.

    Note: For those without a Chronicle subscription, use this link to get behind the firewall for 24 hours after this post publishes.

  • SRI’s Study on Gates Personalized Learning Grants Is Out

    This is almost old news now, but we just haven’t been able to dig into it yet. As part of its Adaptive Learning Market Acceleration Program (ALMAP) program, the Gates Foundation funded SRI to do a study of the results of the grants after two years. I hope to finally clear some time to parse through it this week, but two high-level points jump out at me at first glance (neither of which is a big surprise):

    1. There are no conclusive wins here. This is not a robot-tutor-in-the-sky moment. A few programs did well here and there. A handful produced promising incremental gains. But this is not a report that screams, “Wow, adaptive courseware works!” The most you can say is that adaptive learning looks like it could be another arrow in the quiver that helps out in some situations—like developmental math courses at two-year colleges, for example.
    2. Large-scale educational research is incredibly hard and may actually be impossible to do rigorously for certain kinds of questions. I’ll probably get into this more when I’m ready to really write about the report, but one reason the conclusions are murky is because there so many variables in each class—not just each course subject, not just each course at one university, but even with each section of each class taught by one teacher—that really matter, some of which are impossible to control and others of which are unethical to control. It would be a mistake to overinterpret the study as showing that adaptive learning doesn’t help much. I’ve seen studies with narrower focus that have gotten clearer gains. This brings us back to the arrow-in-the-quiver theory. The wider the scope of the research focus, the more that pockets of real benefit will be obscured by noise and uncontrolled variables.

    The report is here.

  • We Need a More Robust Learning Sciences Research Community

    My latest Chronicle column is on how inherently difficult it is to evaluate learning science claims, particularly when they get boiled down to marketing claims about product efficacy, and how deep academic distrust of vendors makes this already incredibly difficult challenge nearly impossible.

    Here’s where I stand on vendor participation in ed tech and learning science research:

    On the one hand, vendors have access to data and resources that, for a variety of reasons, are difficult or even impossible for universities to access. They sometimes have millions of students using their products and relatively few internal barriers to doing certain kinds of large-scale effectiveness research (although to a certain degree, territoriality, bureaucracy, and poorly designed data architectures are universal problems). On the other hand, vendors should not under any circumstances be allowed to define the research agenda and arbitrate the validity of learning sciences claims for all of education. Theoretically, there is a simple solution to this. Higher ed knows how to do science. It knows how to review scientific claims. Vendor research should be put through the peer review process. They should publish enough information that their results can be evaluated and, when feasible, duplicated. Their reputations should be based in part on who is doing (or, at least, properly using) good scientific research. After all, most of the researchers at these companies are PhDs who are trained in the academic research process. There is no reason in the world why these companies can’t contribute to real, rigorous, peer-reviewed dialog and progress in the field. (more…)

  • Undepersonalized Teaching vs. Learnification

    Amy Collier was kind enough to post the video and notes from a recent keynote she gave. (For those of you who don’t know Amy, she is the Associate Provost for Digital Learning at Middlebury College and well worth following. She doesn’t blog that often, but when she does, she has interesting things to say.) A central element of her talk was on the “learnification” of education; that is, how the teacher disappears from the conversation about “good” education and the whole thing gets reduced to learners gobbling up little learning objects to get their competency level-up, like a human game of Pac-Man. This is one reason why Phil and I felt it was important to reframe “personalized learning” as a set of teaching practices that we called “undepersonalized teaching” in our recent EDUCAUSE Review article. When we visited actual classes (including Middlebury’s) and saw what actual teachers were doing as part of our e-Literate TV case study series, it was clear that there were thoughtful teachers using these tools in interesting ways that enabled them to spend more time on the high-value aspects of their teaching craft. But the student-as-Pac-Man story is the one that gets heard.

    I would like to bring out one additional point about the learnification of education that Amy briefly hints at in her keynote but that you will miss if you only read her notes is that “learnification” is not something that only comes from the “neo-liberal” ((Honestly, I still don’t really know what that word means, other than “bad” and “vaguely associated with people who have money.”)) emphasis on measurement of learning outcomes, career readiness, scale, and student as consumer. The Gert Biesta paper that she references actually pins some of the blame on constructivism, which very often gets flattened into some version of “Hey, teacher, leave them kids alone!” In its historical context, constructivism is a useful antidote to Pac-Man reductivism of the prevailing educational theory of the time, but like the modern, mechanized variant that it is most frequently deployed to counter today, it tends to de-emphasize the role of the teacher to the point where that role all but disappears.

    The picture that I have in my own head for the role of the teacher involves a kind of a bell curve for formal education. At the left-hand tail is the basic stuff that people need to know in order to start reasoning and learning for themselves—facts, basic concepts, habits of mind and of study, and so on. The specifics of what goes into that category are somewhat up for grabs and depend a lot on the student’s long-term goals. But whatever they are, many of them lend themselves to the sort of learning objectification/adaptive learning approach just fine. On the far end of the bell curve of formal education (but hopefully in the fat part of bell curve of normal adult life) is the complex reasoning that students apply on their own to synthesize what they are seeing into their own individual perspectives on the world. It’s the sort of thing that one hopes goes on in an undergraduate’s capstone or senior thesis project (and often sooner for different students in different classes). At that point, the training wheels of formal education come off.

    The fat middle part of the curve is where good teachers should be spending most of their time. It’s the work of getting the students ready to ride their bikes without the training wheels. The “not yetness” that Amy talks about in her keynote (and elsewhere) is basically about taking your hand off the kid’s shoulder to give her a few seconds to ride on her own. It’s the moment when she realizes that she is free to steer, to ride a head, and yes, maybe to fall. You catch her if you see her in danger of falling badly enough to hurt herself, but not before. It’s the important how part of Vygotsky’s zone of proximal development.

    The shape of the curve changes somewhat depending on the student population, but a classic bell curve image fits well both with the typical student progression in a four-year college as well as within the work to be done in a typical 100- or 200-level course. Most of the work of college should be about coaching students on how to think for themselves. Unfortunately, teachers face some practical problems in getting to what is supposed to be the fat part of the curve. The first one is that some students get stuck on the foundational stuff—different students in different places—and the standard lecture-format class doesn’t give teachers much opportunity to see where they are stuck and help them to get unstuck. As a result, teachers either have to deal with the “slowest car on the freeway” problem in their class or else decide to let more students fail rather than taking the time to help them get unstuck. (Or both. If you slow down too much, the students who understand most of the basics tune out and can miss the few critical pieces that they still need to learn.) Second, we have trained students through that very same lecture format that the basics are not something that they should even try to master for themselves. Basics are transmitted during class time as lectures.

    Personalized learning can be seen as a set of techniques designed to fix those two problems through the use of supporting technology. Proper use of adaptive “skill and drill” platforms, student retention early warning systems, and the like can actually free up the teacher to create that environment of “not-yetness” in the classroom. It’s not either/or but both/and.

    ERO Graphic 1

  • IHE Essay: Getting the political facts straight about State Authorization Reciprocity Agreement

    The following essay, co-written by Russ Poulin and Phil Hill, was originally published at Inside Higher Ed in response to articles in the New York Times and Inside Higher Ed regarding whether New York state should sign the SARA agreement.

    A coalition of consumer groups, legal aid organizations and unions object to the state of New York joining an agreement that would change how colleges offering distance education courses in the state would be regulated. As coalition members asserted in an Inside Higher Ed article, the state would be ceding its authority to other states. Students would be left with no protection from predatory colleges and it would make it easier for “bad actors to take advantage of students and harder for states to crack down on them.”

    That all sounds ominous. It would be, if it were true.

    Even in the digital era, the regulation of educational institutions is left to each state. The resulting array of requirements confuses both students and institutional faculty and staff. The State Authorization Reciprocity Agreement (SARA) was created to apply consistent review standards across the states. An institution approved in its home state is eligible to enroll students (within limits) in any other SARA member state. As of this writing, 36 states have joined in a little over two years. That number may approach 45 by the end of 2016.

    (more…)

  • The Odd Couple: How Ed Tech Must Support Vastly Different Types of Professors

    An edited version of this post was first published at The Chronicle of Higher Education

    Let’s admit it, there can be some real tension when a college is faced with choosing a new learning-management system, or any software used by more than one department.

    Since the decision involves the administrators who will support the system — commonly called an LMS — and professors who will use it, who should lead the process? Should staff members just get input from faculty members, or should professors vote on the final decision? Or should professors run the process?

    This is when distrust between the two sides can emerge. IT administrators may fear that professors will make the pace unbearably slow with an overly deliberative approach. And professors may assume that IT has already determined what system to buy, and may not take their input seriously. Or they may worry that the meetings will be dominated by those needy power users from the psychology department.

    (more…)

  • Dammit, the LMS: The Play at Home Game

    A while back, I wrote a rant about why LMSs fail to evolve. It got some…notoriety. Since Phil and I are now writing a bi-weekly column for the Chronicle‘s new Re:Learning section, I thought I’d try trimming down the piece and rewriting it for a general audience. It was an interesting exercise. On e-Literate, we have the luxury of writing pieces that are as long as we want them to be and of assuming that our audience is pretty knowledgeable about a wide range of tech issues. We can also choose to target different subgroups of our audience with each post. Honestly, there have been times when I’ve written a post that was primarily intended to persuade one (influential) person, although I always try to make my posts interesting to a broader range of people. When we write for the Chronicle, we are always writing for a broad audience of non-techies, and we have to keep our ideas small and focused enough that we can adequately express them in 1500 words or less. But to the degree that we succeed, we have an opportunity to be ambassadors for ed tech issues to broader academia.

    Since many of you play that role of ambassador every day, I’d be grateful for your feedback. If you have a moment, please go read the article at the Chronicle and then come back here and comment on it. Did I pick the right aspects of the issue to focus on? Do you think this piece would resonate with your campus stakeholders? Would it be helpful to you in your conversations at work?