e-Literate

Present is Prologue

Tag: MIke Caulfield

  • Comparing Fully-Online vs Mixed-Course Enrollment Data

    Mike Caulfield wrote a post yesterday about a new Blackboard report on design findings regarding online students. The focus of Mike’s post was that people often assume that the norm for an “online” student is taking all courses online, when in fact it is more common for students to take some courses online and some face-to-face (what Mike calls “Mixed-Course”). This distinction is important:

    What shocks some people reading this, I think, is that online students would have face-to-face courses as a comparison or option, or that they would be consciously choosing between online and face-to-face in the course of a semester. But this is the norm now at state universities and community colleges; it’s only a secret to people not in those sorts of environments.

    I agree that this is an important topic, so let’s look at the data in more detail (and for those wondering if you can connect to Tableau Public to edit online data visualizations while flying, the answer is yes if you’re patient). The best source of information is the IPEDS database, with the most recent data for Fall 2014. WCET put out an excellent report analyzing the distance education data in IPEDS – more on that later. Both IPEDS and WCET use the term “Some but not all” in the same manner as “Mixed-Course”. Based on overall data combining undergraduate and graduate students, Mike is right that more students are Mixed-Course than they are Fully-Online – 2.926 million vs. 2.824 million. (more…)

  • What Homework and Adaptive Platforms Are (and Aren’t) Good For

    I was delighted that we are able to publish Mike Caulfield’s post on how ed tech gets personalization backwards, partly because Mike is such a unique and inventive thinker, but also because he provided such a great example of how “personalized learning” teaching techniques are different than adaptive content and other product capabilities.

    The heart of his post is two stories about teachable moments he had with his daughters. In one, he helped his middle school-aged daughter understand why an Iranian author was worried that people in the Western world have harmful stereotypes of Iranians. In the other, he helped his high school aged-daughter see how her knowledge of the history of rocket science could be useful in answering a question she was asked about Churchill’s Iron Curtain speech. Mike’s stories show truly significant learning of the kind that changes students’ perspectives and, if we’re lucky, their lives. It is not just personalized but deeply personal. He was able to reach his daughters because he understood them as humans, well beyond the boundaries of a list of competencies they had or had not mastered within the disciplines they were studying.

    For now and the foreseeable future, no robot tutor in the sky is going to be able to take Mike’s place in those conversations. This is the kind of personal teaching that humans are good at and robots are not. But neither are the tools we have today useless for this sort of teaching. Vendors, administrators, and faculty alike have broadly misunderstood their role and potential. In this post, I’m going to talk about both how these tools are useful for the kind of education that Mike cares about (and I care about) as well as, perhaps more importantly, why we are so prone to getting that role wrong.

    (more…)

  • We Have Personalization Backwards

    [Note – an earlier version of the first half of this post was first published at Mike’s Hapgood site. We asked him to make some alterations for the e-Literate audience and republish here. – ed]

    Indie Rock and Donald Trump

    I drive my oldest daughter to high school every day. She goes to a magnet STEM school in the district that’s on the campus where I work. I’ve been brainwashing her into liking indie rock one car ride at a time using carefully planned mix CDs.

    Last week she tells me I need to put more Magnetic Fields songs in the mix. Why? I ask.

    “Physics homework.” she says.

    It turns out that there’s a number of principles of physics that she remembers through a complex set of associations she’s developed referencing indie rock songs. I don’t pretend to get them all, but the 69 Love Songs hit “Meaningless” plays an apparently crucial role.

    Later that day, my youngest daughter is asking me about the book Persepolis, a book about growing up Iranian during and after the 1979 Islamic Revolution. The author of that book spends the preface talking about the reasons she wrote it, and how she felt the understanding of her native country of Iran was too narrow, and in a way, too exotic. My daughter tells me that she doesn’t quite get what the author is talking about. After all, there’s a lot of fundamentalism in the early parts of the book — and people really are in a revolution in 1978, so what are we getting wrong in the West?

    I know that this daughter, a middle schooler, has had some stress about Donald Trump. She has people in her class who like him, and she can’t understand why when he’s so mean. It worries her.

    I ask her if Trump gets elected, how would she feel if everyone assumed all Americans were like Donald Trump. Well, we wouldn’t be, she says.

    Oh, she says.

    We Have Personalization Backwards

    (more…)

  • Data To Back Up Concerns Of Textbook Expenditures By First-Generation Students

    David Wiley has added to the conversation ((My initial post, Mike Caulfield response, Bracken Mosbacker, my response to Mike, Mike follow-up)) over use of data on college textbook pricing and student spending patterns with “The Practical Cost of Textbooks”. The key argument is to go beyond prices and spending and look at the most direct measure of asking students themselves how textbooks costs have impacted them. He then looks at the Florida Virtual Campus surveys (also included in my post), concluding:

    What impact does the cost of textbooks have on students? Textbook costs cause students to occasionally or frequently take fewer courses (35% of students), to drop or withdraw from courses (24%), and to earn either poor or failing grades (26%). Regardless of whether you have historically preferred the College Board number or the student survey number, a third fact that is beyond dispute is that surveys of students indicate that the cost of textbooks negatively impacts their learning (grades) and negatively impacts their time to graduation (drops, withdraws, and credits).

    And yes, we need to do something about it.

    Amen. Surveying over 18,000 students, the FVC surveys are quite important and should be on everyone’s radar.

    More Out Of Data

    (more…)

  • Asking What Students Spend On Textbooks Is Very Important, But Insufficient

    Mike Caulfield responded to my post on data usage to understand college textbook expenditures. The core of my argument is a critique of commonly cited College Board data. That data originating from financial aid offices leads to the conclusion that students on average either spend or budget $1,200 per year with that number rising, while there is more reliable data originating from students showing the number to be half that amount and dropping.

    In Mike’s response post yesterday, he generally agreed with the observation but is concerned that “readers of that piece are likely to take away the wrong conclusion from Phil’s figures (even if Phil himself does not)”. There is a risk that people see the lower numbers and conclude the “crisis is overblown”, leading to this observation:

    If we’re looking to find out if prices for some set of goods are too high, then by definition we cannot look at what people are spending as a reliable gauge, because one of the big effects of “prices too high” is that people can’t afford what they need.

    If you don’t pay attention to this you get in all sorts of tautologies.

    In the specific world of textbooks, Mike considers the lower-cost method of renting used textbooks, noting:

    So which figure do we use here? The chances of getting everything you need as a rental are low. Sure, you could be the super-prepared student who knows how to work the system and get them *all* as rentals — but not every student can be first in line at the bookstore. And the ones at the back of the line — guess their socio-economic class and first generation status?

    This is an important issue, and I appreciate Mike’s understanding that I am not arguing that college textbook pricing is an overblown crisis. I agree that the crisis is real and that the hardest-hit are likely low socio-economic class and first generation students.

    But let’s move past these agreements and drop the gloves. (more…)

  • NPR and Missed (Course) Signals

    Anya Kamenetz has a piece up on NPR about learning analytics, highlighting Purdue’s Course Signals as its centerpiece. She does a good job of introducing the topic to a general audience and raising some relevant ethical questions. But she missed one of the biggest ethical questions surrounding Purdue’s product—namely, that some of its research claims are likely false. In particular, she repeats the following claim:

    Course Signals…has been shown to increase the number of students earning A’s and B’s and lower the number of D’s and F’s, and it significantly raises the chances that students will stick with college for an additional year, from 83% to 97%. [Emphasis added.]

    Based on the work of Mike Caulfield and Al Essa summarized in the link above, it looks like that latter claim is probably the result of selection bias rather than a real finding. So who is at fault for this questionable claim being repeated without challenge in a popular venue many months after it has been convincingly challenged?

    For starters, Purdue is. They never responded to the criticism, despite confirmation that they are aware of it—for one thing, they got contacted by us and by Inside Higher Ed—and despite the fact that they apparently continue to make money off the sales of the product through a licensing deal with Ellucian. And the uncorrected paper is still available on their web site. This is unconscionable.

    Anya clearly bears some responsibility too. Although it’s easy to assume from the way the article is written that the dubious claim was repeated to her in an interview by Purdue research Matt Pistilli, she confirmed for me via email that she took the claim from the previously published research paper and did not discuss it with Pistilli. Given that this is her central example of the potential of learning analytics, she should have interrogated this a little more, particularly since she had Matt on the phone. Mike Caulfield also commented to me that any claim of such a dramatic increase in year-to-year retention should automatically be subject to additional scrutiny.

    I have to put some blame on the higher ed press as well. Inside Higher Ed covered the story (and, through them, the Times Higher Education). In fact, Carl Straumsheim actually advanced the story a bit by putting the question to researcher Matt Pistilli (who gave a non-answer). The Chronicle of Higher Education did not cover it, despite having run a puff piece on Purdue’s claims the same day that Mike Caulfield wrote his original piece challenging the results. It is very clear to Phil and me that we are read by the Chronicle staff, in part because they periodically publish stories that have been obviously influenced by our earlier coverage. Sometimes without attribution. I don’t care that much about the credit, but if they thought Purdue’s claims were newsworthy enough to cover in the first place then they should have done their own reporting on the fact that those claims have been called into question. If they had been more aggressive in their coverage then the mainstream press reporters who find Course Signals will be more likely to find the other side(s) of the story as well. Outside of IHE, I’m having trouble finding any coverage, never mind any original reporting, in the higher ed or ed tech press.

    I have a lot of respect for news reporters in general, and I think that most people grossly underestimate how hard the job is. I think highly of Anya as a professional. I like the reporters I interact with most at the Chronicle as well. Nor will I pretend that we are perfect here at e-Literate. We miss important angles and get details wrong our fair share. For example, I doubt that I would have caught the flaw in Purdue’s research if Mike hadn’t brought it to my attention. But collectively, we have to do a better job of providing critical coverage of topics like learning analytics, particularly at a time when so much money is being spent and our entire educational system is starting to be remade on the premise that this stuff will work. And there is absolutely no excuse whatsoever for a research university to not take responsibility for their published research on a topic that is so critical to the future of universities.

  • Efficacy Math is Hard

    David Wiley has a great post up on efficacy and OER in response to my original post about Pearson’s efficacy plan. He opens the piece by writing about Benjamin Bloom’s famous “2 sigma” problem:

    The problem isn’t that we don’t know how to drastically increasing learning. The two-part problem is that we don’t know how to drastically increase learning while holding cost constant. Many people have sought to create and publish “grand challenges” in education, but to my mind none will ever be more elegant than Bloom’s from 30 years ago:

    “If the research on the 2 sigma problem yields practical methods – which the average teacher or school faculty can learn in a brief period of time and use with little more cost or time than conventional instruction – it would be an educational contribution of the greatest magnitude.” (p. 6; emphasis in original)

    So the conversation can’t focus on efficacy only – if there were no other constraints, we actually know how to do “effective.” But there are other constraints to consider, and to limit our discussions to efficacy is to remain in the ethereal imaginary realm where cost doesn’t matter. And cost matters greatly.

    David then launches into a discussion of what he calls his “golden ratio,” or standard deviations per dollar. I have long been a fan of this formulation and quote it frequently. I’m not going to try to summarize his explication of it in his post; you really should go read it. But I would like to tease out a few implications here.

    (more…)