e-Literate

Present is Prologue

Tag: Anya Kamenetz

  • Is Standardized Testing a Pediatric Disease?

    In my last post, I wrote about the tension between learning, with the emphasis on the needs and progress of individual human learners, and education, which is the system by which we try to guarantee learning to all but which we often subvert in our well-meaning but misguided attempts to measure whether we are delivering that learning. I spent a lot of time in that post exploring research by Gallup regarding the workplace performance of adults, various dimensions of personal wellbeing, and the links of both to each other and to college experiences. One of Gallup’s findings were that workers who are disengaged with their work are less healthy. They are more likely to get clinically depressed, more likely to get heart conditions, and more likely to die young. I then made a connection between disengaged adults and disengaged students. What I left implicit was that if being disengaged as an adult is bad for one’s health, it stands to reason that being disengaged as a child is also bad for one’s health. We could be literally making our children sick with schooling.

    I am in the midst of reading Anya Kamenetz’s new book The Test. It has convinced me that I need to take some time making the connection explicit.

    (more…)

  • NPR and Missed (Course) Signals

    Anya Kamenetz has a piece up on NPR about learning analytics, highlighting Purdue’s Course Signals as its centerpiece. She does a good job of introducing the topic to a general audience and raising some relevant ethical questions. But she missed one of the biggest ethical questions surrounding Purdue’s product—namely, that some of its research claims are likely false. In particular, she repeats the following claim:

    Course Signals…has been shown to increase the number of students earning A’s and B’s and lower the number of D’s and F’s, and it significantly raises the chances that students will stick with college for an additional year, from 83% to 97%. [Emphasis added.]

    Based on the work of Mike Caulfield and Al Essa summarized in the link above, it looks like that latter claim is probably the result of selection bias rather than a real finding. So who is at fault for this questionable claim being repeated without challenge in a popular venue many months after it has been convincingly challenged?

    For starters, Purdue is. They never responded to the criticism, despite confirmation that they are aware of it—for one thing, they got contacted by us and by Inside Higher Ed—and despite the fact that they apparently continue to make money off the sales of the product through a licensing deal with Ellucian. And the uncorrected paper is still available on their web site. This is unconscionable.

    Anya clearly bears some responsibility too. Although it’s easy to assume from the way the article is written that the dubious claim was repeated to her in an interview by Purdue research Matt Pistilli, she confirmed for me via email that she took the claim from the previously published research paper and did not discuss it with Pistilli. Given that this is her central example of the potential of learning analytics, she should have interrogated this a little more, particularly since she had Matt on the phone. Mike Caulfield also commented to me that any claim of such a dramatic increase in year-to-year retention should automatically be subject to additional scrutiny.

    I have to put some blame on the higher ed press as well. Inside Higher Ed covered the story (and, through them, the Times Higher Education). In fact, Carl Straumsheim actually advanced the story a bit by putting the question to researcher Matt Pistilli (who gave a non-answer). The Chronicle of Higher Education did not cover it, despite having run a puff piece on Purdue’s claims the same day that Mike Caulfield wrote his original piece challenging the results. It is very clear to Phil and me that we are read by the Chronicle staff, in part because they periodically publish stories that have been obviously influenced by our earlier coverage. Sometimes without attribution. I don’t care that much about the credit, but if they thought Purdue’s claims were newsworthy enough to cover in the first place then they should have done their own reporting on the fact that those claims have been called into question. If they had been more aggressive in their coverage then the mainstream press reporters who find Course Signals will be more likely to find the other side(s) of the story as well. Outside of IHE, I’m having trouble finding any coverage, never mind any original reporting, in the higher ed or ed tech press.

    I have a lot of respect for news reporters in general, and I think that most people grossly underestimate how hard the job is. I think highly of Anya as a professional. I like the reporters I interact with most at the Chronicle as well. Nor will I pretend that we are perfect here at e-Literate. We miss important angles and get details wrong our fair share. For example, I doubt that I would have caught the flaw in Purdue’s research if Mike hadn’t brought it to my attention. But collectively, we have to do a better job of providing critical coverage of topics like learning analytics, particularly at a time when so much money is being spent and our entire educational system is starting to be remade on the premise that this stuff will work. And there is absolutely no excuse whatsoever for a research university to not take responsibility for their published research on a topic that is so critical to the future of universities.

  • Changing the Narrative

    As Phil mentioned, he and I were both lucky to attend the MOOC Research Initiative conference, which was a real tour de force. Jim Groom observed that even the famously curmudgeonly Stephen Downes appeared to be enjoying himself, and I would make a similar observation about the famously curmudgeonly Jonathan Rees. If both of those guys can be simultaneously (relatively) pleased at a MOOC conference, then something is going either spectacularly right or horribly wrong. I believe it was the former in this case.

    We are at one of those rare moments when there’s enough confusion that real conversation happens and possibilities open up. The sense I got is that everybody is really grappling with the questions of where we can take the concept of a “MOOC” and what MOOCishness might be good for. That is fun and hope-inducing. Phil and I spent a lot of the time interviewing folks for a future e-Literate TV series (coming to a computing device near you in March or April of 2014), so we were lucky to hear a lot of perspectives. There is some very good exploration happening now. George Siemens and his fellow conference organizers (as well as the Bill and Melinda Gates Foundation, which sponsored the event and the research) did a real service by bringing people together to talk about these issues at this pregnant moment.

    One thing happened toward the end of the conference that has me puzzled, though. Jim mentioned it in his blog post:

    At the same time[,] Bon Stewart’s admonitions for some kind of organized response to start filling the temporary void of direction with alternative narrative still rings in my ears—and it is very much the lesson I took away from Audrey Watters keynote at OpenEd.

    There was a lot of conversation, really throughout the conference but coming to a head at the end, that the term of MOOC is somehow damaged goods and that…something…should be done about it. Usually the word “narrative” was brought up. But this talk of “alternative narratives” or, as Bonnie put it, “changing the narrative”, confuses me. As far as I’m concerned, the connectivist/open ed crowd has been spectacularly, stunningly successful at “changing the narrative,” and I’m not at all clear what it would look like to somehow do it differently. I don’t understand what they mean here. Unfortunately I had to rush out the door to try to catch a plane shortly after the panel discussion and didn’t have an opportunity to follow up with some of the attendees. So I’m going to try to express my confusion in this blog post and hope that somebody can help me figure out what I’m missing.

    Warning: This post is long and lit crit wonkish.

    (more…)

  • Why Higher Education Is In Trouble–In One Graph

    Here’s a graph from a recent presentation by Anya Kamenetz:

    U.S. Tuition Costs Over Time

    The word that comes to mind is “Yikes!”

    Here’s the full preso:

  • Sakai Conference: Kamenetz Keynote

    OK, this was worth the wait. I have video of Anya Kamenetz’s keynote, which set the tone for the Sakai Conference 2010 in some important ways. I also have a short video interview with her, some related video content from Dan Pink, and of course, analysis of what all this means for educational technology in general and for the design of Sakai 3 in particular.

    Be warned: It will take you over 90 minutes to consume all the content from this post. Not counting post-consumption rumination. ((Did I or did I not warn you about impending blogorrhea? I believe I did.))

    (more…)

  • DIY U: The Modern Guild at Work

    I’m waiting for the video of Anya Kamenetz’s keynote to be available online before I kick off my post series on the Sakai conference. In the meantime, here’s a quick update on a previous (and Kamenetz-related) post. A while back, I suggested that a modern variant on the guild approach could pull apprentices straight out of high school and train them in a craft while getting them started in a career directly. I also speculated that the software industry would be a good candidate for helping such a career path become socially acceptable for students looking to get into white collar jobs:

    [W]ould a young person who is already from a relatively high economic bracket consider this guild system to be an acceptable career path? Would the guild path be accepted as a substitute for four years of full-time college study by middle-class students and their parents? It would take some work, but I believe it could be possible. Software development is one example of an industry that might be a good pioneer of this approach. If, say, Microsoft or Google were to take students out of high school to become paid employees and put them on an apprenticeship path where they would be able to earn their degrees over time at lower cost while earning good salaries and becoming shareholders in the company, this approach could become acceptable in a hurry.

    Interestingly, it turns out that Zoho has a highly successful apprenticeship program at their India Development Center. Here’s an interview about it with CEO Sridhar Vembu that I found via this O’Reilly Radar post:

    Basically, they’re taking students from the 85% to 90% of Indian high school students who would be considered poor by global standards, supplying them with some mentoring and open educational resources (translated into their native Tamil language), and paying them to go to a nine- to twelve-month crash course on how to become programmers. The students are highly successful on average and can aspire to high positions within the company. Many of them also go on to get college degrees while working, but many don’t. It never occurred to me that India would be an ideal seed bed for this approach, but it’s obvious in retrospect. There’s a huge potential workforce, including many talented young people who don’t have access to conventional educational paths. It also strikes me as only a matter of time before some enterprising university strikes a deal with Zoho to create a Walmart U-style deal where students get college credit for their apprenticeship and are tracked to a continuing degree program should they want to do so.

    There’s a lot more that’s interesting in this interview, including reflections on why such a program would be harder to start with poor U.S. neighborhoods, the challenges and secrets to the program’s success, and so on. The whole interview is well worth your time.

  • DIY U: Digital Apprenticeship and the Modern Guild

    A while back, I had the pleasure of chatting with Paul Lefrere. Paul is a guy who thinks about large-scale questions like, “How can we create and fill a couple of million green jobs in a country within a couple of years?” One problem he was grappling with regarding higher education was time-to-market for students. He needed to get them working and productive in less than four years—and possibly less than two. One idea he talked about was what he called “digital apprenticeship.” Imagine a solar panel installation trainee wearing a helmet with a wireless webcam on it. You could have an expert watching her work virtually, either coaching her through the steps in a process or testing her by watching her perform the process and then certifying that she’s done it right. She could be both a productive worker and a student on a long-term educational path on day one.

    More recently, Jim Groom suggested that apprenticeship might be a good model for networked learning, particularly for those students who are not auto-didactic. ((It’s worth noting that, while we tend to refer to “auto-didacts” as if there is a special class of people who can learn everything on their own, the truth of the matter is that many people can be auto-didactic in some subjects and not in others. For example, I’m pretty good at teaching myself many kinds of book learning but am not good at all when it comes to figuring out how to, say, fix a squeaky floor board or replace the switch in my hallway light, even with detailed instructions. And I have given up all hope of ever teaching myself how to juggle. I could learn to do all of these things, but I would need coaching, because I’m not good at noticing what I’m doing wrong—or sometimes even whether I’m doing wrong—in these areas.)) I think that Jim and Paul are both really onto something. Apprenticeship is a model that is well understood, can work for a variety of topics in less formal settings, can get learners productive on certain tasks quickly, and can get the benefits of scale by using modern communications technologies. At the same time, it’s one thing to have a model that works well in theory and quite another to have a set of practices that are likely to be adopted widely under given social and economic conditions. So in this post, I’m going to consider the question of what it would take to embed apprenticeship into the fabric of our society as a broadly accepted career path across a wide range of career types.

    (more…)