e-Literate

Present is Prologue

Tag: educational research

  • Lost in Translation: Why it’s important to consult the original research

    Lost in Translation: Why it’s important to consult the original research

    Last week the Brookings Institute published an article in its Economic Studies: Evidence Speaks series titled “Promises and pitfalls in online learning,” by Eric Bettinger and Susanna Loeb of Stanford University. The article is a condensed summary of a lengthier draft research paper, “Changing Distributions: How Online College Classes Alter Student and Professor Performance” that was supported by the Stanford Center for Education and Policy Analysis (CEPA) and includes two additional authors, Lindsay Fox (Stanford) and Eric Taylor (Harvard). The Brookings’ article was first reported by Inside Higher Ed earlier this week in a piece called “Is Online Ed Missing the Mark?”

    I note the cascading levels of publication to draw attention to the occasional pitfalls of relying on reporting and journal summaries and the benefits of reading the original research. It reminds me a little bit of the time I spent working for a school board member in the L.A. Unified School District. An article would sometimes appear in a local paper, and what was reported would have little to do with the reality of what was actually being covered, or at least miss a lot of the complexity (that said, there was excellent reporting as well).

    (more…)

  • Giving D2L Credit Where Credit Is Due

    Michael and I have made several specific criticisms of D2L’s marketing claims lately culminating in this blog post about examples based on work at the University of Wisconsin-Milwaukee (UWM) and California State University at Long Beach (CSULB).

    I understand that other ed tech vendors make marketing claims that cannot always be tied to reality, but these examples cross a line. They misuse and misrepresent academic outcomes data – whether public research-based on internal research – and essentially take credit for their technology “delivering results”.

    This week brought welcome updates from D2L that go a long way towards addressing the issues we raised. As of Monday, I noticed that the ‘Why Brightspace? Results’ page now has links to supporting material for each claim, and the UWM claim has been reworded. Today, D2L released a blog post explaining these changes and admitting the mistakes. D2L even changed the web page to allow text selection for copy / paste. From the blog post: (more…)

  • D2L Again Misusing Academic Data For Brightspace Marketing Claims

    Update 5/17/16: I made a mistake in my math on the UW Milwaukee improvements. The number of A’s and B’s increased 163% increased 220% for “unprepared” students and 170% for “prepared” students. I apologize for the error.

    Update 7/23: Read this blog post for D2L admission of mistakes and changes to claims.

    At this point I’d say that we have established a pattern of behavior.

    Michael and I have been quite critical of D2L and their pattern of marketing behavior that is misleading and harmful to the ed tech community. Michael put it best:

    I can’t remember the last time I read one of D2L’s announcements without rolling my eyes. I used to have respect for the company, but now I have to make a conscious effort not to dismiss any of their pronouncements out-of-hand. Not because I think it’s impossible that they might be doing good work, but because they force me to dive into a mountain of horseshit in the hopes of finding a nugget of gold at the bottom. Every. Single. Time. I’m not sure how much of the problem is that they have decided that they need to be disingenuous because they are under threat from Instructure or under pressure from investors and how much of it is that they are genuinely deluding themselves. Sadly, there have been some signs that at least part of the problem is the latter situation, which is a lot harder to fix. But there is also a fundamental dishonesty in the way that these statistics have been presented.

    Well, here’s the latest. John Baker put out a blog called This Isn’t Your Dad’s Distance Learning Program with this theme: (more…)

  • Pearson, Efficacy, and Research

    A while back, I mentioned that MindWires, the consulting company that Phil and I run, had been hired by Pearson in response to a post I wrote a while back expressing concerns about the possibility of the company trying to define “efficacy” in education for educators (or to them) rather than with them. The heart of the engagement was us facilitating conversations with different groups of educators about how they think about learning outcomes—how they define them, how they know whether students are achieving them, how the institution does or doesn’t support achieving them, and so on. As a rule, we don’t blog about our consulting work here on e-Literate. But since we think these conversations have broader implications for education, we asked for and received permission to blog about what we learn under the following conditions:

    • The blogging is not part of the paid engagement. We are not obliged to blog about anything in particular or, for that matter, to blog at all.
    • Pearson has no editorial input or prior review of anything we write.
    • If we write about specific schools or academics who participated in the discussions, we will seek their permission before blogging about them.

    I honestly wasn’t sure what, if anything, would come out of these conversations that would be worth blogging about. But we got some interesting feedback. It seems to me that the aspect I’d like to cover in this post has implications not only for Pearson, and not only for ed tech vendors in general, but for open education and maybe for the future of education in general. It certainly is relevant to my recent post about why the LMS is the way it is and the follow-up post about fostering better campus conversations. It’s about the role of research in educational product design. It’s also about the relationship of faculty to the scholarship of teaching.

    (more…)