Back around New Year, Michael wrote a post examining Pearson’s efficacy initiative and calling on the company to engage in active discussions with various communities within higher education about defining “efficacy” with educators rather than for educators. It turns out that post got a fair bit of attention within the company. It was circulated in a company-wide email from CEO John Fallon, and the blog post and all the comments were required reading for portions of the company leadership. After a series of discussions with the company, we, through our consulting company, have been hired by Pearson to facilitate a few of these conversations. We also asked for and received permission to blog about them. Since this is an exception to our rule that we don’t blog about our paid engagements, we want to tell you a little more about the engagement, our rationale for blogging about it, and the ground rules.
Tag: Pearson-PLC
-
Blackboard’s Perceptis Acquisition Offers Clues into Company’s Strategy
Yesterday Blackboard announced that they acquired Perceptis, a provider of help desk and financial aid support services for colleges and universities. In and of itself, this is not a huge acquisition. Perceptis has 33 clients, offers services that Blackboard was already offering, and has no substantial new technology. But as we approach BbWorld next week, the move provides some early hints into the strategic direction that the company may highlight at the conference.
I had the opportunity to talk with Blackboard’s Vice President of Education Services Katie Blot about the move.
-
Efficacy, Adaptive Learning, and the Flipped Classroom, Part II
In my last post, I described positive but mixed results of an effort by MSU’s psychology department to flip and blend their classroom:
- On the 30-item comprehensive exam, students in the redesigned sections performed significantly better (84% improvement) compared to the traditional comparison group (54% improvement).
- Students in the redesigned course demonstrated significantly more improvement from pre to post on the 50-item comprehensive exam (62% improvement) compared to the traditional sections (37% improvement).
- Attendance improved substantially in the redesigned section. (Fall 2011 traditional mean percent attendance = 75% versus fall 2012 redesign mean percent attendance = 83%)
- They did not get a statistically significant improvement in the number of failures and withdrawals, which was one of the main goals of the redesign, although they note that “it does appear that the distribution of A’s, B’s, and C’s shifted such that in the redesign, there were more A’s and B’s and fewer C’s compared to the traditional course.”
- In terms of cost reduction, while they fell short of their 17.8% goal, they did achieve a 10% drop in the cost of the course….
It’s also worth noting that MSU expected to increase enrollment by 72 students annually but actually saw a decline of enrollment by 126 students, which impacted their ability to deliver decreased costs to the institution.
Those numbers were based on the NCAT report that was written up after the first semester of the redesigned course. But that wasn’t the whole story. It turns out that, after several semesters of offering the course, MSU was able to improve their DFW numbers after all:
That’s a fairly substantial reduction. In addition, their enrollment numbers have returned to roughly what they were pre-redesign (although they haven’t yet achieved the enrollment increases they originally hoped for).When I asked Danae Hudson, one of the leads on the project, why she thought it took time to see these results, here’s what she had to say:
I do think there is a period of time (about a full year) where students (and other faculty) are getting used to a redesigned course. In that first year, there are a few things going on 1) students/and other faculty are hearing about “a fancy new course” – this makes some people skeptical, especially if that message is coming from administration; 2) students realize that there are now a much higher set of expectations and requirements, and have all of their friends saying “I didn’t have to do any of that!” — this makes them bitter; 3) during that first year, you are still working out some technological glitches and fine tuning the course. We have always been very open with our students about the process of redesign and letting them know we value their feedback. There is a risk to that approach though, in that it gives students a license to really complain, with the assumption that the faculty team “doesn’t know what they are doing”. So, we dealt with that, and I would probably do it again, because I do really value the input from students.
I feel that we have now reached a point (2 years in) where most students at MSU don’t remember the course taught any other way and now the conversations are more about “what a cool course it is etc”.
Finally, one other thought regarding the slight drop in enrollment we had. While I certainly think a “new blended course” may have scared some students away that first year, the other thing that happened was there were some scheduling issues that I didn’t initially think about. For example, in the Fall of 2012 we had 5 sections and in an attempt to make them very consistent and minimize missed classes due to holidays, we scheduled all sections on either a Tuesday or a Wednesday. I didn’t think about how that lack of flexibility could impact enrollment (which I think it did). So now, we are careful to offer sections (Monday through Thursday) and in morning and afternoon.
To sum up, she thinks there were three main factors: (1) it took time to get the design right and the technology working optimally; (2) there was a shift in cultural expectations on campus that took several semesters; and (3) there was some noise in the data due to scheduling glitches.
There are a number of lessons one could draw from this story, but from the perspective of educational efficacy, I think it underlines how little the headlines (or advertisements) we get really tell us, particularly about components of a larger educational intervention. We could have read, “Pearson’s MyPsychLabs Course Substantially Increased Students Knowledge, Study Shows.” That would have been true, but we have little idea how much improvement there would have been had the course not been fairly radically redesigned at the same time. We also could have read, “Pearson’s MyPsychLabs Course Did Not Improve Pass and Completion Rates, Study Shows.” That would have been true, but it would have told us nothing about the substantial gains over the semesters following the study. We want talking about educational efficacy to be like talking about the efficacy of Advil for treating arthritis. But it’s closer to talking about the efficacy of various chemotherapy drugs for treating a particular cancer. And we’re really really bad at talking about that kind of efficacy. I think we have our work cut out for us if we really want to be able to talk intelligently and intelligibly about the effectiveness of any particular educational intervention.
-
Efficacy Math is Hard
David Wiley has a great post up on efficacy and OER in response to my original post about Pearson’s efficacy plan. He opens the piece by writing about Benjamin Bloom’s famous “2 sigma” problem:
The problem isn’t that we don’t know how to drastically increasing learning. The two-part problem is that we don’t know how to drastically increase learning while holding cost constant. Many people have sought to create and publish “grand challenges” in education, but to my mind none will ever be more elegant than Bloom’s from 30 years ago:
“If the research on the 2 sigma problem yields practical methods – which the average teacher or school faculty can learn in a brief period of time and use with little more cost or time than conventional instruction – it would be an educational contribution of the greatest magnitude.” (p. 6; emphasis in original)
So the conversation can’t focus on efficacy only – if there were no other constraints, we actually know how to do “effective.” But there are other constraints to consider, and to limit our discussions to efficacy is to remain in the ethereal imaginary realm where cost doesn’t matter. And cost matters greatly.
David then launches into a discussion of what he calls his “golden ratio,” or standard deviations per dollar. I have long been a fan of this formulation and quote it frequently. I’m not going to try to summarize his explication of it in his post; you really should go read it. But I would like to tease out a few implications here.
-
New e-Literate TV Episode: Adaptive Learning and Learning Analytics
In our latest episode, the penultimate in the pilot series, we explore the topics that are likely to be moving up the curve of the hype cycle this year—adaptive learning and learning analytics. Like many of the topics in the pilot series, we could have made an entire series about this one. (And maybe we will at some point.) But since the first adaptive learning product faculty will run into is most likely to be from a textbook publisher, we interviewed McGraw Hill’s Al Essa and Pearson’s Jason Jordan about their respective takes on what this trend is all about.
-
An Employee View of Textbook Publishers and Ed Tech Companies
Update: For reference, 2U has pointed out to me that they were recently recognized by Glassdoor as the 17th-best medium-sized company to work for in 2014. With an average score of 4.2 (based on 21 reviews), this puts them on top of the other ed tech companies discussed here. The only other education company on the Glassdoor best places lists is Edmentum which, oddly, only has a 3.1 average rating based on 37 reviews.
Second Update: All the education companies that have scored well on Glassdoor are coming out of the woodwork now. W.W. Norton has pointed out to me that the have a 4.2 average rating, based on 47 reviews. That puts them tied for first among education companies (that I know of) with 2U.
One way to get a sense of how the companies whose products you depend on are doing is to talk to their employees. This is particularly helpful at times like this, when huge changes are underway. So I thought it would be interesting to take a survey of the employee reviews on Glassdoor of some of the major companies in the industry. The results are revealing, if not entirely surprising.
-
Lessons from the Boundless Copyright Infringement Suit
On December 17th, the Boundless OER-based textbook startup issued a press release describing the settlement they had reached with Pearson, Cengage, and Macmillan in the lawsuit those three companies had filed against the company. (Full disclosure: Pearson has been a client of MindWires Consulting.) Actually, a lot of the press release wasn’t really about the lawsuit, and the description of the settlement consisted of the following:
Today, we’re excited to announce that we’ve settled the lawsuit. In agreeing to a confidential settlement agreement, along with a public judgment and injunction entered by the Court, the parties have resolved the dispute. The resolution allows the parties to move forward and focus on their mutually shared goal of helping students learn. Boundless now has a clear path for building and marketing its OER-driven textbook alternatives without treading upon the Plaintiffs’ rights, and it is confident that it is in compliance and will not have further legal issues with the Plaintiff publishers. In turn, Plaintiffs have reinforced the strong protection they have in and to their copyrighted works and the related goodwill that they and their authors have established, and look forward to Boundless operating its business within the agreed upon framework.
This seemed like a strangely muted ending to a strange story. It’s hard to tell from the press release what actually happened. But having read the consent decree and injunction, I have come to two conclusions. First, Boundless lost. Second, the suit and its outcome tell us very little about the future of OER but rather more about business strategy for ed tech startups.