e-Literate

Present is Prologue

Tag: The Chronicle of Higher Education

  • Recommended Reading: Article in Chronicle on Measuring Learning

    Dan Barrett’s piece over the weekend in The Chronicle, “The Next Great Hope for Measuring Learning,” deserves a close read. He describes in some detail a ground up effort by faculty and administrators across several institutions to define and measure what it is that students are learning and why it’s important. In doing so, these faculty and administrators are moving beyond looking simply at content mastery and focusing on broader skills of quantitative reasoning, writing and critical thinking. It’s forcing them to develop new approaches to reviewing student work, moving away from the idea of “grading” and toward “scoring” against a rubric that looks more sophisticated learning outcomes.

    The effort, directed by the Association of American Colleges and Universities (AAC&U) – an active advocate in this space (see Michael’s earlier blog post, “Outcomes-Based Education and the Conservative Radicalism of the AAC&U”), is not without its skeptics, including many participants in the actual project. That said, the approach is being watched closely as a way for higher education to define and measure appropriate criteria that lead to improvements in teaching and learning while simultaneously speaking to accountability issues around which there is so much attention.

    Like many promising initiatives in higher education recently highlighted by e-Literate, well thought out, collaboratively developed homegrown solutions usually trump those imposed by third parties working in isolation.

    Note: For those without a Chronicle subscription, use this link to get behind the firewall for 24 hours after this post publishes.

  • A Moment of Clarity on the Role of Technology in Teaching

    This following excerpt is based on a post first published at The Chronicle of Higher Education.

    With all of the discussion around the role of online education for traditional colleges and universities, over the past month we have seen reminders that key concerns are about people and pedagogy, not technology. And we can thank two elite universities that don’t have large online populations — MIT and George Washington University — for this clarity.

    On April 1, the MIT Online Education Policy Initiative released its report,“Online Education: A Catalyst for Higher Education Reforms.” The Carnegie Corporation-funded group was created in mid-2014, immediately after an earlier initiative looked at the future of online education at MIT. The group’s charter emphasized a broader policy perspective, however, exploring “teaching pedagogy and efficacy, institutional business models, and global educational engagement strategies.”

    While it would be easy to lament that this report comes from a university with few online students and yet dives into how online learning fits in higher education, it would be a mistake to dismiss the report itself. This lack of “in the trenches” experience with for-credit online education helps explain the report’s overemphasis on MOOCs and its underemphasis on access and nontraditional learner support. Still, the MIT group did an excellent job of getting to some critical questions that higher-education institutions need to address. Chief among them is the opportunity to use online tools and approaches to instrument and enable enhanced teaching approaches that aren’t usually possible in traditional classrooms. (more…)

  • We Need a More Robust Learning Sciences Research Community

    My latest Chronicle column is on how inherently difficult it is to evaluate learning science claims, particularly when they get boiled down to marketing claims about product efficacy, and how deep academic distrust of vendors makes this already incredibly difficult challenge nearly impossible.

    Here’s where I stand on vendor participation in ed tech and learning science research:

    On the one hand, vendors have access to data and resources that, for a variety of reasons, are difficult or even impossible for universities to access. They sometimes have millions of students using their products and relatively few internal barriers to doing certain kinds of large-scale effectiveness research (although to a certain degree, territoriality, bureaucracy, and poorly designed data architectures are universal problems). On the other hand, vendors should not under any circumstances be allowed to define the research agenda and arbitrate the validity of learning sciences claims for all of education. Theoretically, there is a simple solution to this. Higher ed knows how to do science. It knows how to review scientific claims. Vendor research should be put through the peer review process. They should publish enough information that their results can be evaluated and, when feasible, duplicated. Their reputations should be based in part on who is doing (or, at least, properly using) good scientific research. After all, most of the researchers at these companies are PhDs who are trained in the academic research process. There is no reason in the world why these companies can’t contribute to real, rigorous, peer-reviewed dialog and progress in the field. (more…)

  • The Odd Couple: How Ed Tech Must Support Vastly Different Types of Professors

    An edited version of this post was first published at The Chronicle of Higher Education

    Let’s admit it, there can be some real tension when a college is faced with choosing a new learning-management system, or any software used by more than one department.

    Since the decision involves the administrators who will support the system — commonly called an LMS — and professors who will use it, who should lead the process? Should staff members just get input from faculty members, or should professors vote on the final decision? Or should professors run the process?

    This is when distrust between the two sides can emerge. IT administrators may fear that professors will make the pace unbearably slow with an overly deliberative approach. And professors may assume that IT has already determined what system to buy, and may not take their input seriously. Or they may worry that the meetings will be dominated by those needy power users from the psychology department.

    (more…)

  • UNC Learning Technology Commons: Easing the procurement problem with NGDLE

    I was planning to write a descriptive post about the new UNC Learning Technology Commons, but there is already some excellent coverage. UNC’s Matthew Rascoff wrote a blog post on Medium that captures the basics quite well:

    A compelling recent report from EDUCAUSE proposes that the “Next Generation Digital Learning Environment” will be based on a “‘Lego’ approach,” in which “components … allow individuals and institutions the opportunity to construct learning environments tailored to their requirements and goals.” The authors, Malcolm Brown, Joanne Dehoney, and Nancy Millichap, envision an app-like ecosystem of interoperable ed tech tools, each of which does one thing well, rather than the monolithic collection of functionality that is the learning management system.

    Today, with the launch of the UNC Learning Technology Commons, the University of North Carolina system is taking a step in the direction of this more modular, flexible, and learner- and educator-centered approach.

    We’re reimagining the way university faculty and staff buy the ed tech they need to help their students learn. Our goal is to empower educators with the best instructional tools available — in the classroom or online.

    The UNC Learning Technology Commons is a system-wide effort to curate an annotated catalogue of digital learning products available for accelerated purchase by the 20,000 faculty members of the UNC system, and to build a community of educators who share (anonymized, aggregated) learning outcomes and user experiences with those products.

    (more…)

  • Empowering Students in Open Research

    Phil and I will be writing a twice-monthly column for the Chronicle’s new Re:Learning section. In my inaugural column, “Muy Loco Parentis,” I write about how schools make data privacy decisions on behalf of the students that the students wouldn’t make for themselves, and that may even be net harmful for the students. In contrast to the ways in which other campus policies have evolved, there is still very much a default paternalistic position regarding data.

    But the one example that I didn’t cover in my piece happens to be the one that inspired it in the first place. A few months back at the OpenEd conference, I heard a presentation from CMU’s Norm Bier about that challenges of getting different schools to submit OLI student data to a common database for academic research. Basically, every school that wants to do this has to go through its own IRB process, and every IRB is different. Since the faculty using the OLI products usually aren’t engaged in the research themselves, it generally isn’t worth the hassle to go through this process, so the data doesn’t get submitted and the research doesn’t get done. Note that Pearson and McGraw Hill do not have this problem; if they want to look at student performance in a learning application across various schools, they can. Easily. Something is wrong with this picture. I proposed in Norm’s session that maybe students could be given an option to openly publish their data. Maybe that would get around the restrictions. David Wiley, who does a lot more academic research than I do, seemed to think this wasn’t a crazy idea, so I’ve been gnawing on the problem since then.

    I have talked to a bunch of researchers about the idea. The first reaction is often skepticism. IRB is not so easy to circumvent (for good reason). What generally changed their minds was the following thought experiment:

    • Suppose that, in some educational software program, there was a button labeled “Export.” Students could click the button and export their data in some suitably anonymized format. (Yes, yes, it is impossible to fully de-identify data, but let’s posit “reasonably anonymized” as assessed by a community of data scientists.) Would giving students the option to export their data to any server of their choosing trigger the requirement for IRB review? [Answer: No.]
    • Suppose the export button offered a choice to export to CMU’s research server. Would giving students that option trigger the requirement for IRB review? [Answer: Probably not.]

    There are two shades of gray here that are complications. First, researchers worry about the data bias that comes from opt in. And the further you lead students down the path toward encouraging them to share their data, such as making sharing the default, the more the uneasiness sets in. Second and relatedly, there is the issue of informed consent. There was a general feeling that, even if you get around IRB review, there is still a strong ethical obligation to do more than just pay lip service to informed consent. You need to really educate students on the potential consequences of sharing their data.

    That’s all fair. I don’t claim that there is a silver bullet. But the thought experiment is revealing. Our intuitions, and therefore our policies, about student data privacy are strongly paternalistic in an academic context but shift pretty quickly once the institutional role fades and the student’s individual choice is foregrounded. I think this is an idea worth exploring further.

  • Student Course Evaluations and Impact on Active Learning

    The Chronicle has an article out today, “Can the Student Course Evaluation Be Redeemed?”, that rightly points out how student course evaluations are often counter-productive to improving teaching and learning. The article refers to a Stanford professor’s call for an instructor completed “inventory of the research-based teaching practices they use”, but most of the article centers on revised course evaluation tool from a Kansas State University spin-off (the IDEA Center). One of the key problems described is that “administrators often take their results as numerical gospel” as well as faculty misapplying the results.

    However they’re used, a lot of course evaluations simply aren’t very good, [IDEA president] Mr. Ryalls says.

    But as flawed as they are, faculty members still turn to them as some gauge of effectiveness in the classroom. About three-quarters of instructors use formal evaluations and informal feedback “quite a bit” or “very much” when altering their courses, according to the Faculty Survey of Student Engagement.

    One limitation of many tools is that they ask students things they don’t really know. A frequent example: Was your instructor knowledgeable about course content?

    There is one additional problem with most student course evaluations that is not explicitly covered in the Chronicle articles – students newly involved in active learning approaches often rate the course and instructor poorly even if they end up learning more effectively. We saw this in our e-Literate TV case study at UC Davis. In a previous post we highlighted how the routine hard work required of students in active learning courses can lead to poor evaluations, but later in the interview student course evaluations came up as a major barrier to improving teaching practices. (more…)