e-Literate

Present is Prologue

Category: Ed Tech

The “Ed Tech” category includes posts about educational technology products themselves, including LMSs and other learning platforms, adaptive learning and other digital curricular materials products, learning analytics, and educational apps of all types. It also includes technical aspects of ed tech products, especially interoperability.

  • Web3 in Education Isn’t Very Interesting

    Web3 in Education Isn’t Very Interesting

    It’s been a while since I’ve written much on e-Literate. Sorry about that. I’ve been super busy with my new start-up, Argos Education. I’ll be doing more again soon; here, on the Argos site, on Blursdays, and so on. I thought I’d start today with a warm-up post.

    Rebecca Koenig published a piece in EdSurge a few weeks about the potential for Web3 in higher education. Koenig is a decent journalist who gamely tries to tackle a genuinely confusing topic. Something about decentralization and taking the internet back from big corporations (while possibly giving it to other big corporations) and stuff like that.

    I don’t claim to have clarity or answers about Web3 yet but I do have two hypotheses:

    1. Blockchain, the technology upon which Web3 is premised, doesn’t add a whole lot to the ability to decentralize education in ways that were not possible before.
    2. In any event, the real challenge with decentralization isn’t the technology. It’s the humans. Blockchain doesn’t really help there in any way that I can see.

    What the heck is Web3?

    To be completely honest, I don’t know and I’m not convinced anybody else does either.

    At the core of it is something called blockchain, which is the technology that underlies cryptocurrencies like Bitcoin. I won’t try to reproduce the many good explanations of the technology that are available elsewhere, but the key benefits are (1) it provides a transparent chain of custody for digital artifacts and (2) that chain of custody is part of what enables the recipient to be confident that they’re getting what they’re told they’re getting and that it hasn’t been tampered with.

    Beyond that, there’s a lot of hype about decentralization of…um…everything? A lot of the interest in web3 (and in education, as Koenig covers in her piece), is around something called Decentralized Autonomous Organizations (DAOs). DAOs are often painted as the ultimate culmination of the gig economy in which everybody works for themselves, forming virtual organizations as needed.

    No, I’m not sure that I get how blockchain makes that possible either. But we’ll return to the potential usefulness of this theoretical affordance momentarily.

    Another idea that gets a lot less attention but may be more useful in EdTech is the notion of “self-sovereign identity,” The basic notion is that you can control your digital identity (or identities) and not have to rely on “log in with Facebook.” Which, hey, I’m all for. The only meaningful Web3 use case that I’ve seen actually implemented in education so far is ASU’s reverse transfer effort, which uses self-sovereign identity. (Again, I won’t get into the details here.)

    Whenever a hyped concept like Web3 hits, I try hard not let myself get either too spun up or too turned off by the hype itself. Instead, I try to imagine what the thing could be good for and test out the idea in my mind.

    So far, the answer I’ve come up with for Web3 is “ummm…?.” As far as I can tell, Web3 solves the wrong problems for education. I could easily be missing something. It wouldn’t be the first time and it won’t be the last. But none of the ideas I’ve heard so far make much sense.

    Coase vs. Conway

    If we want to think about humans working together in decentralized ways, it helps to look about both existing theory and existing practice. Let’s start with the theory.

    Why do organizations exist? The leading theorist on this question is economist and Nobel laureate Ronald Coase. In two seminal articles—”The Nature of the Firm” and “The Problem of Social Cost”—Coase argued that companies (and other organizations) exist because of transaction costs between actors, including the cost of planning, deciding, changing plans, resolving disputes, and so on. If we could do all the things that organizations do to…um…organize at low enough cost without employees, then there would be no organizations. For example, companies exist when it’s cheaper to create a corporation, hire and pay employees, etc., than it is to coordinate among a bunch of freelancers all working together.

    The idea of lowering transaction costs to eliminate command-and-control structures is hardly new. In 1884, Karl Marx’s collaborator Friedrich Engels wrote,

    The society which organizes production anew on the basis of free and equal association of the producers will put the whole state machinery where it will then belong—into the museum of antiquities, next to the spinning wheel and the bronze ax.

    Friedrich Engels, Origins of Family, Private Property, and the State

    And of course, the idea of collectives or communes is considerably older than that. Organizations, after all, are human inventions. Humans existed before formal organizations were invented and humans have continued to collaborate without them in many different situations.

    Technology can lower transaction costs and makes new kinds of organization possible. In his 2002 paper “Coase’s Penguin: Linux and The Nature of the Firm,” Yochai Benkler argued that open-source software has been made possible by (1) free digital copying of goods (e.g., source code) and (2) the lowering of coordination costs made possible by electronic communications. More recently, we’ve seen the rise of the gig economy, with gig jobs ranging from Uber drivers to social media influencers, enabled by technology.

    The question is, where’s the limit? What sorts of organizations can be broken down by technology-enabled lowering of transaction costs and what sorts can’t? How does that apply to education?

    Before we get to some practical answers, it’s worth looking at the other side of the coin from Coase. Far less well known than Coase’s Theorem is Conway’s Law. It’s been expressed differently at different times, but its simplest formulation is this:

    Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.

    Melvin Conway, “How Do Committees Invent?”

    Conway is a software engineer. He observed that, for example, when an organization had three different departments all working on parts of the same piece of software, the software would end up being developed in three modules. And the quality of the communication among the modules would be similar to the quality of the communication among the three groups.

    Note that this is within an organization. Even the transaction costs within an organization are non-zero. Planning is hard. Coordination is hard. Human behavior is hard. Technology can help sometimes but, fundamentally, this is a biological thing. Clever monkeys with blockchain are still just clever monkeys.

    So how much can technology facilitate some kind of decentralized education? Also, why is that a goal?

    Coase’s graduation gown vs Conway’s learning object repository

    To get at the first question, let’s look at the examples from Koenig’s article.

    First up is a quote from Vriti Saraf, founder of a startup called k20 Educators:

    “Universities are curators of content,” Saraf says. In the past, she explains, in order to get a Harvard education, “I had to go through Harvard; I couldn’t just go straight to a professor.” But in the future, she predicts, “Harvard could still be curating classes of professors, but it’s not Harvard’s intellectual property.”

    What Could Web3 Mean for Education?

    First of all, this is not very radical. Harvard would still exist. Its certification of a credential’s value would still be the coin of the realm. Freelance professors exist today. They’re called “adjuncts,” and many of them don’t love the version of the gig economy they live in. Of course, at a place like Harvard, they have “visiting professors,” “entrepreneurs in residence,” and so on. That’s a better gig economy to inhabit. In fact, it’s so much better than contracts are often negotiated individually to meet the needs of the in-demand superstars. And Web3 makes this easier…how? There’s a bit in here about intellectual property ownership, but here again, it’s not clear how the tech helps with that change.

    Saraf continues with a second idea:

    “For a very long time, our credentials have focused on grades and the name of the verifying institute,” Saraf says. “Instead, what if we focused on the output and performance you’re able to gain? If I took a class on robotics, I could put the actual robot [coursework] on the chain, not the grade. People could track the process, and it’s a much better indicator of who I am and what I learned.”

    What Could Web3 Mean for Education?

    This would be like a portfolio. Except electronic. An electronic portfolio. We could shorten it to ePortfolio. Catchy, right?

    I wrote a piece about the confusing nature of ePortfolios as a product category in 2006. That was before companies perfected the technology to scan résumés for keywords, having humans look at even less of what students do rather than more. Most companies I’ve seen that actually look at the work of prospective employees do so in the context of a challenge they set up as part of their interview process. In other words, they want to watch the prospects working at a task the employer has designed to be relevant. With the exceptions of career niches where portfolios were used even when they were analog—e.g., architecture, fine arts, and education—ePortfolios never caught on. The cost of human-evaluating a portfolio is too high for most employers and most jobs. I don’t see how blockchain or Web3 change that.

    The next example from the EdSurge article is longer, more complex, and probably not worth analyzing completely. It’s The Crypto, Culture, & Society DAO, which aspires to be “an experiment in building an educational institution directed by learners, not administrators.”

    Oh. Like a book club. Or a study group. Or Udemy. Wouldn’t it be awesome if Web3 could make those things possible?

    To make the group financially sustainable, the DAO plans to sell NFTs that grant owners access to future courses, plus provide educational services to other DAOs. Some of the non-fungible tokens will be reserved as “scholarship seats” for people underrepresented in the crypto world, Patel says. And for the general public, the group wrote up summaries of its first-semester courses and made them available for free.

    What Could Web3 Mean for Education?

    They could call these “tuition,” “financial aid,” and “open educational resources.” Imagine a world in which we had those things….

    The next example in Koenig’s article is something called EduDAO. I hesitate to critique it because I really don’t understand it. Seems like it could be a way of organizing and funding student study groups. Which would be a good thing. But yet again…I don’t get what Web3 adds here.

    Of course, a story about Web3 hype wouldn’t be complete without an example that includes the “metaverse,” a concept that always works out super well in science fiction books and movies. (I’m sure Facebook—er, Meta—will make it even better.)

    Imagine a virtual version of New York City, where everything is open 24/7 and geared toward teachers’ needs, desires and crypto-wallets. That’s what Saraf, of k20 Educators, says she’s building. She calls it the Eduverse.

    In the Eduverse, teacher avatars will teleport around to digital edtech hubs, where they can learn new technology skills to add to their credential chains. They’ll pop into marketplaces, where they can, say, trade lesson plans they’ve created for tokens. They’ll check out career centers where they can look for new jobs. And they’ll relax in lounges, where they can chat with fellow teachers from other countries.

    “The purpose is to create this live, central location where educators around the world can connect with each other,” Saraf says.

    What Could Web3 Mean for Education?

    So, basically, it’s Teachers Pay Teachers. In Minecraft. With Bitcoin. (Or Ethereum. Or Dogecoin. Or Educoin. I can’t keep up.)

    Very serious transitional side question: How the f*ck do these things get funded?

    I (hope I) don’t get it

    I’m not saying there is no use case for Web3 in education, but I am saying that I haven’t seen one yet and am having trouble imagining what one would look like (other than ASU’s aforementioned reverse credit transfer, which is not really what the Web3 enthusiasts have in mind).

    I don’t get the excitement.

    Unless I do.

    The most obvious (and saddest) explanation is that there are two forces in play here. The first is our primate instinct to travel in groups. Gartner’s hype cycle exists for a reason. People get excited, and then other people get excited because they trust the people who are getting excited. And then still other people get excited because everybody else is getting excited. After which, disappointment, financial failures, and copious amounts of grifting ensue.

    Sometimes an “innovation” emerges out of the Trough of Disillusionment and eventually reaches the anticlimactic Plateau of Productivity. (I vacationed there once. It was kind of boring, but I got a lot done.) Other times, the idea drops from the Trough of Disillusionment into the Marianas Trench of Bad Ideas We Shall Never Speak of Again. It’s hard to say which way Web3 will go this early in the hype cycle. We’ve just barely reached “WHEEEEEEE!”

    The second force that may be at work is the techno-libertarian mindset of Silicon Valley, which seems to have reached new heights. Never mind that freedom, in this case, may mean freedom to move from our Facebook overlords to our Etherium overlords. This is a kind of Atlas Shrugged crypto-libertarianism (where “crypto” is not meant in the technical sense). It’s a return to “disrupting education.”

    I’ll be among the first to admit that our education system is a hot mess. It’s a thousand-year-old system that certainly wasn’t designed and hasn’t really evolved so much as it has accreted over time, layer by layer, with the lower layers never really being changed so much as they have been driven deeper into the invisible foundation.

    That said, having just spent the better part of two days trying to figure out the very basics of how to set up a Facebook—sorry, Meta—page for my wife’s forthcoming book, I have a hard time believing that the geniuses whose crowning socio-technological achievements to-date are creating massive stock market volatility and selling cartoons for tens of thousands of dollars are going to replace our current educational system with something better. Somebody is going to have to make a pretty good case to me.

    I’m open to listening. Some of the people I’ve talked to about using Web3 in education are really smart folks who I respect and who genuinely care deeply about education. I will continue to hear them out.

    It’s just that no specific proposal I’ve heard so far makes sense. I don’t mean that I’m hearing bad ideas. I mean the ideas I’m hearing literally do not make sense to me. I do not understand them. All I hear is “blah blah blah something that already exists or shouldn’t exist blah blah Web3 blah awesome!”

    Blockchain from XKCD

    Maybe something good will come out of all this that I don’t see coming. For now, I suggest sitting it out and waiting for Web4. Which I hear is going to be awesome and is going to change everything. It will be self-driving, have 5G, and William Shatner will show us how it works live, from space, without having his wig float away.

    Genius.

  • Keeping 2U and Coursera Accountable

    Keeping 2U and Coursera Accountable

    Let’s give them something to talk about.

    This week at Georgia Tech’s annual Affordable Degrees at Scale Symposium, the group took some time away from their usual focus on building good programs to write a declaration of principles about the relationships between MOOC (or “at-scale platform”) providers and the universities that they serve. While this effort was triggered by angst among EdX stakeholders, both EdX and Coursera institutions attended. Their end product—The Atlanta Declaration—is worth a read. I was honored to be on a panel that provided feedback on the document, along with Doug Lederman, Phil Hill, and Paul Fain. (By the way, you should check out Paul’s new project, The Job, which is all about the college/work connection.)

    I’m going to share some of the feedback that I provided to the group.

    My frame of reference

    Since some may interpret my advice as provocative or hostile, I will start by stating my priors.

    First, I am entirely agnostic about the revenue share models that both 2U and Coursera use. While I believe each university should think carefully about the complex set of trade-offs that come with those agreements, I have no problem with universities deciding those trade-offs are, on balance, good for them. The revenue share model is a tool invented for helping universities with program startup costs. Revenue share companies act as banks. They give loans to universities using the courses they create together as collateral. All the usual caveats about financing something important with debt apply. (The same applies to Income Sharing Agreements (ISAs) for students.) There’s nothing wrong with debt financing if it is taken on for the right reasons and with reasonable terms from the lender.

    Second, I have no beef with either of the MOOC companies or their leadership. I know 2U CEO Chip Paucek pretty well. We’ve talked often. I’m convinced he genuinely believes in universities and wants to help them succeed. While I’ve not had the pleasure of meeting Coursera CEO Jeff Maggioncalda, people I trust who know him tell me he is a decent human being. Further, Coursera is a certified Public Benefit Corporation (PBC or B corp), and 2U has promised that its EdX division will also be a PBC. While there is justifiable debate about how effective the accountability mechanisms of PBCs will be, there is no question that gaining and maintaining certification entails considerable work. It demonstrates good intentions. As with the revenue sharing provisions, each university should ask whether its interests align with how a given vendor operates. But I have no opinion about what the answer to that question should be with respect to 2U or Coursera.

    Third, I worry a lot about the forces that shape EdTech companies across generations of leadership. For a long time, Instructure was a genuinely great and ethical company. When the original leadership left, it became a pretty lousy company pretty quickly. Now, under the current leadership, it seems…fine. Not great, and not terrible. Just another EdTech company. One short-lived CEO wiped out an enormous amount of goodwill in an astonishingly short period of time. Warren Buffet said, “Invest in companies that any idiot can run because, sooner or later, one will.” I am interested in building ethical EdTech companies that any jerk can run because, sooner or later, one will.

    Finally, I believe that declarations of the kind that the Atlanta group made are only useful if they catalyze effective action. So a group of MOOC customers made a declaration. Now what? How will it shape the behavior of the companies they were thinking about when they wrote it? How will it create guardrails for future behavior or drive thought or change inside these organizations? By itself, it won’t. 2U and Coursera both seem good at listening to their university partners. They’ll likely read the document, affirm that they support it, and maybe make a gesture or two to show that they’re sincere. And then they’ll move on. The declaration may be useful to help a coalition begin to form, but that coalition then needs to take further action in order to have an impact. And this, in fact, is what the writers of the declaration hope to accomplish.

    My recommendations are aimed squarely at the last two points. I have nothing against either company in their current incarnations. I simply believe we have a moral obligation to be stewards of the future as well as the present.

    All 2U and Coursera university partners should become shareholders

    I recommend that all universities that work with these two publicly traded companies buy the minimum number of shares required to become voting shareholders. They should appoint stakeholders to represent their interests in a way that balances any revenue goals with mission goals. The representatives from the shareholding universities should meet, discuss concerns, attend every shareholder meeting, and plan collective actions at the shareholder meetings. When necessary, they should make comments or motions.

    The goal is to establish a direct line of communication between the university customers and the shareholders, unmediated by senior executives. If customers raise objections or make motions as a group, it will be hard for the shareholders to disregard those concerns. 2U and Coursera may seem big, but they’re nowhere near the top on the spectrum of publicly traded companies. If shareholder activism can force Exxon to change its behavior, then activism can influence any publicly traded EdTech company. Furthermore, PBCs explicitly allow shareholders—and only shareholders—to sue the company for failing to uphold its stated mission goals. I’m not suggesting that such a lawsuit is likely to be necessary, but I am suggesting that this particular feature of a PBC gives university shareholders extra clout.

    The reasons universities should buy the minimum number of shares required to vote are twofold. First, I very much doubt that they could buy enough shares to outvote the traditional shareholders. It would be a waste of money to try. Second, if universities do become significant shareholders, then their internal stakeholders who are responsible for endowment funds will become entangled in the decision-making process and dilute the focus on shareholder activism. The power here isn’t in the number of votes. It’s in the number of customer voices.

    2U and Coursera should embrace this

    I imagine that my friends at 2U and Coursera may not love the idea of university folk showing up at their shareholder meetings. While I think I understand the risks and complications it would entail for them, it could be very good for them in the long run. Universities generally do not know the constraints and pressures that their vendors operate under. As a result, they can get angry about situations that they don’t have the context to fully understand or ask for their needs to be met in unhelpful ways because they don’t know how to work with the grain of the company. Having university stakeholders who feel responsible for understanding how the company functions and the pressures under which it operates could pull them further into the tent.

    If this idea gains traction, the best move these companies could make is to create a university liaison within their investor relations office. Actively engage with the stakeholders. Take the opportunity to educate them on how the company works and how more traditional shareholders think. Give them separate briefings that help them formulate input that is useful and helpful. Provide them with opportunities to share their needs with conventional shareholders in ways that give the companies cover to do the right thing.

    EdTech companies should have academic values woven into their structure

    I’m proposing a method to create greater alignment between the companies and the universities they serve. This isn’t about creating an adversarial relationship. Rather, it’s about creating conditions that reduce the likelihood of an adversarial relationship. It raises the stakes of conflict while creating new channels of communication. When the next CEO takes the helm at one of these companies, she will walk in knowing that her customers speak at her shareholder meetings. She will be motivated to ensure those customers can speak positively about how well she is serving the PBC’s mission goals.

    Even if she’s a jerk or an idiot.

  • Blursday on Engageli and EEP: Still Time to Register!

    Blursday on Engageli and EEP: Still Time to Register!

    I have two reminders and an announcement.

    First reminder: Today’s Blursday will be held on Engageli. It’s open to all (as usual). We’ll be talking to Engageli’s VP of Learning and Teaching Andreina Bloom Parisi-Amon about the platform and generally messing around with it.

    You can RSVP here:

    [mc4wp_form id=”12072″]

    Second reminder: As I described in a recent post, we’ll be running our EEP virtual summit next week on Engageli. The dates are Tuesday, 10/19 through Thursday, 10/21 from 11 AM to 3 PM ET. I’m curating the folks who come but there still are some open slots. If you want to come, please email me at [email protected] with the subject line “Request for invitation to EEP summit”.

    And the announcement:

    The three EEP featured sessions are open to all! Here are the details:

    • Tuesday, 10/19 at 2 PM ET: We’ll be talking about the big open-source collaboration between Carnegie Mellon’s OLI and ASU’s ETX Center. This project arose out of Carnegie Mellon’s open-source announcement at the 2019 EEP Summit and has evolved from there. (If you’ve been following e-Literate you’ll know that I’ve made a big bet, professionally, on this work.)
    • Wednesday, 10/20 at 2 PM ET: We’ll have a hands-on active learning session with the aforementioned Andriena Bloom Parisi-Amon and iDesign’s superstar Whitney Kilgore. I don’t think this one will be the same as the 67,000 other virtual active learning workshops you may have attended since the start of COVID. Engageli is going to let us break out of the Zoom box and have fun.
    • Thursday, 10/21 at 2 PM ET: Daphne Koller is going to chat with me about how we can wrap our heads around machine learning and artificial intelligence as it makes its way toward our classrooms.

    These are going to be great sessions, folks.

    Register here.

  • EEP is BACK! Relatedly, Blursday is on Engageli this Week

    EEP is BACK! Relatedly, Blursday is on Engageli this Week

    tl;dr:

    • EEP is back! (It never really went away, but that’s a longer story.)
    • We’re holding a virtual summit from Tuesday, 10/19 to Thursday, 10/21 from 11 AM to 3 PM ET.
    • I’m thrilled to announce that the summit will be powered by Engageli and that our friends at iDesign helped me put the program together.
    • While EEP is all about the participants and not about celebrity talking heads, I would be remiss if I didn’t mention that Daphne Koller will be one of our featured speakers. We’re not going to be chatting about her past and present EdTech startups but rather about how to think about machine learning and artificial intelligence as they make their way into EdTech.
    • For a variety of reasons, including the shot notice, I’m opening up for requested invitations. (It’s still a closed event, but less so than in the past.) Information for invitation requests is at the bottom of this post. Please read the post before requesting an invitation. You are more likely to get accepted if you do.
    • For all my Blursday Social friends, we’re going to do a Blursday Social session this Thursday, 10/14, from 4 PM to 5:30(ish) PM ET. The session will be held on Engageli, featuring my new Engageli friend Andreina Parisi-Amon.

    Because we’re using Engageli for Blursday, RSVP is a little different. Please sign up for this week’s Blursday here:

    [mc4wp_form id=”12072″]

    If you want to know what’s up with EEP and the summit (and maybe want to join), then read on…

    The back story

    For those who haven’t followed the EEP story, the Empirical Educator Project (EEP) is a thing I started a few years ago to spread the ideas that (1) teaching is a craft, (2) expertise in teaching should be part of every college instructor’s professional identity, (3) part of that identity means constantly learning from colleagues, published research, and one’s own methodical classroom explorations of what works for students.

    If you find yourself reading e-Literate regularly, chances are good that you believe in these principles. But a lot of you don’t know each other. Many of you feel like you’re laboring in the dark and that few others believe in or care about these principles. But you’re wrong. I get to meet a lot of you and talk to you about what you’re up to and what you care about. If you had the same privilege as I do, you would see the growing movement that I see. And you would find more colleagues to advance the cause.

    That, in a nutshell, is what EEP is about. It’s to help you find allies. Grow the movement.

    EEP started as an experiment, became something of a club, and was always intended to evolve into something open. In 2018 and 2019, I gathered some hand-picked folks together to have conversations. From a network-building perspective, it was a raging success. People made new friends. Conversations started. Brainstorms happened.

    To give you a flavor, here are a couple of short video compilations we made from just asking open-ended questions to our 2018 participants:

    EEP from the participants’ perspective.

    From the perspective of generating new collaborative projects, as you would expect, it was harder. We’ve had a few successes and a lot of good intentions that haven’t panned out (yet). The biggest raging success is actually the open-source courseware platform collaboration I’ve been writing about and the company I‘ve cofounded that spun out of it.

    During the pandemic, it was hard to get folks to focus on movement-building when they were so heavily burdened with the daily work of managing the crisis. Once I figured that out, I did two things. First, I continued the quiet work of the aforementioned open-source collaboration. And second, I started Blursday Socials, which was a more low-key way of building networks by creating a space for one thing that everybody was missing: relaxed but engaging hang-out time with old work-friends and the opportunity to meet new ones.

    Now that folks are in a position to start thinking about the future again, it’s time to re-engage with a more public-facing and activist version of EEP.

    But like everything else, it needs to evolve for the new world we’re in.

    What’s changed

    So first, face-to-face conferences still aren’t really easy for anyone (assuming you have a travel budget) and, honestly, Blursday Socials have taught me that we both should and can figure out engaging virtual conferences that are just a series of talking heads. EEP always viewed presentations as ways to meet interesting people and provoke conversations. Never as the main event. So we’re reconceiving the EEP summit as a virtual event. I’m thrilled to have Engageli power the summit and iDesign help think through how to translate the EEP lesson plan—which is how I always thought about the summit’s agenda and structure—into an online event.

    Second, this is the first step toward opening up EEP. I started out small and invitation-only because (a) I wanted to understand how to grow a culture before opening it up and (b) I had budget limitations that come with a physical conference that had a $0 registration fee. While we are still invitation-only, I’m opening up EEP to a kind of friends-of-friends network.

    The truth is that my network is too narrow too. Another lesson of Blursday Socials has been that many of you wonderful people who choose to wade through my long blog posts are people that I don’t actually know—but should. And you, in turn, know other people that I should know. So I’m expanding the field of potential participants. I’ll be providing instructions on how to ask to be included below. Within a few days, we’ll also be announcing other ways that people can participate more broadly.

    What’s on the agenda

    We will have three major themes and a fourth aspirational theme this year:

    1. Building EEP back better together: At our last (pre-COVID) summit, Carnegie Mellon University announced it was releasing $100,000,000 worth of grant-funded educational software as open-source. In addition to orienting everybody on the EEP way, we’ll discuss what happened with that software, with a focus on digital curricular materials (a.k.a. “courseware”). In other words, we want to pick up the project that got folks excited at the 2019 summit and move forward. More broadly, we want to start talking about what y’all want EEP to become.
    2. Active learning in a pervasively blended world: We’re not going to ignore the lessons of the past year. Instead, we’ll be talking about lessons learned from remote teaching, what we can learn, and what we have to rethink for the post-pandemic world.
    3. Machines and humans learning together: This is a look-ahead topic. Wouldn’t it be nice to be anticipating a challenge rather than reacting to a crisis? Ready or not artificial intelligence and machine learning are coming to the classroom. We need to start having conversations about both opportunities and hazards. We need policy frameworks, literacy programs for educators, and other efforts. So we’re going to talk about how to get that started.

    The fourth, aspirational theme is diversity, equity, and inclusion (DEI). We’ll have a couple of speakers on this topic but, to be honest, I have a very personal motivation for including this. At the 2019 summit, I announced that I did not meet my own standards for diverse speakers or participants. I vowed to do better. All indications are that I have failed to do so for our 2021 summit. So while DEI is always an important topic, I am specifically interested in having a conversation about how we can increase the diversity among our network of empirical educators. I, personally, need to diversify my network and learn skills for reaching and including a broader range of folks.

    And, as always with EEP, the most important agenda is yours. Come with a project, a need, an idea, an interest…whatever. Look for colleagues. If you find an opportunity to talk to somebody about something important and you want to tune out a session to have that conversation, you will be able to do that. In fact, we encourage it.

    Expectations for participation

    Here are some things to know about the summit:

    • This will not be a virtual “conference” in which you can listen to talking heads on your computer screen while answering emails. You will be at virtual tables with other humans in actual conversation about meaningful things. That’s what we think conferences are for. So please come prepared to be active. We’ll be conducting the summit on Engageli, which I promise you is not like Zoom or Teams. Look, this is a last-minute thing, so I expect that many of you will not be able to clear three half-days from your calendar. But if you don’t come to participate, then you will not get the intended value from the summit.
    • One of our explicit goals is to make sure that you leave the summit feeling like you have met somebody that you should know but didn’t and talked about an important project to you with somebody who might be helpful. As an introvert myself, I can relate to any hesitation you may feel about this. But even with our more open policy, we are still hand picking interesting people for you. I promise that you will meet people you will be happy to have met.
    • We are encouraging delegations to come from institutions. Please do not sit at the same virtual tables—we want you to meet new people—put please do compare notes with your colleagues about your experiences.
    • While we are realistic that folks have day jobs and need to get work done, we strongly encourage you to come for as much of the summit as you can. In particular, the networking sessions are not filler times for catching up on email. They are critical to the value of the event.

    About vendor participation

    Third, there’s the issue of vendor participation. I have always felt, and continue to feel, that participation by vendors is important because they can contribute to and facilitate this cultural change. The old model was that I hand-picked sponsors I thought would be good for EEP and charged them significant fees to cover the costs of the event. But we’re in a new world in several ways. First, my startup—Argos Education—is the new home for EEP. I’m not sure if that’s permanent or if we’ll spin EEP off. But for now, I’m still driving EEP and I’m working at Argos (and enlisting the help of my Argos colleagues). Therefore, Argos is the home of EEP. Since Argos is a company that may be considered competitive by certain potential sponsors, I have to rethink how this will work. Second, given that this is a transition year, I’m not sure what the new value proposition to sponsors would be yet.

    The upshot is that, while I will still be hand-picking some vendors to participate in EEP, sponsorship is not a requirement for participation in the summit. I have a couple of multi-year sponsors with whom I will be speaking to make sure they are treated fairly. Beyond that, if you are an EdTech company that wishes to attend EEP, you’re welcome to apply.

    Be aware, though, that my filter is still pretty strict. Specifically, since EEP is all about active participation in projects and culture development, I’ll want evidence that you’ve been a good contributor to the space in general and have been focused on either supporting educational research or meaningful promoting evidence-based teaching practices in active collaboration with academia.

    And no sales or marketing people are allowed. I’m looking for vendors who can send researchers, developers, product managers, and people in other roles that can contribute to project work.

    How to get an invitation

    If you’re a past participant, you should have already gotten an email. But maybe you didn’t read it, or it got stuck in a spam filter, or I screwed up somehow. Whether you are a veteran or a new person and want to come and participate for a significant portion of the summit, please email me at [email protected]. Please use “EEP summit request for invitation” as your subject line. Write a couple of sentences explaining why you would like to come. If you have a small group of colleagues who would like to come as a group, please let me know who they are and include their email addresses.

    If you are a vendor, please follow the above instructions and also include how your company contributes to a culture of empirical education and the names and roles of the person or people you would like to send.

    We have a limited number of spaces but we still have room as of this writing. And, as I mentioned earlier, we will be including more ways to participate in a follow-up announcement that’s coming soon.

  • Argos: Making Good Course Design Easy and Fun

    Argos: Making Good Course Design Easy and Fun

    I’ve been building the Argos story slowly in this series for two reasons. First, we’re in something like a trough of disillusionment with courseware products. “Trough of disillusionment” is a term from the Gartner Hype Cycle about how innovations are viewed in the marketplace over time:

    I say “something like” the trough of disillusionment because the hype cycle is really about expectations rather than product/market fit. A new product comes out and we get excited because we’re told it will “change everything.” It doesn’t. We get disappointed. We decide we’ve been scammed. Over time, we figure out that, even though the new thing isn’t going to change everything, it’s useful for some things. Once we finally figure out what it’s good for, we talk about it less and use it more. Throughout the whole process, the thing itself often hasn’t changed. Oh, sometimes the start-ups decide pivot, or they die because they can’t outlast the trough. But Stanford-style MOOCs, for example, haven’t changed all that much since 2012. We just know what they’re good for now.

    I don’t think that’s what’s going on with courseware. It feels like a trough of disillusionment. In 2014, Campus Technology wrote about “The Great Adaptive Learning Experiment.” (One sign that the market is mixed up is that we can’t seem to settle on terminology. Are we talking about adaptive learning, personalized learning, or courseware? What’s the difference?) Since then, we’ve seen a handful of these “personalized learning” companies—e.g., Knewton, Acrobatiq, and Smart Sparrow—sold off for a low price and largely fade into the companies that bought them. CogBooks was the latest to be sold off to Cambridge University Press. While this may be a decent deal for CogBooks—the financials have not been disclosed—I doubt it will be read as a sign that this product category is changing the world. The big textbook publishers have done OK with courseware but not well enough to halt their continuing shrinkage as businesses.

    There’s a general sense of disappointment and malaise with the product category. I’m arguing that this is not a genuine trough of disillusionment but a failure of product/market fit. This is actually much more common in EdTech product categories. Think of retention early-warning analytics, ePortfolios, and learning object repositories, among many others. They all got hyped, disappointed the market, and then got stuck. The problem wasn’t that the market misunderstood the product but rather that the product makers misunderstood the market.

    My colleagues and I believe that the same thing has happened with courseware. The market has gotten stuck on an idea of the value that doesn’t actually appeal to the people who have to adopt and use it. There was some progress when publishers realized that students like low-stakes practice. Before EdTech, many students voluntarily created flashcards for themselves. When publishers realized that students were increasingly refusing to buy the expensive curricular products that their professors were prescribing, they became more “learner-centric,” by which they meant that they thought of students as the buyers. So they built in features that students would like. Low-stakes practice. That helped some. But then they got stuck again.

    Low-stakes practice could give instructors more information about how their students are doing. It could change the way those instructors teach. But so far, it mostly hasn’t. Which means that instructors don’t see value in prescribing courseware over alternative formats, like simple ebooks. The “market” for textbooks is both students who buy the product and instructors who prescribe it. Both use the product in some important sense. But on the instructor side, the paradigm has largely been stuck in the bound book format. It’s immutable. It’s a thing they adopt and then teach around.

    In contrast to the publishers, stand-alone course platform providers went in the complete opposite direction. They gave instructors a blank page. In their later stages before their acquisition, these vendors invariably—invariably—began developing or acquiring catalogs of titles so that instructors wouldn’t have to start from scratch. That tells us something about product/market fit. But the platform vendors also differed from the publishers in that they were selling their platforms first to institutions as enterprise software licenses. Those institutions then typically engaged in top-down efforts to get faculty to adopt the platform and develop content. In this conception, the product has three kinds of customers—the institution paying the license fee, the instructor adopting (or creating) the curricular product on the platform, and the students who are using (and possibly paying for) that curricular product.

    It doesn’t work.

    The joy and magic of product/market fit

    Way back in the first sentence of this post, I said there were two reasons why I’ve been building this story slowly. The second reason is that I take joy in writing this way. It’s a teacher thing. I want to make you care about things that you don’t think you care about. I want to make you spend time thinking about topics that you never thought you’d spend time on.

    You think you wouldn’t spend time reading a long piece on how fMRI machines work, the mathematics behind certain kinds of machine learning, and how these pieces come together in a methodological paper in cognitive psychology? You think there’s no way you would read a seven-thousand-word article on how Pearson is using HR policies to foster efficacy in its organization (which, by the way, was published on New Year’s Day)?

    Heh. Let’s put that to the test. I bet I can make you care. And I’ll spend days or even weeks researching and crafting a blog post to prove it.

    Again, it’s a teacher thing. I have an urge. I will go to irrational lengths to fulfill that urge. But not infinite lengths. If I had to write these blog posts using Dreamweaver in raw HTML, I never would have gotten started blogging. I needed WordPress to lower certain barriers for me so that the urge could take over. Remember “web 2.0”? This is web 2.0. When I started blogging, I never imagined I would still be doing it 16 years later, or that literally almost everyone who has access to the internet is creating and sharing personal content over some platform, whether it’s Facebook, TikTok, Twitter, LinkedIn, YouTube…the list goes on and on.

    Great product design works that way. I have spent a lifetime failing to get myself to exercise regularly. No product and no subscription I tried could crack that nut. Until I bought an Apple Watch and subscribed to Apple Fitness+. Now I do 30-minute yoga sessions at least three times a week. In good weeks, I’ll add a couple of core workouts and high-intensity interval training (HIIT) sessions.

    And then there’s the kayaking. I wrote a post on product/market fit that included a description of origami kayaks that my wife and I use. We had a canoe a while back but it was too heavy and too hard to get on and off the car. Now we go kayaking. And, interestingly, I received a handful of notes from people who read my post and decided to buy themselves the same kayaks.

    Oru Kayak

    I always wanted to exercise regularly. Logistical and psychological barriers prevented me from doing so. It would have been easy to conclude from a distance that people like me don’t want to exercise. I did hate exercising, but I wanted to exercise. I wanted to not hate it. My Apple Watch and Oru Kayak enabled me to find ways of exercising that I didn’t hate. As a result, my lifelong behavioral patterns changed pretty dramatically.

    One of the best examples for the story of courseware is Wikipedia. When it came out, most people thought it was nuts. The big thing in encyclopedias at the time was Microsoft Encarta, which was supposed to be a multimedia, interactive Encyclopedia Britannica killer. The idea that a bunch of randos on the internet would write a credible encyclopedia was literally a joke. Stephen Colbert encouraged his viewers to write absurd details into Wikipedia articles. Random people could never write a comprehensive encyclopedia with no formal coordination and no financial incentive. And even if they did, nobody would trust it.

    Until, in 2009, The New York Times ran the headline, “Microsoft Encarta Dies After Long Battle with Wikipedia.”

    Today, when you search on Google the top-right corner of your results page will often have information summarized from and linked to Wikipedia. I don’t know about you, but I click on that link fairly often. And I generally don’t think about whether I can trust the article it takes me to. As I write this sentence, random people on the internet are fact-checking and updating articles on Wikipedia. I’m not one of them. I’m mostly just a consumer. Most of you probably are too. And that’s fine. It works.

    I started a Wikipedia page once because it was related to a particular EdTech topic that I was passionate about. I only wrote a couple of lines; it wasn’t much more than a stub for the topic. A bunch of other people each contributed a bit to the entry. I haven’t looked at it in years but others continued to contribute and update it This is the page today, complete with 148 footnotes. As I write this, the most recent update to the article was yesterday.

    We forget about Wikipedia because now we’re thinking about Facebook, which is designed to facilitate the opposite kind of behavior. Humans are malleable and adaptable. But you have to understand their drivers if you want to fulfill their needs, particularly if doing so entails a change in their behavior.

    My colleagues and I believe that the courseware product category is in a rut because the market has fundamentally misunderstood both how educators use curricular materials and how they would use them if only certain barriers were lowered for them.

    Upcycling curriculum

    In the first posts of this series, I wrote about how educators think about and use curricular materials in their course design. In this last section of this blog post, I’m going to start telling a story of how that can work using some mock-ups that we’re user testing. The goal here is to give some examples of how upcycling can work.

    Let’s say you’re selecting a textbook for your Chemistry for Non-Majors course. Your goal isn’t really to pick textbook. It’s to design your course. You go into the store that has all the products that run on Argos Education’s Sojourner platform. You have some good choices. You could pick Carnegie Mellon OLI’s General Chemistry course, which has a mastery learning approach and has been carefully engineered for efficacy. You could teach with ASU Inspark Network’s Critical Chemistry, a heavily experiential learning-based product for non-majors that is thematically organized around how chemistry is used to save lives. But in your case, your college puts a strong emphasis on affordability. That’s a major priority. So you search the catalog particularly for inexpensive options:

    You’re still focused on getting through the curricular materials selection process so you can get to the course design process. You review the options and pick your title.

    Now you have a choice. Are you working on your own? Are you leading a departmental adoption process? Maybe working with a learning designer or an OPM to create an online course? Maybe you want collaborators involved:

    You have options based on your use case. And not everybody gets to edit. For example, maybe as a department, you want to have all instructors be able to see what you’re working on, some who are able to comment, and just a couple who are able to actually make changes. On the other hand, if you’re just making the selection yourself, you can skip over all this. Or you can invite your institution’s faculty support person or a colleague in to give you some suggestions.

    But you’re still in adoption mode. You’re getting to know the text so that you can design your course. You’ve picked an unmodified OpenStax title, so it’s going to start off looking a lot like the free PDF version:

    Notice the little notes on the right-hand side. We’ve applied a little machine learning and a little learning design research to make the pedagogical structure of the text more visible to you. If you mouse over the little item, you can see more detail:

    Ah. This is interesting. You get this. It’s a bit like how you think about putting together your lectures. Maybe you want to switch to outline mode so you can see the overall structure of how this thing is put together:

    You’ve collapsed the text so that you can focus more on those little post-it notes and see the structure. It’s pretty much like a lecture. (It’s labeled “directive instruction” for a reason.) You introduce the concept to help students understand why the topic is important to the course, you provide them with some information, maybe you work a problem out for them—if you were doing this in class, you might work that problem out on a whiteboard—you…

    Oh. Hmm. Usually you wrap up the lecture with a summary. This text doesn’t do that. You could fix that, but honestly, you’re not too worried about it.

    The OpenStax text also doesn’t give an end-of-chapter test at the end. The truth is that you’d want to pull together some questions from a test bank anyway. So you’ll worry about that later too.

    You dive back into Design mode to review the text. You want to give it a close reading now as you think through the design of your classes. And you find another unpleasant discovery. The section on acid-base reactions is just not right for your class. The language is a bit garbled and probably unnecessarily complex for your students.

    This is the point when you would normally write the following in your syllabus:

    When you get to Section 7.2.2 on acid-base reactions, do not read it. Instead, please go to this folder in Canvas and read the document called “Acid-Base Reactions.” Follow the link at the bottom of the page and take the quiz there. Then read the rest of Section 7.”

    Which is not ideal for anyone. For you, you’re writing documents and organizing folders and diving into LMS quizzing tools and then trying to knit together what the students are doing in the LMS with what they’re doing in the courseware. For the students, they’re jumping through hoops.

    But instructors do it all the time. Locking down the courseware might create enough friction that they will do it less often than they might otherwise. But many of them will do it. It’s a bad experience for everyone. And there’s no way for anyone to tell if the net result is better or worse for the student because the activity is happening in different systems that don’t really talk to each other. But as a dedicated instructor, you will do it if you think it will help the students.

    Or, if you’re in a platform that allows you to edit the curricular materials, you could just do this:

    This is better for everyone. Students don’t have to jump through hoops. Instructors spend much less time setting up the content. And now that all the information is in one system, it’s possible to track whether these changes actually help students. That last piece is another topic for another blog post, but it’s incredibly important. If we want to create a virtuous cycle, we want to share changes, test which ones improve learning in which contexts, and generally create positive evolutionary pressure.

    Finding teachable moments in the course design process

    This idea of pulling editing “inside the tent” is central to understanding product/market fit and the levers available to change behaviors. As an instructor, you are mostly not trying write or rewrite textbooks. That’s not you think about what you’re doing, in part because the textbook is the raw material for their course design. It’s not a thing you care about in and of itself.

    We want to catch you in those moments when you’re thinking about their course design and pull you into a kind of dialog as you use the raw materials to build that course.

    And we’re always making choices about what we want to ask you to think about and what we want to get out of your way. If you want to add a virtual lab, maybe we give you the option of some scaffolding to help you think through all the steps you want students to go through:

    When you pick their actual virtual lab, maybe we don’t bother you by calling attention to the learning objectives but we do show you a selection of labs that are tagged with the learning objectives associated with the content section you are working on:

    When you choose the lab, we let you see the cost change and, if you have set a cost cap, we warn you when you go over. We even give you a breakdown of what you’re asking the students to pay for:

    We don’t always want to hide learning objectives from you. Rather, we pick the moment when learning objectives are most useful to think about, like when you’re picking out assessment questions:

    Notice that the platform introducing a bit of implicit community here. Other instructors are also designing courses around the same curricular product that you are. Maybe they have some good ideas that they’re willing to share.

    And the real value of the sharing is not just the content but the rationale behind the course design. Why did the author write a particular answer in a particular way?

    I could go on, but hopefully the theme is clear.

    Professionally designed courseware has a lot of useful information either explicitly or implicitly built into it. We hide that design information from the instructors and then lock the product down. In doing so, we turn it into a static artifact that they have to work around. However the publisher might conceive of the product’s value is irrelevant if instructors are using it as raw materials for their course designs.

    But if you start from that assumption, let them do what they’re going to do anyway in an environment where you can scaffold, collaborate, and provide feedback on the results of their designs, you can engage them in a much deeper way while saving them time and giving them more control over the things they care about. Also, we no longer have to separate conversations about course design from conversations about effective teaching practices. The fact that we do so now is a strong indicator that something is broken in our ecosystem. Course design and in-the-moment teaching are a continuous cycle. The two are constantly in dialog with each other. That’s the way teachers actually teach in the real world. Lesson plans change. You adjust. You improvise. Much of this is not arbitrary but rather driven by your encounters with your students.

    If we want courseware to grow in adoption and educational impact (and perhaps as a useful and self-sustaining industry), we have to get away from the book. That doesn’t just mean getting away from linear, non-interactive designs. More importantly, it means getting away from the textbook as a static artifact that educators teach around.

    By the way, today’s Blursday Social (9/16 at 4 PM ET) will be a chat about Argos. So you can come to that if you’re curious. Sign up here.

  • Chegg vs. Pearson: Are back-of-chapter answers intellectual property?

    Chegg vs. Pearson: Are back-of-chapter answers intellectual property?

    We interrupt our regularly scheduled post series on Argos Education for this breaking news…

    No, I’m not talking about the news that Blackboard was acquired by some ERPish rollup creation of a private equity company. I searched my soul to examine my deepest thoughts and feelings on that topic. Here’s what I came up with:

    • I’m happy that Bill Ballhaus finally got paroled. While he and I clashed once or twice, he seems like basically a nice guy. He didn’t deserve to be stuck trying to unload that company for five years.
    • I’m both happy and worried for the nice people who work at Blackboard. I hope this means that most of them will still have jobs.
    • I have similar feelings about Blackboard customers. I hope this works out for them.
    • I am deeply grateful that Phil Hill exists so that I don’t have to spend my time trying to make sense of this deal.

    That’s it. That’s all I’ve got. My therapist tells me this is good progress in recovering from the formative trauma that turned me into the blogger that I am today.

    No, I’m talking about Pearson suing Chegg for copyright infringement because Chegg has reconstructed Pearson’s back-of-chapter homework answers:

    Specifically, the suit revolves around Chegg Study, a subscription service that among its offerings features answers to end-of-chapter homework questions from various texts for $14.95 a month, with the textbook questions often copied nearly verbatim or with just slight changes, the suit alleges. Pearson lawyers told the court that “the majority” of Chegg’s roughly $644 million in total revenue in 2020 came from its sales of answers through Chegg Study, which reportedly counted some 6.6 million subscribers as of 2020.

    Pearson Education Sues Chegg, Alleging ‘Massive’ Copyright Infringement

    You may recall my post responding to the Forbes article about them, which was entitled “This $12 Billion Company Is Getting Rich Off Students Cheating Their Way Through Covid.” The main piece of new information I added was that most of the large publishers themselves had licensed these answers to Chegg a while back. Those contracts have expired. And now we have at least a hint of how much Pearson thinks their contract with Chegg may have cost them. While the numbers here don’t translate directly into Pearson losses, they do give us an order-of-magnitude sense of the amount of business here.

    More immediately interesting to me, though, is this paragraph from the complaint:

    As part of Pearson’s focus on pedagogy, Pearson and its authors devote
    significant creative effort to develop effective, imaginative, and engaging questions to include in the textbooks it publishes. Pearson’s end-of-chapter questions are strategically designed and carefully calibrated to reinforce key concepts taught in the textbooks, test students’ comprehension of these issues, enhance students’ problem-solving skills, and, ultimately, improve students’ understanding of the subject matter. Pearson’s textbooks can contain hundreds or thousands of end-of-chapter questions. These end-of-chapter questions form core components of the teaching materials contained in Pearson textbooks and are frequently hallmarks of Pearson titles. As such, the availability, quality, and utility of these questions are often important considerations when educators select which textbooks to adopt for their courses.

    Pearson v. Chegg complaint

    While the lawsuit doesn’t interest me, the core issue about the value of assessment question construction does. At first blush, it’s tempting to think that the answer to “Solve for x: x2 + 3x +9 = 0″ could not possibly be intellectual property. But, as Pearson’s complaint states further down,

    One of the points of my original post was that companies like Chegg exist in part because instructor grading policies put students in a position between wanting to learn and needing to pass. While that tension is unavoidable in some cases, it is largely avoidable in many cases. We simply haven’t taught instructors about productive ways to grade formative assessments (or even what formative assessments are).

    But the lawsuit brings up a separate and equally important angle. When we step back from the deforming effect of grading and simply look at the assessments themselves, we see teaching craft. Both the construction and the order of well-written questions are designed to probe the finer points of students’ understanding. And in some cases, it teaches those finer points. When students are asked to solve a problem that has one new twist from the previous one, sometimes they learn just by figuring out that twist in the moment.

    Pearson is arguing in their complaint that substantial teaching craft is built into their question construction. It has a financial value to the company because it’s part of the value proposition of their product. Setting aside the legal question of whether recreating homework answers constitutes copyright infringement under current US law, I agree with Pearson about the value. I also think that the publishers should not have a de facto monopoly on the craft that creates that value.

    Sometimes the questions don’t work as intended. It could be a flaw in the question construction, like an obvious correct answer or, conversely, confusing wording that trips up some students who understand the concept, causing them to give an answer that is considered “incorrect.” Or the sequencing could be off. If the order of questions can be carefully crafted for pedagogical reasons, then anti-cheating functions like question randomization or algorithmically generated questions can take away that tool. And sometimes the question may be written in a way that requires cultural or other context that not all students have in order to understand them. (IQ and other standardized tests were particularly notorious for their cultural biases in their early decades.)

    For a variety of reasons, good educators will tweak and invent assessment questions to fit the needs of the students in front of them. And yet, how many assessment EdTech tools can you think of that do a good job of helping instructors learn and share craft in this regard? It’s quite easy to break a good, psychometrically validated assessment design unintentionally by adding problems that are easier or harder than the instructor thinks they are, for example. On an even more basic level, nobody ever taught me how to write a good distractor or even what that term meant. I had to pick that craft up on my own, first by instinct and later by reading and by asking experts. (I am lucky enough to have had access to such experts.)

    There has to be a balance. On one hand, instructors should be taught how to recognize and analyze the embodied craft of the assessment questions that they use as part of a curricular product. On the other hand, we should not assume that those assessments are perfectly designed or even that there is any such thing as a single perfectly designed assessment for all teaching contexts in which a curricular product may be used.

    Instructors will very often use a mix of pre-made assessment questions and their own. And if they don’t have one tool that lets them do both, then they will mix bits of assessments from different tools with very little way to integrate the information usefully. No product I know of is currently doing a good job of helping instructors think through how to mix and match well. Even dedicated publisher homework platforms like Pearson Mastering or MHE ALEKS, which are theoretically intended for supporting exactly this use case, struggle to strike a balance between the features where the platform is making design- and data-driven decisions and the features where the instructors can insert their own problems. Product designers tend to think of the former as their real mission and the latter as something they have to accommodate because some instructors demand a measure of control.

    This is a market failure. Some of the consequences are as follows:

    • Instructors don’t make full and appropriate use of advanced curricular products.
    • Instructors often have no way, either functionally in the product or intellectually with the skills they have been taught, to appropriately modify a crafted assessment to make it better rather than worse.
    • There are not tools that enable instructors to look across their entire collection of assessments and easily evaluate how the pieces are working together.
    • There are no clear and easy means for instructors to collaborate with the designers of their curricular products, other adopters of the product who are master teachers, or even their campus faculty support staff, to help them upcycle the curricular product into a course design that is as good as or better than what they’re starting with.

    One way to look at copyrighted material is as embodied craft. We have gotten hung up on the idea that the particular embodiment is the locus of educational value. Hence, the lawsuit. Instead, we should really be working on how we can enable and encourage career educators to become expert practitioners and empower them with tools that enable them to practice their craft with more skill and precision. While they shouldn’t have to craft everything from scratch, neither should they be forced to live in a world where there’s a hard and opaque wall between the course design components that they are upcycling and the ones that they are crafting themselves.

  • Argos: Upcycling Course Design

    In my first post in this series, I wrote about what good teaching is. While the argument was wrapped in a lot of autobiography, it makes a number of assertions that are foundational to the mission of Argos:

    • The best evidence we have suggests that teachers change lives, by I we mean that students who have had good teachers will make more money, be physically and mentally healthier, and have more fulfilling relationships over the entire course of their lives.
    • Good teaching combines the emotional and the intellectual. In particular, teaching the diverse group of students that can be found in almost any classroom effectively requires diagnostic skill and creativity.
    • The intellectual aspect of good teaching is largely invisible to our society, including even many proponents and practitioners of education. The average person may sense it from time to time but, as a rule, we don’t talk about how it works and don’t think about it as an aquired skill set (nevermind a discipline that evolves over time).
    • As a result, there are social barriers to educators sharing craft with each other on top of the barriers created by the employment conditions in higher education.

    One result is a poor understanding of the textbook’s role and product/market fit. In turn, this has contributed to several decades of slow, painful, and often unproductive efforts to creating a vision for a digitally transformed version of the product. We can’t see what we can’t see. As Henry Ford put it, “If I had asked customers what they wanted, they would have told me a faster horse.”

    The primary purpose a textbook serves is to enable an instructor to upcycle the author’s course design. They adopt and adapt that design to fit their local needs and context rather than spending enormous amounts of time writing all the course elements from scratch. In the Industrial Age, where editing, typesetting, printing, distributing, and selling paper books were all labor-intensive and specialized tasks, it made sense to have one author, one editor, and one distributor to produce an artifact that each instructor would then independently upcycle. In the digital world, we can continuously upcycle course design by crowdsourcing improvement, both directly through collaboration and indirectly through shared data.

    That’s an essential part of what Argos Education does. We enable a virtuous cycle where educators can continuously upcycle their course designs with their peers. This enables them to teach more effectively with less work. It also creates a positive evolutionary pressure on their course design and delivery. While educators are still free to exercise their judgment in their respective classrooms, they will do so knowing how their peers are upcycling the same basic course design as well as how well various aspects of the course design are working based on analytics. In other words, they will have peer and expert feedback and suggestions that they don’t have now.

    How paper textbooks work

    What is an educator doing when she selects a textbook? What is she looking for? What are her criteria? On a micro level, then answers vary dramatically from one educator to the next. One is looking for good back-of-chapter questions. Another is looking for readings that will be engaging and accessible to her students. Or a particular angle on the curriculum. Or a particular teaching method.

    On a macro level, all of these fall into the larger bucket of looking for pieces of course design to upcycle. The specifics of what professors look for can be heavily inflected by a variety of factors. For highly procedural subjects like math, many educators will focus on the progression of scaffolded and unscaffolded problems. In economics, it could be philosophical/political. Conservative economists may like Greg Mankiw’s explanation of inflation while progressive economists may prefer Paul Krugman’s. A biology professor may choose a text that emphasizes an ecological lens to the topic over a microbiological one for several reasons. Maybe her university is known for its environmental studies programs but not for its pre-med programs. Or maybe she herself feels better equipped to teach the subject well from one angle rather than the other.

    These differences can be neutral with respect to some imagined ideal of an “effective” class. How much does it really matter if a first-year biology student learns a little more ecology and a little less microbiology? Or the other way around? In those early courses, we mainly want to help the students develop their interest in the subject, some general knowledge, and some discipline-related thinking skills.

    But in context, the differences can be dramatic. Teaching biology 101 from a microbiology perspective to a student who is trying to decide whether she wants to pursue a major in environmental studies may cause her to drop the course or even her major. Likewise, having a professor teach the course from an angle that they’re not comfortable with is likely to lead to a disengaged, paint-by-numbers teaching approach. If the educator is bored or uncomfortable, then the students will likely be bored or uncomfortable too.

    Educators look for textbooks that give them more materials that they can upcycle into their course designs. The more the materials fit the course design in the teacher’s head, the more that teacher can focus on adding value in her context rather than filling in holes in the base course materials. And make no mistake: Educators almost always customize, even if they don’t think about what they’re doing in those terms.

    When I taught a variety of subjects to middle school students, the one I was least opinionated about was pre-algebra. The book I had been given was pretty good and I didn’t feel terribly confident in my ability to come up with better teaching approaches to that subject. So I taught the book, chapter by chapter. I didn’t write a lot of supplemental materials or skip around. But I improvised all the time. For example, at that age, some students acquire abstract thinking later than others. This makes teaching concepts like negative numbers difficult. The book I was using talked about number lines but those were still too abstract. So I took the kids out onto the streets of Hoboken and we turned one of them into a number line that the students could walk. Later, when we were graphing lines, we turned it into a two-dimensional graph. (I was lucky that Hoboken streets had a grid layout.)

    I created lessons that weren’t in the book so that the course design would fit the real needs of my actual students. I didn’t do it for pleasure or out of a sense that I was a better mathematician than the textbook author or even that I was a better curriculum designer in the global sense. But I knew better what my students needed from me because I knew my particular students’ needs and my own capabilities better. So I upcycled the course design. I took a found object that didn’t quite fit the purpose and turned it into something that did.

    Credit: Surfer Today

    Even though I was more opinionated about how to teach science, I found a textbook that I really liked and followed that fairly closely too. But when a student asked a question that was provoked by but not answered by the experiment in the book, what was I supposed to do? Ignore it? Tell the student to go read about it?

    No. I wanted to visibly reward genuine scientific curiosity in front of the class. So we made up experiments together to answer her question.

    I didn’t think of it as either following or not following the textbook. I didn’t care about the textbook one way or the other. The textbook was just a tool for producing a better course design with less work. I cared about helping my students learn what they needed to learn in the ways that they needed to learn it, some of which the textbook’s author could anticipate and some of which he could not. (It was a male author.)

    Sometimes—often, if we’re being honest—educators are not lucky enough to find a book that fits their contextual needs perfectly. For example, some of my students had never been out of a city. Taking them to the zoo helped stimulate some of them, but a zoo is not the same as a natural habitat. And even if I had VR and could take them out to the African savannah or an alien planet virtually, it still would have been an abstract concept to them. The best way to hook a lot of them was to create a transect in the park directly across the street from the school. I showed them how the space where they hung out and kicked a ball around was absolutely teeming with life that they never noticed. Suddenly, ecology was real to them. It was salient. At that point, the information in textbook would become much more meaningful to them.

    The book I had adopted had some ideas about creating transects but I had to improvise a lot and fill in gaps where the text failed to address urban ecologies. I ended up creating a fair bit of original curriculum. Again, that had not been my intention. I didn’t even think about it as altering the course design. I was just trying to fill in holes for my students. It was often an improvisational act.

    College is no different. First-generation students at Georgia State University are just as likely to have never seen the countryside as my Hoboken middle schoolers. In every course at every school, we have to adjust to the people who are in front of us using the skills that we have. There is no chance that we can “instructor-proof” courses with “personalized learning” any time in the foreseeable future, even if that were a desirable outcome.

    We have fetishized the artifact of the textbook without really understanding how it is used by an expert practitioner (partly because we don’t think of educators as expert practitioners). We have constructed a fantasy of A Young Lady’s Illustrated Primer as the apotheosis of the textbook when textbooks have never functioned that way.

    Biologists don’t select “Pearson Biology.” The top-selling biology textbook sold by Pearson is named after the biologist who authored it—Neil Allison Campbell. Educators who adopt Campbell Biology do so, in part, because they like Campbell’s course design, freeze-dried into textbook format. They upcycle the materials for their own classes. They attribute that course design to a presumed colleague, even if he’s one they haven’t met: noted biologist and biology educator Neil Allison Campbell. Pearson actively maintains that fiction even though Campbell has been dead for sixteen years and three editions have been published since his death. So baked into the very heart of today’s textbook branding system is a sense of collegial sharing that is increasingly a facade created by the publishers.

    Two-cycle upcycling

    Sometimes academic departments select textbooks as a group. And they do so as part of designing the curriculum for their entry-level courses. In this case, the department is upcycling the course design collaboratively—or, at least, the tenured and tenure-track professors are—and then that design on to the individual educators. Who inevitably upcycle it themselves from time to time. Even if they color inside the lines, there are always things that come up with the students that the course design didn’t anticipate. And many educators take liberties with the course designs they are given.

    In an ideal world, educators within a department would meet regularly to share their upcycling and improve on the core course design template. But they generally don’t because they don’t have the time and because these upcycling changes aren’t easy to share. Sometimes they happen in class, either planned or unplanned. Sometimes they happen in the LMS, or in individual tools connected to the LMS. The departmental course adoption process demonstrates the need and desire for collaborative, multi-cycle course design upcycling. But it’s too hard and too time-consuming. As a result, all that extra work that educators put in to improve their course designs, and that could contribute to a collegial upcycling conversation, are lost.

    EdTech has made upcycling harder

    You would think that having everything in digital would make upcycling easier. At the moment, the opposite is true. Suppose, for example, that you decide to adopt the latest edition of Campbell Biology. You have a couple of choices. Your students could use the eText or a textbook. The eText is no more editable than the textbook. You could also augment by adopting the bundle of the eText and Mastering. So you’ll have to tell students, “First, read this section in your eText. Then follow this link to the homework platform. After that, come back to the LMS.” Is that easier than it was to accomplish the same tasks in a physical classroom? Is it easier for the instructor to set up and customize? Or easier for the students to follow along? I think not. It is easier to grade the homework. And it’s super easy for students to Google the answers to those homework problems.

    Now suppose you don’t think a section of the textbook is appropriate for your students. Maybe it’s too hard for the mix of students you have, for example,

    Read sections 7 to 7.2.2 in your eText. Do not read section 7.2.3. Instead, follow this link to the LMS folder called “Chapter 7” and read the document called ‘Alternate 7.2.3.’ Then, follow the link at the bottom of the document and take the quiz in the LMS. After that, go back to your eText and read 7.2.4 through to the end of the chapter.

    Easy, right? While we’re at it, let’s throw in a virtual lab and a discussion in the special, non-LMS forum tool that you like. Upcycling in this environment is incredibly hard. Sharing your upcycling for somebody else to use—or even just migrating it from one LMS to another—is way too hard and time-consuming to expect full-time educators to do.

    Attempts to address this problem have largely failed. OER creates a licensing structure that is supposed to facilitate upcycling. But upcycling requires infrastructure as well as a license. Just because you can theoretically edit that OER-licensed PDF doesn’t make it any easier to upcycle in reality.

    On the technical side, Learning Object Repositories (LORs) designed by technologists for large organizations fail because the product category takes the wrong approach. The whole idea of a “re-usable learning object” is built on the analogy to re-usable software objects. But software and content aren’t re-usable in the same way. Educational content is all about context. You can’t chop up a bunch of courses on the same topics, paste pieces together, and have them make sense to learners. Creating modular, ready-to-reuse content (as opposed to upcyclable content) is incredibly hard. Often impossible. So LOR creators often spend tremendous amounts of time creating elaborate content taxonomies, only to discover that most people are using their fancy systems as media libraries and quiz question banks.

    I’m not suggesting that either open licenses or LORs are useless. They just don’t solve the whole problem of continuous upcycling.

    Continuous upcycling is possible

    What if we could dramatically lower the barrier for continuous, collaborative, data-informed upcycling of course designs? Catch that teacher at the moment of improvisational need and make a good suggestion? Share brilliant teaching ideas that often never leave the classroom of the person who created them? Nudge teachers who are in a bit of a rut to show them some exciting things their colleagues are doing? All while keeping them focused on their core teaching challenges?

    What if every educator who adopts a “textbook” could join one or more communities of adopters? What if they could edit, delete, and insert new content into the product they were upcycling? What if they could choose to share those edits with their department, their learning designer, the original author of the curricular product, or the whole community of educators who are adopting it? What if the product could recommend changes that colleagues are making, particularly if those changes seemed connected to better learning outcomes? What if educators could selectively adopt those changes? And what if they could choose to have conversations with their colleagues about the design, how they’re changing it, why, and how they could tell if it works?

    This is how Argos Education aspires to change the fundamental value proposition of the “textbook,” or “courseware,” or whatever awful name you want to invent for that freeze-dried course design that educators upcycle to meet their local needs.

    In my next post in this series, I’ll talk concretely about how this ecosystem—and economy—works.

    In the meantime, this week’s Blursday Social will be a chat about Argos. So you can come to that if you’re curious. Sign up here.