The reaction to my last post—AI, Cheating, and the Future of Work—was mixed. Some readers who have data science backgrounds seemed to like it and think it’s funny. Folks from outside that circle tended to shrug and move on.
The thing is, my analogy between cheating and artificial intelligence was meant to be taken both seriously and literally. For a layperson, one good way to think about artificial intelligence (AI) and particularly machine learning (ML) is to make analogies to cognitive shortcuts that humans use. Like all analogies, this one is imperfect. But it’s a helpful first approximation.
Have you ever been in a conversation where you only half understood what was being talked about? Maybe the room was noisy. Maybe the other person had a heavy accent. Maybe the subject was one that you didn’t understand as well as the other person.
Perhaps, for whatever reason, you decided not to let on that you were only getting bits of the conversation. Maybe you didn’t want to embarrass yourself. Or the other person. Maybe you didn’t want to interrupt the conversational flow. Or you could have believed you had understood enough to make fairly confident guesses at the gaps in your understanding.
How did you manage to keep up your side of the conversation? What strategies did you employ? Usually, we’re drawing on whatever context we can. Who is this person? What is the general topic of the conversation? Why are you talking about it? What might this person know about it? What is the person’s body language telling you about the reaction they expect? (“Is this the punchline of a joke? Should I laugh?”)
We can be very good at faking understanding in various situations. Except when we aren’t. Sometimes we guess hilariously and/or disastrously wrong. You’ve probably been on the other side of such a conversation and suddenly been surprised when the person you are talking to responds with a non sequitur or a wildly inappropriate reaction.
This is one way to think about what ML and AI algorithms do. They fake understanding by employing strategies that make inferences from available information and context. Of course, I anthropomorphize the algorithms when I say they “fake understanding.” It would be more accurate to say they “simulate” understanding. But even that’s not right, especially in ML. These algorithms are designed by humans to implement problem-solving strategies. Some of the strategies look a lot like the ones that humans use while others look almost nothing like human cognitive strategies. Regardless, when we try to apply these algorithms to produce human-like results in a problem-solving situation, the output can seem similar to humans faking it or students cheating. Particularly with AI chatbots, we can see a mix of uncanny accuracy and bizarre mistakes.
Credit: Chatbot Life
This explanation is not deep enough to help educators make judgments about when and how to trust specific ML- and AI-based tools. To do that, they would need to understand a bit about the specific strategies that a given tool is employing, the kinds of mistakes it is prone to make, and the educational impact of those potential mistakes.
But it does provide a starting point. If we’re going to continue using the phrase “artificial intelligence” in a layperson’s context, then we need to start finding analogies that laypeople can understand. Humans cheating or otherwise faking understanding seems like one good place to start.
Artificial intelligence will soon be able to research and write essays as well as humans can. So will genuine education be swept away by a tidal wave of cheating – or is AI just another technical aid that teaching and assessment will evolve to take account of? John Ross reports[.]
Does AI Spell the End of Education?
This is an excellent article. I don’t mean that it is insightful or well-written. While it has its moments, overall, it’s an unenlightening mess wrapped in clickbait packaging. It is not good writing or good journalism.
But it is a near-perfect illustration of how the popular representations of both artificial intelligence (AI) and cheating can be harmful. ((The THE article also completely elides the difference between artificial intelligence (AI) and its cousin machine learning (ML). This is forgivable because the reader doesn’t need to understand the difference for the purpose of the piece. I’m not going to delve into the distinction in this blog post for the same reason. But I’m aware there is one. When I refer to AI, please read that as shorthand for the larger family of AI and ML techniques.))
It also shows a way for educators to understand AI better because AI and cheating sometimes work in similar ways. I will explain the parallel in this blog post. In the process, I will also argue that framing cheating in the context of “academic integrity” is harmful. And I will argue that all of these misunderstandings are counterproductive to preparing students for the future of work.
People who cheat are not “cheaters”
As you’ve probably figured out by now, I’m going to treat the THE article harshly. I’ll try my best to avoid the oh-so-tempting cheap shots. (The original working title for my post was “Does AI Spell the End of Education Journalism?”) The deeper problem at the heart of this article deserves serious treatment. I’m going to argue that “Does AI Spell the End of Education” is an example of journalistic “cheating.” In the process, I’m going to take a somewhat unconventional position on what it means to “cheat.” That position is relevant not only to how AI is used in the classroom but also to how we should think about AI and knowledge work and to how we should think about so-called “academic integrity.”
As part of that reframing, I want to be very careful to separate judgments about the writing from ones about the article’s writer, John Ross. I don’t know the man. I also don’t know the assignment he was given that led to him producing this article. I have no opinion of him as a writer or a human being. I only have opinions about the quality of this piece and the writing process that led to it.
I define “cheating” as “engaging in behaviors that are intended to facilitate passing without learning.” This definition avoids passing a blanket judgment on the person engaging in the behavior. It doesn’t accuse them of lacking “academic integrity.” It simply identifies behaviors that facilitate students getting good grades—which in the workplace we might call “scoring well on key performance indicators (KPIs)—without actually doing the hard thought work necessary to complete the assignment as intended. Any scoring system can be gamed. People game scoring systems for all kinds of reasons. One might be pressure. Perhaps a student wants to learn but needs to pass. Or a journalist wants to write an insightful piece but needs to complete a hugely ambitious assignment with an unrealistic deadline or word count limit. Sometimes we engage in sloppy or lazy shortcuts not because we are sloppy or lazy people but because we feel forced to do so by the circumstances. Whether in the classroom or the workplace, our primary focus should be on reducing the incentives to game the scoring system rather than on punishing “cheaters” for their lack of “integrity.”
From here forward, I will distinguish between John Ross, the human author of “Does AI Spell the End of Education?”, and the mental algorithm he employed to write this piece, which I will call Journobot 2000. These two are not the same. John Ross may very well be a smart guy. Journobot 2000 is a set of mental shortcuts that John Ross employed to avoid the hard work of thinking and learning when writing parts of his article. It does not understand AI, cheating, or the teaching of writing. It is capable of assembling passages about such topics in ways that sound coherent. It can even fool some intelligent readers into thinking that its output reflects some understanding of these topics. But Journobot 2000 does not understand anything. It is simply a sophisticated pattern-matching algorithm that can copy/paste in interesting ways and employs a souped-up thesaurus to rephrase sentences.
Journobot 2000 is a cheating strategy. It enables a writer under pressure to produce an article that sounds coherent without forcing that writer to invest the time necessary to understand the subject. When students employ Journobot 2000—which many do—they do not learn. When knowledge workers do the same, they do not perform useful knowledge work.
Knowledge work and learning are the same. Knowledge workers solve novel problems. How do they do that? By learning. Learning, in turn, requires thinking. Shortcuts that reduce drudge work are fine, but ones that reduce thought work are dangerous if your work requires you to think and learn.
Writing as collage
Journobot 2000 has assembled a series of quotes and facts related to the topics of AI, writing, and/or cheating in some combination. Before we analyze how it does this, let’s look at a few of the individual quotes from interviewees that appear in the article. I’ve arranged these out of order from their placement in the article for a specific reason. Think about each of these passages on its own and consider which issue or issues each speaker is concerned about.
I’ll provide fairly extensive quotes from every person to provide the flavor of their concerns. The first passage quotes Lucinda McKnight, a senior lecturer in pedagogy and curriculum at Deakin University:
“How do we prepare teachers to teach the writers of the future when we’ve got this enormous fourth industrial revolution happening out there that schools – and even, to some extent, universities – seem quite insulated from?” McKnight asks. “I was just astonished that there was such an enormous gap between [universities’] concept of digital writing in education and what’s actually happening out there in industry, in journalism, business reports, blog posts – all kinds of web content. AI is taking over in those areas.”
McKnight says AI has “tremendous capacity to augment human capabilities – writing in multiple languages; writing search engine-optimised text really fast; doing all sorts of things that humans would take much longer to do and could not do as thoroughly. It’s a whole new frontier of things to discover.”
Moreover, that future is already arriving. “There are really exciting things that people are already doing with AI in creative fields, in literature, in art,” she says. “Human beings [are] so curious: we will exploit these things and explore them for their potential. The question for us as educators is how we are going to support students to use AI in strategic and effective ways, to be better writers.”
And while the plagiarism detection companies are looking for more sophisticated ways to “catch” erring students, she believes that they are also interested in supporting a culture of academic integrity. “That’s what we’re all interested in,” she says. “Just like calculators, just like spell check, just like grammar check, this [technology] will become naturalised in the practice of writing…We need to think more strategically about the future of writing as working collaboratively with AI – not a sort of witch-hunt, punishing people for using it.”
Does AI Spell the End of Education?
That’s interesting. I agree with some of McKnight’s comments and have questions about others. For example, there’s an enormous difference between writing search-engine-optimized (SEO) text really fast and writing informative and well-written SEO text really fast. What is the relationship between the tool and the knowledge worker here? I have an SEO tool in my blog. It hates my writing. The feeling is mutual. If I followed its recommendations slavishly, I would have many more people coming to my site and many fewer reading it.
For now, the takeaway is that McKnight is interested in teaching students about how they might use AI text generation tools in the workplace. Let’s save further exploration of this line of thinking for later in this piece.
The next person in the article whose concerns I’d like to explore is Dr. Jesse Stommel, Digital Learning Fellow and Senior Lecturer of Communication and Digital Studies at the University of Mary Washington. Stommel is concerned about anti-plagiarism software. Here is how he is quoted:
“They have data about student writing,” he says. “They have data about how student writing changes over time because they have multiple submissions over the course of a career from an individual student. They have data where they can compare students against one another and compare students at different institutions.”
The next step, Stommel argues, is the development of an algorithm that can capture “who my students are, how they grow, if they’re likely to cheat. It’s like some dystopic future that is scarily plausible, where instead of catching cheaters, you are suddenly trying to catch the idea of cheating. What if we just created an algorithm that can predict when and how and where students might plagiarise, and we intercede before they do it? If you’ve seen Minority Report or read Nineteen Eighty-Four or watched Metropolis, you can see the dystopic place that this will ultimately go.”
Does AI Spell the End of Education?
Stommel is focused here on student data privacy, which can be a critical issue of certain applications of both AI and non-AI EdTech. While I don’t agree with his assessment regarding the plausibility of his nightmare scenario, I completely agree with the concern he is highlighting and would like to see it unpacked and explored. I could easily write an entire long post explaining which fears are realistic and why or why not. Notice, though, the concern Stommel expresses here isn’t about text generation tools or even AI specifically.
The third quote from the article that I’d like to highlight is from Andrew Grauer, CEO of Course Hero. He said,
“I’ve got a blinking cursor on my word processor. What a stressful, inefficient state to be in!” he says. Instead, he could use an AI bot to “come up with some kind of thesis statement; generate some target topic sentences; [weigh up] evidence for a pro and counter-argument. Eventually, I’m getting down to grammar checking. I could start to facilitate my argumentative paper.”
Does AI Spell the End of Education?
This, too, is interesting and worth exploring. When is this sort of support scaffolding that helps students learn, and when is it a crutch that helps them avoid learning? I did write about this topic as part of a larger post on scaling the digital seminar and could easily write more about it.
Grauer’s quote does seem related to McKnight’s. They’re both interested in how AI can scaffold writing. When John Ross interviewed people for the article that would eventually be named “Does AI Spell the End of Education?”, he did seem to probe his interviewees to foster a genuine dialog on this aspect of the article. He even introduces a quote from Turnitin’s Chief Product Officer Valerie Scheiner that acts as connective tissue between the two others. Here’s her relevant passage:
Turnitin is now using AI to give students direct feedback through a tool called “Draft Coach”, which helps them avoid unintentional plagiarism. “‘You have an uncited section of your paper. You need to fix it up before you turn it in as a final submission. You have too much similarity [with a] piece on Wikipedia.’ That type of similarity detection and citation assistance leverages AI directly on behalf of the student,” [Scheiner] says.
But the drawing of lines is only going to get more difficult, she adds: “It will always be wrong to pay someone to write your essay. But [with] AI-written materials, I think there’s a little more greyness. At what point or at what levels of education does using AI tools to help with your writing become more analogous to the use of a calculator? We don’t allow grade-three students to use a calculator on their math exam, because it would mean they don’t know how to do those fundamental calculations that we think are important. But we let calculus students use a calculator because they’re presumed to know how to do those basic math things.”
Schreiner says it is up to the academic community, rather than tech firms, to determine when students’ use of AI tools is appropriate. Such use may be permissible if the rules explicitly allow for it, or if students acknowledge it.
Does AI Spell the End of Education?
This seems to be a direct response to McKnight’s quote while nodding at some of the ethical issues raised elsewhere the piece. The most interesting part of “Does AI Spell the End of Education?” is the tension—and arms race—between text generation tools and plagiarism detection tools.
But the piece never quite manages to fully focus on this dilemma. It’s weirdly fragmented. There’s a one-sentence reference to “word spinners,” which are text paraphrasers that can be used to disguise plagiarism. But Ross never follows up on this angle, despite the fact that it fits perfectly with the dialog on text generation he’s assembled with the quotes from McKnight, Grauer, and Scheiner. Instead, he just supplements that one-sentence mention with a link to an article about word spinners on Turnitin’s web site. And then there’s Stommel’s quote, which is stuck in the middle of the piece and doesn’t seem directly related to the rest of the narrative. Student data privacy is not raised either before or after. The quote is just…there.
Why?
The answer is that John Ross, the human writer, cheated. This article seems like the result of a reporter who has interviewed a range of experts on the topic of AI in education as part of an effort to understand and report on the issues.
But it isn’t.
Several interviewees told me that they were interviewed months ago on topics other than AI and the teaching of writing. One of them, Jesse Stommel, went on record for me on this topic. He told me that he was originally interviewed about Turnitin’s acquisition of one of its competitors. While he does not object to authors using his quotes in other articles, he said, “[M]y quotes were not direct reflections on AI.” In fact, AI did not even come up in his interview.
When read with this in mind, the article makes much more sense. The most coherent parts of the writing were on threads that would have fit in the context of an article on Turnitin and anti-plagiarism software. The parts that get messy are precisely those where John Ross’s original research on a Turnitin story did not line up well with the purported topic of the article. For example, Stommel’s quote would have fit more naturally in the anti-plagiarism software piece because he was voicing concern about how anti-plagiarism software uses student data.
When John Ross decided to use some of the material from his original, never-published piece on Turnitin, he could have gone back to Stommel and asked him for questions that would have been directly relevant to the AI article. But he didn’t. Why not? I don’t know. Maybe he was lazy. Maybe he was under time pressure. Maybe his editors wanted something particular from him. I’m not going to judge the human being based on one article.
But I am going to judge his work on the article itself. For whatever reason, Ross fired up Journobot 2000. Rather than conducting further research, he took what he had already from a piece on another topic. He rearranged the pieces to look like they had always been intended to be parts of an article on AI. Journobot did so by following a simple pattern that I’ll analyze in the next section.
This is remarkably like the strategy students take of plagiarizing an essay on a similar topic to the one they’ve been assigned and then rearranging it to try and make it fit. The only difference is that he was plagiarizing himself. The problem here isn’t taking somebody else’s thoughts and claiming them as your own. It’s claiming to have thought about and analyzed a topic when you haven’t.
When students do this sort of thing, we call it “cheating.” It results in them failing to think and learn. When journalists do it, we call it “lazy journalism.” It results in messy articles that fail to enlighten the reader. More generally, when knowledge workers do it…well, we don’t have a specific name for it, but it results in low-quality work.
In data science, we call it “artificial intelligence.”
What cheating looks like
Journobot 2000 does not understand the relationship between Jesse Stommel’s data privacy concern and AI. It’s matching two kinds of patterns. First, since this is an article on a controversial topic, it represents controversy by alternating between quotes with positive sentiment scores and ones with negative sentiment scores. It’s simulating point/counterpoint. John Ross, the human journalist, could have chosen to leave out the hyperbolic end of Stommel’s quote and focused instead on the underlying concern. Journobot 2000 likely found that quote to fit its pattern-matching algorithm precisely because of the ending, which expresses a strong negative sentiment about something related to the topics at hand. It also knows how to write transitional phrases so that one passage appears related to the next.
Speaking of which, Journobot 2000 knows that anti-plagiarism software, AI, cheating, and writing are related topics. It organizes the quotes in ways that show relatedness among the topics. Because it doesn’t really understand the topics the same way humans do, a careful reader can see the seams where the piece doesn’t really hold together. But a casual reader might not notice that Stommel’s quotes have been spackled into places where they only loosely fit with the analysis that comes before or after. He’s not really part of the dialog in the same way that some of the others were.
Likewise, there’s that largely unutilized reference to word spinners. In an article about Turnitin, the topic might have only made sense to mention as one of many aspects concerning the company and its acquisition of a competitor. But in an article about AI potentially ending education, word spinners should have received significant attention. John Ross might have seen that and researched accordingly. Journobot 2000 did not make the connection.
Let’s pick up on a couple of the threads missed by Journobot 2000 to get a sense of the article that could have been if John Ross had applied the same level of attention that the archeological evidence in his published piece suggests he put into the original, unpublished version.
Articles written by actual machines
Let’s start with the wonders of machines writing articles. You have almost certainly read articles written by a machine. For example, if you follow stocks, you may have already learned to recognize the articles written by bots. Imagine a massive drop in the stock price of a biotech stock because they had bad clinical trial results. You might read a perfectly well-written financial news story in your inbox, telling you all about the technical indicators on the stock price, complete with a headline suggesting the article will provide insight as to whether to buy or sell…but no mention whatsoever of the news that drove the price move. The technical analysis is data-driven and seems perfectly cogent. The writing has just a dash of colorful language, suggesting the barest hint of a simulated authorial voice. If you didn’t know about the news, it would seem normal. But it’s not really a financial analysis news piece. It’s a data analytics report written in narrative form with a formulaic headline tacked on the top. The machine doesn’t really understand the topic it’s writing about.
In this example, there may be little to no actual artificial intelligence involved in the writing. A human might have written a template covering the topic of a certain type of stock movement. The software fills in the data. It has been provided with a handful of colorful phrases to substitute for different common phrases. “The stock took a nosedive.” “The stock tanked.” “The stock plummeted.” These can be interchanged randomly to create the appearance of an author behind the piece.
Genuine AI can generate original writing using a family of techniques called Natural Language Processing (NLP). A particular product called GPT-3 produced by a company called OpenAI is getting most of the buzz right now, but there are others. It can produce uncanny writing. By which I mean writing that falls in the uncanny valley. It’s writing that seems sort of human but not quite. The result is weird and sometimes creepy. (To get a delightful sense of just how weird and creepy, read Janelle Shane’s blog AI Weirdness. And then read her book, You Look Like a Thing and I Love You: How Artificial Intelligence Works and How It’s Making the World a Weirder Place.)
A recent article on NextWeb, “Don’t mistake OpenAI Codex for a programmer,” is illustrative. It’s all about how the Microsoft-owned Github software repository platform took a highly customized version of GPT-3 and trained it to write computer code. The idea is that if GPT-3 can learn English, then it should be able to learn Javascript. Programming languages are languages, after all.
A good part of the article is devoted to the No Free Lunch Problem, “which means that generalization comes at the cost of performance. In other words, machine learning models are more accurate when they are designed to solve one specific problem. On the other hand, when their problem domain is broadened, their performance decreases.” Even an enormous, computationally expensive, state-of-the-art AI program like GPT-3 is mediocre at performing a wide range of tasks. Developers invest enormous time and energy tuning it to do one thing really well. And even then, “really well” isn’t always…um…all that well. Here’s the money quote from the piece:
In their paper, the OpenAI scientists acknowledge that Codex “does not sample efficient to train” and that “even seasoned developers do not encounter anywhere near this amount of code over their careers.”
They further add that “a strong student who completes an introductory computer science course is expected to be able to solve a larger fraction of problems than Codex-12B.”
Don’t mistake OpenAI Codex for a programmer
While I don’t know how much money Microsoft spent on developing Codex, I’m confident it cost at least several orders of magnitude than the typical EdTech AI. And yet, it can’t match a first-year computer science undergraduate.
Why not? The piece goes into some technical detail, but it boils down to the fact that today’s AI still has some sharp limitations relative to humans when it comes to problem-solving. It can’t hold as many relevant facts in its “head” as we can. It doesn’t match patterns in the same way. It’s not as good at catching nuances of meaning in language and relationships among ideas. While the progress being made in AI today is miraculous, it’s not biblically so. It’s not magic. If one of the most expensive and technologically advanced algorithms in human history can’t match a first-year college student, then we should probably let go of the breathless hyperbole about AI “ending education” for a while.
Rather than employing Journobot 2000, John Ross could have engaged his full human faculties as a learner, thinker, and knowledge worker to engage with the purported topic of his article. He has many of the raw ingredients for something genuinely interesting. But he didn’t take the time to follow the threads.
Word spinners are another example.
Spinning words
John Ross’s article mentions “word spinners”—tools that rewrite sentences using AI—as cheating tools to get around plagiarism detectors. But it doesn’t name any or explore the topic in detail. The most he does is link to an article about word spinners on Turnitin’s website (which is probably another artifact of the original article).
In the absence of John Ross’s due diligence, I conducted a little of my own by employing an advanced AI research tool called Google. It turns out not all word spinners are the same. For example, Rewriter Tools Article Spinner all but explicitly advertises itself as a tool that is designed for cheating:
Today, almost everything is done online – including work assignments, student essays, and anything else you can think of. As a result, a large amount of written work also has to be done online.
The problem is that so much has already been written about pretty much everything, that creating completely new and unique content is quite difficult. Not to forget, also time-consuming and rather tiring, too. As a result, many people get confused and frustrated while trying to create unique content.
Do you want to create original, fresh content but are pressed for time? Rewriting a document to make it unique is not always an easy task. This is why we present you with Article Spinner – the perfect to help you create fresh content in very little time.
Probably some bot
Ladies and gentlemen, welcome to the future of knowledge work! Papers that are badly rewritten by a tool created by a bad writer because thinking is too hard and who has original ideas anymore anyway?
The future of work?
On the bright side, their search engine optimization algorithm must be good because this text put them near the top of my search results page.
Quillbot, on the other hand, positions itself as a tool that helps writers tune their language to their audience:
Your words matter, and our paraphrasing tool is designed to ensure you use the right ones. With 3 free modes and 4 premium modes to choose from, QuillBot’s paraphraser can rephrase any text in a variety of different ways, guaranteeing you find the perfect language, tone, and style for any occasion. Just enter your text into the input box, and our AI will work with you to build the best paraphrase from the original piece of writing.
A slightly more sophisticated bot
Is that better than Article Spinner? I think it may be worse. First, it appears to be more sophisticated at rephrasing other people’s work. When McKnight talks about the Fourth Industrial Revolution and AI helping humans do their jobs better, I don’t think she means AI helping college students take pieces written by somebody else and paraphrasing them in varied ways to pass a plagiarism detector.
Siri, make this plagiarized essay sound more friendly.
Second, again, I’m having a hard time coming up with legitimate use cases that aren’t just shortcuts to avoid thinking. I use a grammar checker that makes style suggestions—more on that momentarily—but it doesn’t wholesale rewrite for me. Instead, it highlights choices that I can make as a knowledge worker. Quillbot calls itself a “paraphraser.” (Side note: Judging from the text on both sites, I’m guessing that “paraphrase” may be a good SEO term for both products.) Maybe there are some legitimate uses for a tool that can quickly paraphrase a longer document. If I write a follow-up post to this one, I may try using it on a previous post to see if anything useful comes out.
Then there are grammar checkers, which are mentioned but—again—never explored in “Does AI Spell the End of Education?” I use Grammarly Premium regularly. In fact, I am using it right now. It helps me catch mistakes and write clearer, punchier prose. Even though I am a pretty good writer, Grammarly improves almost everything I write (when I use it). But it is only useful to me because I know when—and why—I should ignore or overrule its suggestions. If I were to ask students in a writing class to use it, I would have to teach them to do the same. The problem is that I don’t know how Grammarly works. I can’t teach students how to anticipate all the mistakes it might make.
This is particularly true with students who have language patterns that Grammarly might not anticipate. For example, second-language learners whose native language is Chinese or Russian may write English sentences that drop certain types of words (like articles or pronouns), mix up verb tenses, mess up idiomatic expressions, and change the word order. And even fluent second-language learners may make mistakes that the grammar checker won’t diagnose correctly when the writers are stressed, such as when they are trying to express difficult ideas while writing under time pressure. In combination, these problems could confuse a grammar checker and cause it to make a bad suggestion.
As a result, I would have to think hard about whether, when, and how to use Grammarly as a teaching tool, even if I believed it would help most students improve their writing the majority of the time. As a writing teacher, my job isn’t to get students to produce better writing. It’s to teach them how to be better writers. As a writer, while I use Grammarly to help me edit my text more quickly and effectively, I also use it to help me make mindful decisions about when to break the rules. Good writers balance clarity against expressiveness all the time. Sometimes I override Grammarly not because its suggestion is wrong but because I have chosen to write a more challenging sentence to read to communicate a challenging idea more effectively.
I would have liked to read a researched article on this topic. I suspect John Ross could have written it. Journobot 2000 cannot.
The bottom line
The future of work is knowledge work. Knowledge work and learning are the same. Therefore, if we want to prepare students for the future of work, we need to teach them how to think and learn. Cheating is behavior intended to achieve a passing grade without learning. Cheating is bad because it leaves students ill-prepared for the future of work (not to mention for life). Tools or strategies that help knowledge workers (including students) avoid mindless work are probably good more often than not. Tools or strategies that help knowledge workers avoid thought work are probably bad. More often than not.
“Does AI Spell the End of Education?” raised (but did not explore) authentic assessment as one way out of the cheating problem. While I’m a fan of authentic assessment, the article itself is proof that it is not a panacea. Because it is, in fact, an authentic assessment of John Ross’s writing. As a writing portfolio artifact, the piece shows that the author could pass, i.e., get his article published, without learning anything new about the promise and perils of AI in education.
Many decent educators have faced the challenge of trying to break students out of algorithmic behaviors that have enabled them to pass without learning, whether the behavior is writing a robotic five-paragraph essay or memorizing physics equations without understanding them. If cheating is the set of behaviors designed to succeed without learning, then these behaviors, which have been taught to students as perfectly appropriate, are cheating just as much as copying somebody else’s answer is. It matters in the classroom, it matters in the workplace, it matters in the home, and it matters in the ballot booth. I hope the next article I read about AI and cheating will be about applying AI to solve that problem.
We’re keeping a somewhat irregular summer schedule on Blursday Socials to allow for the customary socially distanced games of Marco Polo as people rush out of their COVID caves and into the world at large. But that doesn’t mean we’re stopping!
We have three great guests coming up:
7/15: Sasha Thackaberry, LSU’s VP of Online and Continuing Education: Prior to LSU, Sasha worked at SNHU and at Cuyahoga Community Colleges. So she has a range of perspective that is rare. She’s also hilarious, so make sure you’re not drinking while she’s talking unless you want liquid up your nose. RSVP.
7/22: Paxton Riter, CEO of iDesign: After growing up in the OPM world, Paxton has been running one of most widely recognized names in post-OPM, design-led fee-for-service companies. I’m interested in his take on how things are shaping up post-pandemic. And also in what he’ll be drinking. RSVP.
7/29:Emily Foote, Principal at Osage Venture Partners: I met Emily shortly after she sold her wonderful EdTech company Practice to Instructure. It was at least a three-beer conversation (and I’m a slow drinker). Now an EdTech investor, Emily brings the passion of an educator and the experience of an entrepreneur to her new(ish) gig. I’m curious to get a sense of how she’s processing what she’s learning given her broad experience. RSVP.
I always try to invite guests that I would seek out to have a drink with at a conference (as opposed to those I would run away from or grudgingly endure). These three folks are all on my shortlist of the good ones.
I’ve been inundated with questions regarding what I think about 2U’s acquisition of edX. What do I think? I’ve been struck by how much less I care about this deal today than I would have a few years ago. That change is entirely due to my change in focus rather than external circumstances. e-Literate used to be, in part, an EdTech industry analysis site for its own sake. While I still do some EdTech industry analysis, I’m much more focused on how EdTech influences the direction of the education sector as a whole, particularly with regard to becoming more effective at sustainably helping more students. Due to that shift in perspective, the edX acquisition moves from “a huge deal” to “somewhat interesting” for me.
How much you should care about the edX acquisition depends on what you care about. So I’m going to write about that.
Who cares about the edX acquisition?
You can learn a lot about whether you, personally, should care about the acquisition by looking at who else cares and why. It’s a little early to answer this question entirely; I’m asking around at the moment. But we do have some early obvious answers.
MIT and Harvard
First of all, MIT and Harvard care. On one hand, edX was a money loser. Most EdTech companies are. They lose money for a very long time and then only become slightly profitable. Investors can be OK with this if “slightly profitable” is also “reliably profitable.” They can take a long view, in a way. I say “in a way” because many types of investors that put money into an EdTech company on its way to profitability sell their stakes long before their investments achieve their goal. As long as a critical mass of investors believe that these companies will eventually be profitable, then one investor may well be willing to buy a stake in an unprofitable company from another.
On the other hand, universities aren’t in that game in the same way. As far as I’ve been able to piece together, edX was rushed out the door to get ahead of the imminent launch of Coursera. It wasn’t a strategy. It was a reaction. It was also a money loser. According to Tech Crunch, “The institutions, of course, have thrown in a cumulative $80 million in donations into edX to keep the operation free.” That’s a surprisingly shallow and naive assessment from a publication that’s all about tech companies. ((I also think their contribution number is low, but it’s hard to find hard data.)) First, edX fees to students (or lack thereof) have more or less aligned Coursera’s. Second, both edX and Coursera are two-sided markets. While edX ultimately makes its money off of student purchases, it does so via a revenue share agreement with the universities, which, once again, is not terribly different from Coursera’s revenue share agreement. The basic idea in both cases is to be the Amazon of MOOCs. Get everyone to sell everything through your storefront. Collect a small dollar amount (but a large percentage) of each transaction as your fee. Attract enough customers so that the small dollar amounts add up over time. To the degree that edX had a business plan beyond “do whatever Coursera does,” this was it. It wasn’t obvious that edX would be an asset they would ultimately sell. Their external justifications for edX were always mission-related. Since I have only been able to gather fragments of information about their internal deliberations over the year, I’ll take them at their word on that. I don’t know the degree to which they understood that their commitment entailed losing money every year for the long haul.
When 2U swooped in, that gave MIT and Harvard an opportunity to get out of that money trap, declare victory, and make a handsome return on their investment. If Tech Crunch’s numbers are right, then the two universities made 10X on their 9-year investment. Is that good? It depends on your perspective. VCs generally look to make a 10X return in five years, and I suspect that those numbers may be less ambitious in EdTech specifically. For MIT and Harvard, I suspect it was a massive unexpected windfall that got rid of some problems and created some opportunities for them.
2U
Obviously, 2U wouldn’t have forked over $800 million—in cash—if they didn’t think edX would be a big deal for them. Why? Phil Hill writes:
Coursera’s market value is roughly 18 times its annual revenue whereas 2U’s is roughly 4. These are rough numbers, but I believe 2U’s leadership believes it an command an increased value as this deal completes. Note that I am not predicting stock prices here, just showing the potential change in market perception.
But that only pushes the question back a level. Why do investors think that Coursera is so much more valuable than 2U? The short answer is that one of the most expensive parts of an OPM business—and Coursera definitely is an OPM, as this transaction demonstrates—is marketing for new students. Investors believe both Coursera’s and 2U’s claims that owning a MOOC business helps lower the marketing costs for their core OPM businesses. 2U’s public estimates are that they can save 15% on marketing costs. I’m somewhat skeptical of this claim but I’m also not a financial analyst, so I’ll take it at face value.
2U has a handful of other potential business justifications, most of which I won’t break down here because, again, it’s not the focus of my writing anymore. I’ll briefly share a few of them, not because they’re the most important but because they’re illustrative and easy to explain succinctly. First, 2U has always aspired to transform the entire education sector by bringing it online. However you may feel about that aspiration—I recognize that feelings tend to run very hot about OPMs—owning a major MOOC platform gives the company’s aspiration more depth.
All roads lead to….
Second, edX reaches a lot of students in a lot of countries. The international EdTech market has been a long time coming, but it’s finally arrived. edX greatly expands 2U’s international footprint by some measures, which once again gives them a good story to tell.
And this brings me to the final advantage I’ll point out in this post. 2U has always been about telling a compelling narrative about the future. One of Chip Paucek’s previous ventures was a company that had comedians explain educational concepts on a television show. He understands how to build a story. He knows how to hit the beats. When WeWork was at peak hype, he made a deal with WeWork. When code academies were red hot, he bought one of the hottest code academies. Given Coursera’s recent success in the markets, it makes sense that he would look to make the biggest, boldest MOOC move possible.
To be clear, I’m not saying that the CEO of this publicly traded company made these deals solely or primarily to spin a good story. I’m simply pointing out that Chip has a method for responding to market changes. If he believes that getting into a particular business is good for 2U, he will not make his move by quietly dipping his toe in the water. He’s going to jump in with both feet and make a big splash in the process. The edX acquisition fits with that method.
2U’s and edX’s university partners
While it’s too early to make pronouncements with any confidence, early reactions I’ve heard indicate that 2U’s university partners are pretty happy with the transaction while edX’s university partners are pretty unhappy. For 2U’s partners, they already decided to go with the big publicly traded corporation that, for better and worse, is heavily associated with revenue-sharing deals. Now they can do MOOCs with the same company. This deal gives them nothing but upside. On the other hand, many edX partners specifically went with edX because they did not want to deal with the for-profit Coursera. Yes, edX had its revenue-sharing agreement too, but it was a non-profit run by universities. That made it feel different for some.
2U may have to deal with some of the kind of backlash that Blackboard did when it bought other LMS companies—particularly Moodle support companies. They may lose some universities. Then again, 2U is definitely not Blackboard. Especially Blackboard circa 2012 (although, ironically, edX was formed the same year Blackboard acquired Moodlerooms). 2U built its business by winning over faculty senates. Also, while the company may lose some edX customers, it may gain some by cross-selling to its existing customer base. It’s hard to say how all this will play out, net-net. All I can say with confidence right now is that people at the edX schools I’ve talked to so far are understandably nervous.
Who doesn’t care
I doubt students will even notice. I take that as one strong indicator of how I should feel. It’s not clear to me that Coursera has done anything edX hasn’t and that would or should concern students. I could be wrong; I haven’t looked closely at this aspect. (Please correct me in the comments section if you know something I don’t.) Further, I have no reason to believe that 2U’s behavior will be worse than Coursera has been.
MOOCs strike me as a relatively low-risk corner of EdTech for heavy corporate involvement. My sense is that, if 2U sees edX primarily as a way of making marketing dollars go further, then they have a vested interest in keeping students happily engaged. Yes, they’ll use student data to target them for marketing, but if you are shocked by that, then you should maybe take a look at all free products that you use (possibly including the email service that you are reading this blog post through). As long as the MOOCs are transparent about what they’re doing, it’s probably OK in that particular market. In fact, those particular learners may want to receive targeted ads about other learning opportunities.
The ways in which I care
In and of itself, I’m indifferent to this deal. I’m not opposed to revenue-share OPMs in general or 2U in particular. (This is an outdated argument anyway since the major OPMs generally offer non-revenue-sharing arrangements of various flavors these days.) At first blush, I don’t see any big harm to students or institutions in it. (I reserve the right to change my mind in either direction as I learn more.) Since xMOOCs have not turned out to be the end of academia as we know it, either in the revolutionary sense or the armageddon sense, I’m inclined to feel mildly positive toward an arrangement that gives a major provider a sustainable path forward. MOOCs are one more arrow in the quiver as we try to offer everyone in the world the opportunity to fulfill their potential through education. I’m not going to turn my nose up at that. I like the folks I know at both edX and 2U. While I don’t always agree with them, I’d like to see them make a contribution with their new venture. I’ll wait and see and wish them well.
Beyond that, I mostly care about two aspects that I haven’t seen talked about much in any of the coverage. First, there’s the open-source code. While I frankly think the early iterations of OpenEdX were embarrassingly bad and improved over time to “surprisingly OK given how embarrassingly the foundations were,” I do think there is value in maintaining an open-source MOOC platform. I am skeptical that 2U has the DNA necessary to steward an academic open-source community. I know how good 2U can be at working with academics and I also know what it’s like to steward an academic open-source software community. These two things are not the same.
Second and more importantly, I’m worried about the loss of research. Thanks to the efforts of researchers like Justin Reich and Rene Kizilcec, edX has been one of very few public testbeds we have for conducting credible learning efficacy research at scale. 2U, in contrast, has done nothing visible in the area of learning science. They’ve started talking about it in the past year or two, but frankly, if e-Literate were still doing the cop-on-the-beat thing, I probably would have shredded them about it by now. In 2021, there is absolutely no excuse for any EdTech company of 2U’s (or Coursera’s) size and scale not to be engaging actively with academics in serious applied learning science and contributing to our collective knowledge. If 2U can spend $800 million—in cash—for a MOOC organization that loses money every year, the company can surely afford to invest one-half of one percent of that every year in a credible program to advance the state of knowledge and literacy in effective teaching practices. And now that they own a platform for conducting such research at scale, the onus on them has only increased.
The same goes for MIT and Harvard, by the way. Despite the excellent work of a few researchers, and despite the rhetoric of the institutions at the time that edX was launched, one reason we have not gotten more and better research out of edX is that the platform, incredibly, was poorly designed for educational research. How did MIT build a platform for massive-scale learning in 2012 and fail to think about what sorts of educational data and metadata they would need to facilitate research? What does that tell us about the real priorities behind the initial push to production? It’s a mystery.
I’m not particularly interested in the vague promises of two rich universities to do good in the world with their $700 million windfall from a non-profit that was supposed to educate the whole world. I’d like to see a credible plan this time, including a theory of change.
Lately I’ve been thinking a lot about the problem of fragmented digital learning experiences. The pandemic really highlighted how digital learning journeys are fragmented across multiple tools—an LMS, maybe some tools that plug into the LMS and take the students somewhere else, maybe some digital courseware, maybe a web conferencing tool, and so on.
This fragmentation creates a few serious problems. First, every hop a student (or educator) must make from one platform to another represents a chance to get lost. Where am I supposed to go? Do I need a separate login? What am supposed to do here? How does this thing work again? And so on. While we live in a world of phone apps where people are used to switching from one to another, the mobile app switching experiences differ in two respects. First, they are relatively well orchestrated. For example, when I launch a web conference from my calendar app, not only does it take me smoothly into the new app with one click, it also maintains a back-link to the app that got me there. Second, the workflows that seem natural are simple and atomic. When I’m hopping once from one app to another and back, I’m fine. But if I’m performing a complex work task that requires me to switch between Zoom, Slack, Google Docs, and maybe a browser, I lose my way easily and often. I spend a non-trivial amount of my day hunting for the correct browser tab in the correct browser window, to the point where, by the time I’ve found what I’m looking for, I sometimes forget why I was looking for it. If this feeling of getting lost among open windows and tabs is familiar to you, too, then you have a visceral sense of the problem in our digital learning experiences. This is the cognitive load we are placing with students who may also be struggling with unfamiliar material, learning new study skills, working in an environment with distractions (like kids and pets), and so on. Again, the pandemic has brought this problem home to all of us.
The impact on individual learners is bad enough. But the fragmentation also prevents us from having a view into what the learners are doing. In the virtual workplace, it’s easy to lose a sense of what your coworkers are doing in a project—and what they need from you—because you keep losing track of where the Google Doc is that they’re working in. Or maybe you’ve even forgotten that doc exists because it’s completely out of sight for you. In an educator’s context, this means that you often struggle to keep track of how students are doing, and they often struggle to keep track of what they should be doing.
When we try to understand why such a problem exists—not generically, but for a particular common workflow that feels like it should have a better solution by now—we often find ourselves going down a rabbit hole. There might be a missing technical specification. And maybe that technical specification is missing because there aren’t business drivers for software companies to coalesce around that specification. And maybe the business drivers aren’t there because the customers make purchasing decisions in a certain way. And maybe they make their purchasing decisions the way they do because of their internal organization and culture. Or laws and regulations. Or both. And so we go deeper and deeper into the rabbit hole. How many conference panels have attended that end with you shrugging your shoulders at the seemingly intractable complexity that the panelists have brought to light?
One of the jobs I take on at e-Literate (and more generally in my professional life) is to selectively go down the rabbit holes that may actually lead somewhere other than further down. I don’t care much about technical specifications, business models, university procurement processes, faculty professional indoctrination, and so on for their own sake. I care about the educational problems that those factors influence. It’s a two-step process: (1) find the subset of hard problems that are not impossible to solve at the moment, and (2) look for an answer that is as simple as possible but no simpler. I often don’t get past step one, even with multiple lengthy posts. These are the sorts of problems that require multiple stakeholders to solve in concert. Sometimes the first step toward solving them is getting everyone to see them in roughly the same way.
We are at a moment now where the rabbit hole of digital learning experience fragmentation may more tractable than it was before. Not easy, but perhaps possible. While there are a variety of reasons why this is true, the most obvious one is that our shared experience of pandemic life has created the widespread and direct experience of digital frustration that has engendered increased empathy and urgency. More of us get the problem because more of us have lived something like it. So deep, in fact, that I’m going to draw on blog posts that are 15 and 16 years old. I confess this is uncomfortable to me. It’s a little like taking out your journal from middle school and reading it at a party.
EE litter it
But 2005 – 2008 was the period when the LMS began a very important step toward solving the fragmentation problem. It’s worth taking a look back to see what’s changed, what hasn’t, and why some problems turned out to be harder than others.
Adding windows to the walled garden
Going back to one of the earliest posts on e-Literate, I wrote this in 2004:
The analogy I often make with Blackboard is to a classroom where all the seats are bolted to the floor. How the room is arranged matters. If students are going to be having a class discussion, maybe you put the chairs in a circle. If they will be doing groupwork, maybe you put them in groups. If they are doing lab work, you put them around lab tables. A good room set-up can’t make a class succeed by itself, but a bad room set-up can make it fail. If there’s a loud fan drowning out conversation or if the room is so hot that it’s hard to concentrate, you will lose students.
A good [LMS] allows for flexibility in classroom set-up. For example, when I used [a now-defunct open-source LMS]…I was able to meet with stakeholders and try different arrangements of the virtual chairs until we found one that they were comfortable with. The default set-up, which was optimised for on-campus students who took four or five courses at once and needed an aggregation portal, was completely unsuited for my audience, which was a group of full-time bond traders and other financial services people who were taking time out of their twelve-hour work days for one course. It wasn’t worth their time to learn the complex (cluttered?) interface that made sense for a full-time college student. So we streamlined. We turned off functionality that we didn’t need. We renamed pages to fit the students’ expectations. We re-arranged portlet windows on pages and page order on the interface. We did all of this in 15 minutes without any programming.
In Blackboard, you can’t do that. Sure, you can change the way the buttons look. And you can hide a button. But that’s it. It’s essentially only trivially configurable by the instructor. I strongly believe that this has a significant impact on distance learning drop-out rates.
That was the state of the LMS in 2004. You could turn menu items on and off. That was the extent of customization. Over time, these products became more flexible, to the point where at least some of them—I’ll call out Brightspace here, because I’ve seen this flexibility in action fairly recently in that product—are incredibly configurable. Of course, there’s always a trade-off between configurability and ease-of-use, particularly when you’re adding configurability to an existing product with an installed base that expects it to work a certain way. But we’ve seen progress on this first problem over the past 17 years.
This was also the period when both LMS usage was growing and we began seeing the growth of web 2.0 tools. (“Web 2.0 was a term first used in public at the O’Reilly conference in late 2004, according to Wikipedia.) So we went through a moment when LMS companies started adding half-baked blogs and terrible wikis to their products. It became instantly clear that almost nobody wanted to use these LMS-internal tools.
But it quickly became apparent that educators wanted an increasing number of specialized tools into their pedagogical workflows. Here’s a story from a 2012 post but which actually happened in the 2005-2006 time frame:
I often tell the story of when Beth Harris and Steven Zucker (formerly of FIT but now of Khan Academy) took me to see an image annotation tool developed by Columbia University that they were excited about. They were looking for a tool for teaching art history online. Columbia’s tool was really cool, but it was developed for a histology professor. It turns out that the way histology professors want to use and annotate images in the classroom is completely different than the way art history professors do. Some of these may not be sustainable as commercial applications and may work better as non-commercial open source. But, for example, teaching good writing is a pretty large niche application spanning multiple disciplines and should support significant commercial efforts.
This passage hints at the rabbit hole problem I referenced at the top of the post, but the main point is that a long tail of learning applications was beginning to develop to accommodate subject-specific learning journeys.
The trouble was that instructors had no way to integrate these into the LMS, which was the main—and in many cases, only—platform available to them for creating digital learning experiences.
I was getting increasingly involved with the Sakai LMS community around this time, in part because it seemed like a place to make progress on this problem. Here’s something I wrote in 2007, the year before IMS Global released LTI 1.0:
Sakai could foster inter-institutional resource-sharing at the level of hosting an LMS, it could also host resource sharing at the level of individual tools. As Chuck Severence pointed out in a relatively recent blog post, there are now at least three methods for integrating tools into Sakai using web services. In addition to lowering the barrier of specialized Java skills, and indeed removing the requirement that your tool be written in Java at all, these methods open up the possibility of remotely hosted tools. Add WSRP and SAML support into the mix, and developers truly have a wide array of options for developing remotely hosted tools.
I would like to see these techniques refined to make Sakai the absolute best [LMS] for developing remotely hosted tools. I would like to see new support models that enable system administrators to say “yes” more often when faculty members ask for specialized tools. I would like to see new economic models that make it feasible for resource-rich universities developing specialized teaching applications to share them with even the poorest of their peers in the Sakai community, rather than keeping the use of those tools confined to the few professors within the developing institution that had the money to do the work (as happens much of the time today). I would like to see the Sakai Board seek funding for developing this newer, richer kind of open educational resource. The particular strengths of the Sakai community uniquely position it to accomplish these goals.
If you’re technical enough to know what WSRP, SAML, and Java portlets are, you’ll know that these were pretty labor-intensive and clunky solutions to the long tail integration problem. LTI was much easier to implement and was a direct driver of the explosion of learning tools that are available today for integration into LMSs. This list shows 463 apps that can be integrated with an LMS via LTI today.
But integration via LTI itself mostly means single sign-on and grade return. Depending on how the integration is implemented and how many integrations are used, we still can easily have the lost-in-too-many-windows-and-tabs navigation problem and the I-can’t-see-what-you’re-doing problem. We have integration but not true interoperability.
Here’s a bit I wrote in 2005 from a speculative piece about an idea that some colleagues and I were (mis)labeling a “Learning Management Operating System (LMOS) Service Broker”:
Now, it turns out that RSS feeds carry quite a bit of information that could be useful in a learning context. Here’s a list of just a small subset of the information available from my RSS feed, for example:
The URL of my blog’s home page
The ID of the blog’s author (in my case, it gives my email address as my ID)
The software application that generated the posts
The URL, title, description, contents, time stamp and category labels for each post
Notice two things about this data. First, it’s very generic and could be useful information about just about any content online. Second, the post-specific information (in the last bullet point) is pretty much exactly what you need to know about any assignment that a student submits for a class.
In order to start making use of this data in the context of an LMOS, the blog developer need only make a few relatively minor technical enhancements in order to plug into the service broker:
Write an adapter that enables the RSS feed to talk to the broker. (Since RSS is a very common format, chances are good that such an adapter would already exist.)
Tie the blog into the single sign-on mechanism, so that the LMOS knows that the person that owns a particular blog is also, say, a student in the Psychology 101 class.
Extend the blog to be able to subscribe to category labels that are related to the groups to which the student belongs (e.g., the Psych 101 student should see a “Psych 101” category tag show up in her post category list).
Now we’re ready for some service broker automagic. Let’s say the student decides to write a blog post on a topic related to her Psych 101 class. As she writes her post, she looks over to the category list. Because the system knows that she is a registered student in Psych 101, it automatically adds “Psych 101” to her category list. She selects the appropriate category heading(s), writes her post, and publishes. The service broker, seeing that the content is labeled “Psych 101”, announces to all the applications within the Psych 101 course environment that it has some student-created content. “Can any of you applications do anything with this student-created content?” it asks.
It gets the following responses:
The class RSS aggregator responds, “Yeah, I can do something with it.” It takes the student’s post and publishes it along with those of the other students in the class.
The course activity tracker says, “Me too. Gimme some of that.” It notes the student ID, the time stamp, and the title and URL of the post. Using this information it adds an entry for the student’s class activity on that particular date.
The grade gook says, “I can also use that.” Noting that the content is generated by the weblog application, it pulls the post text and URL into the student’s row in the gradebook under the “weblog entries” heading. The instructor can now assign a grade and comment to it.
Notice that the weblog developer didn’t have to write separate integration code for course activity tracker and grade book. The service broker was able to integrate the new application on-the-fly because the blog publishes the basic required knowledge in a standard format. All the blog developer had to do was write a connector that picks up the categories from the system and works with its single sign-on mechanism. The broker does the rest. It would be the same for any other application, too. You could, for example, use more or less the same mechanism to integrate your discussion board with the grade book and the course activity tracker.
But wait. There’s more.
Suppose we make one more minor enhancement. Suppose that individual applications within the course environment could publish categories to share with each other. Suppose, for example, that the grade book could publish a category corresponding to a particular assignment. Our student could select that particular assignment category for her post and the instructor would automagically have it show up in the appropriate grade book column, with the appropriate point scale and weighting, and so on. Let’s imagine, too, that you could set your discussion board to generate a forum topic for particular category (such as the assignment heading in the grade book) and generate a new thread for each post that comes in labeled with that category. Students could continue to post to their personal blogs that travel with them beyond the class, but the instructor could also create class-internal discussions based on those posts. This is all done using fairly generic mechanisms, so developers creating new applications won’t need to do anything special to integrate their new wiki, or simulation, or flux capacitor, or whatever with individual applications already in the course environment.
But wait. There’s still more.
Suppose that, in addition to having students publish information into the course, the service broker also let the course publish information out to the student’s personal data store (read “portfolio”). Imagine that for every content item that the student creates and owns in her personal area–blog posts, assignment drafts in her online file storage, etc.–there is also a data store to which courses could publish metadata. For example, the grade book, having recorded a grade and a comment about the student’s blog post, could push that information (along with the post’s URL as an identifier) back out to the student’s data store. Now the student has her professor’s grade and comment (in read-only format, of course), traveling with her long after the system administrator closed an archived the Psych 101 course. She can publish that information to her public e-portfolio, or not, as she pleases.
[Editor’s Note: Sorry for the formatting glitch; the newish WordPress block system still doesn’t handle nested layout formats very easily or intuitively.]
It’s the meaning in the middle
The unsung hero in the example above is RSS, which stands for “Really Simple Syndication.” It carries a lot of important information about a document, including but not limited to who wrote it, when they wrote it, what topic(s) it was related to, and what the title is. That information is valuable regardless of whether the document being shared is a blog post, a journal entry, an essay, or a collaboratively authored Google Doc. (There’s nothing in the format that limits us to one author.) Atom, which is a slightly newer cousin to RSS, also supports threading. With that additional information, we could share discussion posts and pull in contextual information about the discussion threads that the posts are in.
If we think a little bit about how we want to use the metadata, we can think about how we want to write and read to the app creating the document. We might want to supply an assignment name, a learning objective, or a group of collaborators from a class. The options become quite rich without us having to change anything about the metadata structure. It turns out that we share a lot of different types of narrative documents which share common traits. By making those traits incomprehensible to our learning platform, we can build a fundamentally more legible, navigable, and sensible learner journey with accompanying data that enables the students’ educators to act as guides in ways that are difficult in today’s digital learning environments.
I honestly think that if we took the RSS/Atom structure and added in a representation of pedagogical intent in our curricular materials design, as I wrote about in January, we would cover quite a bit of ground in terms of creating a more seamless and supported learning journey.
Furthermore, this kind of interoperability creates more entry points for views into that learner journey. If the data across the platforms can be tied to a coherent pedagogical pathway, and if we have structures for getting permission to see those data, then we can view the relevant parts of that journey from anywhere that make sense to view it from. If, for example, you’re in class using a clicker to check students’ understanding of a concept, you might want to be able to pull up relevant data from how they performed on related activities in the LMS, courseware, and maybe even other platforms. In this particular scenario, you’d likely be looking at class aggregate performance rather than the individual learner journey. In yet another scenario, one could imagine a department sitting down and discussing how to fine-tune the course design for a 101 class that is used by a number of different instructors. So they might want to aggregate information across the whole department to see, for example, that students struggle with one particular unit or activity across the various course sections and instructors. The group might then drill down to investigate whether the assessment questions need to be adjusted, the content needs to be enriched, whether those in-class clicker activities help improve later performance, and so on.
I’m describing a semantic mesh of pedagogical information about a course’s design and the learner’s journey through it across application boundaries. There are no technological barriers to building such a mesh, although it requires hard work and hard thinking about preserving appropriate privacy in this new world.
I contend that we do not have such a mesh and are not making much progress toward it because the people who think about the learning journey coherence problem don’t tend to be the same people who think about data interoperability. Two types of situations have historically driven the development of EdTech interoperability specifications:
You sell a thing. I sell another thing. Our customers want our things to talk to each other, typically to make their own lives easier. For example, they want the students from your thing to get into my thing without a lot of manual effort that tends to lose them along the way. Or they want the grade from my thing to be reported back to your thing. It’s typically simple administrative stuff.
Speaking of grades, we need some. Our customers use our things to give students grades. Whatever else they are doing, they must also give students grades. Or progress. Or other fundamental stuff related to grading. Not learning. Grading.
Most technical interoperability conversations about tracking learning tend to involve a mix of people who know too little about pedagogy and people who know too much about it within too narrow a context. Both types tend to get lost in the weeds. We either talk about magical reusable learning objects, which could be anything from a picture to an hour-long module with many parts, or we debate taxonomies of assessment types or pedagogical philosophies round and round in circles.
We can try a number of different strategies to solve this problem once we put our mind to it. Some are social, some are technological, and the best are a mindful mix of the two. But rather than making this post even longer, I’m going to stop on this point: Before we can settle on an effective method developing a semantic mesh for pedagogy, we need to collectively decide that it is an urgent problem to solve. I think people feel it viscerally from their pandemic experiences. But I don’t think we’re collectively articulating the problem yet.
In my continuing tradition of inviting guests who are a lot smarter than me, our next two Blursday guests always challenge me and will challenge you too if come join us.
This week (6/3) will be John Whitmer or, as he was affectionately called when he led learning analytics work at Blackboard, Dr. John.
Dr. John
Dr. John
Family Resemblance?
John is one of the folks I go to first when I have questions about machine learning, AI, analytics, and similar topics in EdTech. He’s worked at a major university system, a big LMS company, a huge high-stakes testing company, and now at the Federation of American Scientists. He’s a smart dude. And funny. Good bar company. All that said, I have not yet heard him sing or play the piano, so we’ll see if we can get him do a little New Orleans-style jamming for us.
On next Blursday, June 10th, we’ll have Matthew Rascoff, who some of you may know from his tenure at Duke University but who has recently moved to Stanford. His new title is Special Advisor to the Provost for Education Outreach and Digital Innovation.
Mull on that one.
Matthew Rascoff
Matthew is one of those people who makes you sit back in your chair every time he opens his mouth. He’s incredibly thoughtful, articulate, and passionate. I’m excited about the new project he’s taken on and hope y’all will be able to brainstorm with hm a bit.
I’m delighted to tell you that this week’s Blursday Social guest (on 5/20 at 4 PM ET) is my friend Lauren Herckis, an anthropologist from Carnegie Mellon University who has studied faculty attitudes toward adopting EdTech and evidence-based teaching practices (among other subjects, both human and intellectual). Lauren is terrific. I’ve mentioned her or discussed her work a handful of times here on e-Literate.
Next week’s Blursday Social (5/27) we’ll be bringing back SEI Ventures’ Matt Tower. His recent post in his weekly Substack (which you should subscribe to) reminded me that it’s probably time to revisit Calbright, the PASSHE/SNHU deal, maybe the Perdue Online thing, and other attempts at creating innovation centers within academia.
Matt Tower
As always, you can find links to the registration pages for these events on my Run the World profile page. And, as always, these events are BYOB.
