e-Literate

Present is Prologue

Tag: IMS Caliper

  • Pedagogical Intent and Designing for Inquiry

    I’ve been asked by several folks to write up some version of the talk I gave at the recent IMS learning analytics summit. The focus was on how, going forward, interoperability standards will need to capture pedagogical intent if we are going to develop meaningful learning analytics.

    This isn’t a word-for-word transcription of that talk, but it does capture the gist. The subtitles and literary quotes are mostly from the original presentation. Many thanks to Rob Abel and Cary Brown for inviting me and giving me the opportunity to speak to the IMS community about this important inflection point.

    Interoperability is communication

    We are not here to curse the darkness, but to light the candle that can guide us through that darkness to a safe and sane future.

    John F. Kennedy

    Ten or fifteen years ago, when we talked about what we wanted from EdTech software interoperability, most of the time the things we wanted seemed like they ought to be simple. Mostly, we just wanted to transfer student roster and grade information from one system to another. When we got a little more ambitious, we asked for single sign-on and the ability to put a little window of one application into another one. This was foundational interoperability. It wasn’t “disruptive” or “revolutionary” or otherwise life-altering, but it was important.

    My first close encounter with an IMS interoperability effort was when I worked at Oracle in on Peoplesoft Campus Solutions team. We just wanted to be able to send a course roster to the LMS and have the LMS send a final grade for each student back. Going in, I couldn’t understand why that was hard or why it hadn’t already been done. All I knew was that (a) it was a problem that IMS had tried and failed to solve at least once before, since we were working on the revision of an existing standard, and (b) a consequence of that failure was that IT professionals on campuses everywhere had to cook up their own, often time-intensive, duct-tape-and-chewing-gum solutions to getting roster data into the LMS. As for grades, it made no sense to me that faculty had to copy grades from their electronic LMS grade book and manually re-enter them into their electronic SIS grade book. It seemed weird that this was a thing.

    It turned out that there was a translation problem. Registrars think about classes very differently than instructors and students do. For a registrar, if a student is taking a “statistics for psychology majors” course, and that course can be taken for credit in either the psychology or the math department, then “statistics for psychology” is actually two completely separate courses. And the decision of which of the two courses a student registers for may make the difference between that student meeting the requirements for graduation or not. On the other hand, the students and instructor experience “statistics for psychology” as one course meeting in one place on one schedule with one syllabus and one group of people.

    Good software reflects and supports the needs and expectations of its users. Accordingly, SIS software represented “statistics for psychology” as two courses, while LMS software wanted to create one course space for it. In order to have roster information show up as instructors and students expect it in the LMS and then return final grades as the registrar expects them in the SIS, the standards committee had to recognize that there was a translation problem and design a specification that could function as a two-way translator.

    This was an important lesson for me. Even seemingly simple interoperability challenges can be complicated because they often aren’t about the software so much as they are about the people who use the software. Interoperability design is at least partly a liberal art.

    Now, today, we would have a different possible way of solving that particular interoperability problem than the one we came up with over a decade ago. We could take a large data set of roster information exported from the SIS, both before and after the IT professionals massaged it for import into the LMS, and aim a machine learning algorithm at it. We then could use that algorithm as a translator. Could we solve such an interoperability problem this way? I think that we probably could. I would have been a weaker product manager had we done it that way, because I wouldn’t have gone through the learning experience that resulted from the conversations we had to develop the specification. As a general principle, I think we need to be wary of machine learning applications in which the machines are the only ones doing the learning. That said, we could have probably solved such a problem this way and might have been able to do it in a lot less time than it took for the humans to work it out.

    I will argue that today’s EdTech interoperability challenges are different. That if we want to design interoperability for the purposes of insight into the teaching and learning process, then we cannot simply use clever algorithms to magically draw insights from the data, like a dehumidifier extracting water from thin air. Because the water isn’t there to be extracted. The insights we seek will not be anywhere in the data unless we make a conscious effort to put them there through design of our applications. In order to get real teaching and learning insights, we need to understand the intent of the students. And in order to understand that, we need insight into the learning design. We need to understand pedagogical intent.

    That new need, in turn, will require new approaches in interoperability standards-making. As hard as the challenges of the last decade have been, the challenges of the next one are much harder. They will require different people at the table having different conversations.

    Data and communication are not the same

    O wonder!
    How many goodly creatures are there here!
    How beauteous mankind is! O brave new world
    That has such people in’t!

    William Shakespeare

    The quote above is from The Tempest. Here’s the scene: Miranda, the speaker, is a young woman who has lived her entire life on an island with nobody but her father and a strange creature who she may think of as a brother, a friend, or a pet. One day, a ship becomes grounded on the shore of the island. And out of it comes, literally, a handsome prince, followed by a collection of strange (and presumably virile) sailors. It is this sight that prompts Miranda’s exclamation.

    As with much of Shakespeare, there are multiple possible interpretations of her words, at least one of which is off-color. Miranda could be commenting on the hunka hunka manhood walking toward her.

    “How beauteous mankind is!”

    Or. She could be commenting on how her entire world has just shifted on its axis. Until that moment, she knew of only two other people in all of existence, each of who she had known her entire life and with each of whom she had a relationship that she understood so well that she took it for granted. Suddenly, there was literally a whole world of possible people and possible relationships that she had never considered before that moment.

    “O brave new world / That has such people in’t”

    So what is on Miranda’s mind when she speaks these lines? Is it lust? Wonder? Some combination of the two? Something else?

    The text alone cannot tell us. The meaning is underdetermined by the data. Only with the metadata supplied by the actor (or the reader) can we arrive at a useful interpretation. That generative ambiguity is one of the aspects of Shakespeare’s work that makes it art.

    But Miranda is a fictional character. There is no fact of the matter about what she is thinking. When we are trying to understand the mental state of a real-life human learner, then making up our own answer because the data are not dispositive is not OK. As educators, we have a moral responsibility to understand a real-life Miranda having a real-life learning experience so that we can support her on her journey.

    Intention matters in education

    With regard to moral rules, the child submits more or less completely in intention to the rules laid down for him, but these, remaining, as it were, external to the subject’s conscience, do not really transform his conduct.

    Jean Piaget

    The challenge that we face as educators is that learning, which happens completely inside the heads of the learners, is invisible. We can not observe it directly. Accordingly, there are no direct constructs that represent it in the data. This isn’t a data science problem. It’s an education problem. The learning that is or isn’t happening in the students’ heads is invisible even in a face-to-face classroom. And the indirect traces we see of it are often highly ambiguous. Did the student correctly solve the physics problem because she understands the forces involved? Because she memorized a formula and recognized a situation in which it should be applied? Because she guessed right? The instructor can’t know the answer to this question unless she has designed a series of assessments that can disambiguate the student’s internal mental state.

    In turn, if we want to find traces of the student’s learning (or lack thereof) in the data, we must understand the instructor’s pedagogical intent that motivates her learning design. What competency is the assessment question that the student answered incorrectly intended to assess? Is the question intended to be a formative assessment? Or summative? If it’s formative, is it a pre-test, where the instructor is trying to discover what the student knows before the lesson begins? Is it a check for understanding? A learn-by-doing exercise? Or maybe something that’s a little more complex to define because it’s embedded in a simulation? The answers to these questions can radically change the meaning we assign to a student’s incorrect answer to the assessment question. We can’t fully and confidently interpret what her answer means in terms of her learning progress without understanding the pedagogical intent of the assessment design.

    But it’s very easy to pretend that we understand what the students’ answers mean. I could have chosen any one of many Shakespeare quotes to open this section, but the one I picked happens to be the very one from which Aldous Huxley derived the title of his dystopian novel Brave New World. In that story, intent was flattened through drugs, peer pressure, and conditioning. It was reduced to a small set of possible reactions that were useful in running the machine of society. Miranda’s words appear in the book in a bitterly ironic fashion from the mouth of the character John, a “savage” who has grown up outside of societal conditioning.

    We can easily develop “analytics” that tell us whether students consistently answer assessment questions correctly. And we can pretend that “correct answer analytics” are equivalent to “learning analytics.” But they are not. If our educational technology is going to enable rich and authentic vision of learning rather than a dystopian reductivist parody of it, then our learning analytics must capture the nuances of pedagogical intent rather than flattening it.

    This is hard.

    Some more examples

    The lesson assessment example is easy enough to understand. (My post series on content as infrastructure explores it in more detail.) But the more one looks around at the full range of analytics that are truly aimed at supporting student success, the clearer the lesson about capturing intent becomes.

    Take, for example, summer melt. I recently just hosted an entire hour-long Standard of Proof webinar on this topic. (You should watch it. It’s good.) Here’s a slide from that webinar which illustrates the obstacles that first-generation students face in getting from their college acceptance letter to the first day of class:

    The Summer Melt Maze

    Credit: Lindsay Page

    Each of the text labels represents an obstacle that first-generation students in particular may struggle to overcome, because their circumstances are complex (e.g., no parent to provide a parental signature), because they don’t have parents or guardians who have the training and knowledge to help them, and/or because they’re seventeen-year-old kids. I don’t know about you, but I don’t think I had to navigate a single one of these obstacles without parental help, and I don’t know if I could or would have done so without it.

    Now think about developing a software solution to identify the specific barrier a student is struggling with and provide appropriate help. Could you solve the problem simply by sucking in enough data and running a machine learning algorithm? I very much doubt it. And even if you could, think about how invasive you would have to be to do so. Think about the privacy implications. The cure would be worse than the disease.

    Georgia State University took a different approach. They have invited students to share their intent. They use a chatbot. A chatbot is a conversational interface. Students can ask directly about the problems they were encountering. Once students specify their intent, then machine learning can be used to further disambiguate it. For example, the software can figure out that a student who writes “I have no money” may be asking for help obtaining financial aid. But only because there is a conversational interface, and because the software behind that interface has been programmed to anticipate and respond to a certain range of questions that a student might come to the chatbot for answers. Through a combination of the interface layer, the data layer, and the usage context, the educational intent was encoded into the system.

    Here’s another example that postdates my talk. ACT recently released a paper on detecting non-cognitive education-relevant factors like grit and curiosity through LMS activity data logs. This is a really interesting study that I hope to write more about in a separate post in the near future, but for now, I want to focus on how labor-intensive it was to conduct. First author John Whitmer, formerly of Blackboard, is one of the people in the learning analytics community who I turn to first when I need an expert to help me understand the nuances. He’s top-drawer, and he’s particularly good at squeezing blood from a stone in terms of drawing credible and useful insights from LMS data.

    Here’s what he and his colleagues had to do in order to draw blood from this particular stone:

    The online interaction features were generated from the LMS clickstream data. After manual inspection, we determined that the action field alone (e.g., “opened”) was insufficient to address our research questions and needed to be joined with the label of the item that the action was taken in reference to, which was a complex pairing. For example, course item values in the LMS data include “Week 12: Electrochemistry” or “CHEM 102 Practice Exam 4B,” which were easily interpretable from the course syllabus, while others (e.g., “2/27 CL” or “18.7 RQ”) required confirmation from the instructor. Hence, we created broader activity categories for these activity events in the LMS data using the course syllabus with confirmation from the instructor which resulted in 110 unique activity events in the LMS data that were recoded to a total of 21 activity categories as described in Table 4.

    First, they had to look at the syllabi. With human eyeballs. Then they had to interview the instructors. You know, humans having conversations with other humans. Then the humans—who had interviewed the other humans in order to annotate the syllabi that they looked at with their human eyeballs—labeled the items being accessed in the LMS with metadata that encoded the pedagogical intent of the instructors. Only after they did all that human work of understanding and encoding pedagogical intent could they usefully apply machine learning algorithms to identify patterns of intent-relevant behavior by the students.

    LMSs are often promoted as being “pedagogically neutral.” (And no, I don’t believe that Moodle is any different.) Another way of putting this is that they do not encode pedagogical intent. This means it is devilishly hard to get pedagogically meaningful learning analytics data out of them without additional encoding work of one kind or another.

    Interoperability without intent creates chaos

    If Jorge Luis Borges’ Library of Babel could have existed in reality, it would have been something like the Long Room of Trinity College.

    Christopher de Hamel

    I want to underscore the point that simply collecting more data and writing more clever algorithms will not help us find a way out of the problem of that last example. It is a problem of epistemic closure. Data and knowledge are not the same, and more data do not necessarily unlock more knowledge.

    “The Library of Babel” is a short story by the great Jorge Luis Borges. (It’s only nine pages long. You should read it.) The story describes a world that perfectly captures the nature of the problem we face:

    The universe (which others call the Library) is composed of an indefinite and perhaps infinite number of hexagonal galleries, with vast air shafts between, surrounded by very low railings. From any of the hexagons one can see, interminably, the upper and lower floors. The distribution of the galleries is invariable. Twenty shelves, five long shelves per side, cover all the sides except two; their height, which is the distance from floor to ceiling, scarcely exceeds that of a normal bookcase. One of the free sides leads to a narrow hallway which opens onto another gallery, identical to the first and to all the rest. To the left and right of the hallway there are two very small closets. In the first, one may sleep standing up; in the other, satisfy one’s fecal necessities. Also through here passes a spiral stairway, which sinks abysmally and soars upwards to remote distances….

    There are five shelves for each of the hexagon’s walls; each shelf contains thirty-five books of uniform format; each book is of four hundred and ten pages; each page, of forty lines, each line, of some eighty letters which are black in color. There are also letters on the spine of each book; these letters do not indicate or prefigure what the pages will say….

    The orthographical symbols are twenty-five in number. This finding made it possible, three hundred years ago, to formulate a general theory of the Library and solve satisfactorily the problem which no conjecture had deciphered: the formless and chaotic nature of almost all the books. One which my father saw in a hexagon on circuit fifteen ninety-four was made up of the letters MCV, perversely repeated from the first line to the last. Another (very much consulted in this area) is a mere labyrinth of letters, but the next-to-last page says Oh time thy pyramids. This much is already known: for every sensible line of straightforward statement, there are leagues of senseless cacophonies, verbal jumbles and incoherences….

    Five hundred years ago, the chief of an upper hexagon came upon a book as confusing as the others, but which had nearly two pages of homogeneous ines. He showed his find to a wandering decoder who told him the lines were written in Portuguese; others said they were Yiddish. Within a century, the language was established: a Samoyedic Lithuanian dialect of Guarani, with classical Arabian inflections. The content was also deciphered: some notions of combinative analysis, illustrated with examples of variations with unlimited repetition. These examples made it possible for a librarian of genius to discover the fundamental law of the Library. This thinker observed that all the books, no matter how diverse they might be, are made up of the same elements: the space, the period, the comma, the twenty-two letters of the alphabet. He also alleged a fact which travelers have confirmed: In the vast Library there are no two identical books. From these two incontrovertible premises he deduced that the Library is total and that its shelves register all the possible combinations of the twenty-odd orthographical symbols (a number which, though extremely vast, is not infinite): Everything: the minutely detailed history of the future, the archangels’ autobiographies, the faithful catalogues of the Library, thousands and thousands of false catalogues, the demonstration of the fallacy of those catalogues, the demonstration of the fallacy of the true catalogue, the Gnostic gospel of Basilides, the commentary on that gospel, the commentary on the commentary on that gospel, the true story of your death, the translation of every book in all languages, the interpolations of every book in all books.

    Jorge Luis Borges

    The rest of the story is a rumination on how humans might make sense of this infinite series of rooms—this “data lake,” in modern parlance—in absence of any information about the intent of its creator.

    Since the Library contains every possible book with that number of pages, lines and characters, somewhere in all these rooms must exist a book that explains exactly what it all means. So it’s a data search problem, right?

    Wrong. Because there also exist books that are extremely similar but differ in minor but critical details. And books that argue why the book with the Truth is actually false. For that matter, there are books that contain many of the exact same words in the exact same order but are written in languages in which the words mean different things. How can one tell which account is the Truth?

    One can’t.

    If we are going to make progress toward educationally useful analytics, then we must ruthlessly expunge all traces of magical thinking about data. There are fundamental limits to what the data can tell us. Even systems that are designed for pedagogical intent do not necessarily encode it in a way that is useful for interoperable analytics. In some cases, it may be encoded at the user interface layer. A courseware authoring platform may never label an assessment as “formative” or “summative” in the data because the intended distinction is obvious to the users. In other cases, the data may be encoded in an idiosyncratic manner that does not map well to other systems (either of software or of thought). In still other cases, it may be designed badly and incorrectly or misleadingly reflect the pedagogical intent. It would be relatively easy to create a data lake of Babel which we could explore infinitely in a fruitless search for meaning.

    That way lies madness.

    If we want useful educational analytics, then we cannot simply worship the data and the algorithms. The humans must do some of the learning.

    The “semantic web” is all about intent

    I love you. You are the object of my affection and the object of my sentence.

    Mignon Fogarty

    One of the triggers for my being invited to speak at IMS about learning analytics in the first place was a previous post I wrote which was (partly) on how the structure of Caliper, which is borrowed from the structure of the semantic web, supports new sorts of interoperability conversations. Since you can read that blog post, I won’t repeat the argument in detail here. But the gist is that we both have to and can boil down chains of inference that combine pedagogical intent into simple human language that educators can understand and articulate for themselves. The triple structure of the semantic web—a simple three-word sentence with a subject, a verb, and a direct object—is designed to enable non-technical humans to string thoughts and inferences together in ways that enable more technical humans to translate those inference patterns into data structures and interoperability requirements.

    Right now, IMS Caliper adopters are largely using this vernacular as just one more IT tool for largely old-school data centralization. So now they use a “lake” instead of a “warehouse.” It’s still a centralized and IT-specialized mindset which is not suited for thinking about making meaning from multiple applications, never mind talking with other humans about interpreting the intent of users working across multiple applications.

    This is the problem that must be solved over the next decade to make real progress on educational analytics. It is at least partly a liberal arts problem. And it will be at least a decade’s worth of work, though we don’t have to wait that long to see early results.

    You get to choose the world we live in

    O wonder!
    How many goodly creatures are there here!
    How beauteous mankind is! O brave new world
    That has such people in’t!

    Shakespeare or Huxley?

    So which will it be? A brave new world in which we experience continuous wonder at how beauteous mankind is, or one in which “learning” is defined by the ability to correctly answer a series of questions and earn some digital badges? The difference between these possible futures is not one in which we embrace or reject technology, or data. It’s one in which we either embrace or ignore the complexity of human learning and the reality that we must make a conscious effort to ensure that some of this complexity is encoded into the data if we are going to design analytics systems that are educationally useful. We need to elicit specific reactions from our students as expert educators and encode the pedagogical intent for eliciting those reactions along with the reactions themselves.

    .The play’s the thing!

    William Shakespeare
  • Learning Engineering: A Caliper Example

    Learning Engineering: A Caliper Example

    In my recent IMS update post, I wrote,

    [T]he nature and challenges of interoperability our sector will be facing in the next decade are fundamentally different from the ones that we faced in the last one. Up until now, we have primarily been concerned with synchronizing administration-related bits across applications. Which people are in this class? Are they students or instructors? What grades did they get on which assignments? And how much does each assignment count toward the final course grade? These challenges are hard in all the ways that are familiar to anyone who works on any sort of generic data interoperability questions. 
    But the next decade is is going to be about data interoperability as it pertains to insight. Data scientists think this is still familiar territory and are excited because it keeps them at the frontier of their own profession. But this will not be generic data science, for several reasons.

    I then asserted the following positions:

    • Because learning processes are not directly observable, blindly running machine learning algorithms against the click streams in our learning platforms will probably not teach us much about learning.
    • On the other hand, if our analytics are theory-driven, i.e., if we start with some empirically grounded hypotheses about learning processes and design our analytics to search for data that either support or disprove those hypotheses, then we might actually get somewhere.
    • Because learning analytics expressions written in the IMS Caliper standard can be readily translated into plain English, Caliper could form a basis for expressing educational hypotheses and translating them into interoperable tools for testing those hypotheses across the boundaries of tech tools and platforms.
    • The kind of Caliper-mediated conversation I imagined among learning scientists, practicing educators, data scientists, learning system designers, and others, is relevant to a term coined and still used heavily at Carnegie Mellon University—”learning engineering.”

    In this post, I’m going to explore the last two points in more detail.

    What the heck is “learning engineering”?

    The term “learning engineering” was first used by Nobel laureate and Carnegie Mellon University polymath Herbert Simon in 1966. It has been around for quite a while. But it is a term whose time as finally has come and, as such, we are seeing the usual academic turf wars over its meaning and value. On the one hand, some folks love it, embrace it, and want to apply it liberally. IEEE has an entire group devoted to defining it. As is always the case, some of this sort of enthusiasm is thoughtful, and some of it is less so. At its worst, there is a tendency for people to get tangled up in the term because it provides a certain je ne sais quoi they’ve been yearning for to describe the aspects of their jobs that they really want to be doing as change agents rather than the mundane tasks that they keep being dragged back into doing, much like the way some folks are wrapping “innovation” and “design” around themselves like a warm blanket. It’s perfectly understandable, and I think it attaches to something real in many cases, but it’s hard to say exactly what that is. And, of course, where there are enthusiasts in academia, there are critics. Again, some thoughtful, while others…less so. (Note my comment in the thread on that particularly egregious column.)

    If you want to get a clear sense of the range of possible meanings of “learning engineering” as used by people who actually think about it deeply, one good place to start would be Learning Engineering for Online Education: Theoretical Contexts and Design-Based Examples edited by Chris Dede, John Richards, and Bror Saxberg. (I am still working on getting half a day’s worth of Carnegie Mellon University video presentations on their own learning engineering work ready for posting on the web. I promise it is coming.) There are a lot of great take-aways from that anthology, one of which is that even the people who think hard about the term and work together to put together something like a coherent tome on the subject don’t fully agree on what the term means.

    And that’s really OK. Let’s just set a few boundary conditions. On the one hand, learning engineering isn’t an all-encompassing discipline and methodology that is going to make all previous roles, disciplines, and methodologies obsolete. If you are an instructional designer, or a learning designer, or a user experience designer; if you practice design thinking, or ADDIE; be not afraid. On the other hand, learning engineering is not creeping Stalinism either. Think about learning engineering, writ large, as applying data and cognitive sciences to help bring about desired learning outcomes, usually within the context of a team of colleagues with different skills all working together. That’s still pretty vague, but it’s specific enough for the current cultural moment.

    Forget about your stereotypes of engineers and their practices. Do you believe there is a place for applied science in our efforts to improve the ways in which we design and deliver our courses, or try to understand and serve our students needs and goals? If so, what would such an applied science look like? What would a person applying the science need to know? What would their role be? How would they work with other educators who have complementary expertise?

    That is the possibility space that learning engineering inhabits.

    Applied science as a design exercise

    One of the reasons that people have trouble wrapping their heads around the notion of learning engineering is that it was conceived of by very unusual mind. Some of the critiques I’ve seen online of the term position “learning engineering” in opposition to “learning design.” But as Phil Long points out in his essay in the aforementioned anthology, Herb Simon both coined the term “learning engineering” and is essentially the grandfather of design thinking:

    Design science was introduced by Buckminster Fuller in 1963, but it was Herbert Simon who is most closely associated with it and has established how we think of it today. “The Sciences of the Artificial” (Simon, 1967) distinguished the artificial, or practical sciences, from the natural sciences. Simon described design as an ill-structured problem, much like the learning environment, which involves man-made responses to the world. Design science is influenced by the limitations of human cognition unlike mathematical models. Human decision-making is further constrained by practical attributes of limited time and available information. This bounded rationality makes us prone to seek adequate as opposed to optimal solutions to problems. That is, we engage in satisficing not optimizing. Design is central to the artificial sciences: ‘Everyone designs who devises courses of action aimed at changing existing situations into desired ones.’ Natural sciences are concerned with understanding what is; design science instead asks about what should be. this distinction separates the study of the science of learning from the design of learning. Learning scientists are interested in how humans learn. Learning engineers are part of team focused on how students ought to learn.”

    Phil Long, “The Role of the Learning Engineer”

    Phil points out two important dichotomies in Simon’s thinking. The first one: is vs. ought. Natural science is about what is, while design science is about what you would like to exist. What you want to bring into being. The second dichotomy is about well structured vs. poorly structured. For Simon, “design” is a set of activities one undertakes to solve a poorly structured problem. To need or want is human, and to be human is to be messy. Understanding a human need is about understanding a messy problem. Understanding how different humans with different backgrounds and different cognitive and non-cognitive abilities learn, given a wide range of contextual variables like the teaching strategies being employed, the personal relationships between students and teacher, what else is going on in the students’ lives at the time, whether different students are coming to class well fed and well slept, and so on, is pretty much the definition of a poorly structured problem. So as far as Herb Simon is concerned, education is a design problem by definition, whether or not you choose to use the word “engineer.”

    In the next section of his article, Phil then makes a fascinating connection between the evolution of design thinking, which emerged out design science, and learning engineering. The key is in identifying the central social activity that defines design thinking:

    Design thinking represents those processes that designers use to create new designs, possible approaches to problem solutions spaces where none existed before. A problem-solving method has been derived from this and applied to human social interactions iteratively taking the designer and/or co-design participants from inspiration to ideation and then to implementation. The designer and design team may have a mental model of the solution to a proposed problem, but it is essential to externalize this representation in terms of a sketch a description of a learning design sequence, or by actual prototyping of the activities which the learner is asked to engage. [Emphasis added.] All involved can see the attributes of the proposed design solution that were not apparent in the conceptualization of it. this process of externalizing and prototyping design solutions allows it to be situated in larger and different contexts, what Donald Schon called reframing the design, situating it in contexts other than originally considered.

    Phil Long, “The Role of the Learning Engineer”

    So the essential feature that Phil is calling out in design thinking is putting the idea out into the world so that everybody can see it, respond to it, and talk about it together. Now watch where he takes this:

    As learning environments are intentionally designed in digital contexts, the opportunity to instrument the learning environment emerges. Learners benefit in terms of feedback or suggested possible actions. Evaluators can assess how the course performed on a number of dimensions. The faculty and others in the learning-design team can get data through the instrumented learning behaviors, which may provide insight into how the design is working, for whom it is working, and in what context.

    Phil Long, “The Role of the Learning Engineer”

    Rather than a sketch, a wireframe, or a prototype, a learning engineer makes the graph, the dashboard, or the visualization into the externalization. For Herb Simon, as for Phil Long, these design artifacts serve the same purpose. They’re the same thing, basically.

    If you’re not a data person, this might be hard to grasp. (I’m not a data person. This is hard for me to grasp sometimes.) How can you take numbers in a table and turn them into a meaningful artifact that a group of people can look at together, discuss, make sense of, debate, and learn from? What might that even look like?

    Well, it might look something like this, for example:

    Higher ed LMS market share for US and Canada, January 2019
    Phil Hill’s famous squid diagram

    Phil Hill has a graduate degree in engineering. Not learning engineering. Electrical. (Also, he’s not a Stalinist.)

    By the way, when we externalize and share data with a student about her learning processes in a form that is designed to provoke thought and discussion, we have a particular term of art for that in education. It’s called “formative assessment.” If we do it in a way such that the student always has access to such externalizations, which are continually updating based on the student’s actions, we call that “continuous formative assessment.” When executed well, there is evidence that it can be an effective educational practice.

    Caliper statements as learning engineering artifacts

    So here’s where we’ve arrived at this point in the post:

    • Design is a process by which we tackle ill-defined problems of meeting human needs and wants, such as needing or wanting to learn something.
    • Engineering is a word that we’re not going to worry about defining precisely for now, but it relates to applying science to a design problem, and therefore often involves the measurement and numbers.
    • One important innovation in design methodology is the creation of external artifacts early in the design process so that various stakeholders with different sorts of experience and expertise can provide feedback in a social context. In other words, create something that makes the idea more “real” and therefore easier to discuss.
    • Learning engineering includes the skills of creation and manipulation of design artifacts that require more technical expertise, including expertise in data and software engineering.

    The twist with Caliper is that, rather than using visualizations and dashboards as the externalization, we can use human language. This was the original idea of behind the Semantic Web, which is still brilliant in concept, even if the original implementation was flawed. Let’s review that basic idea as implemented in Caliper:

    • You can express statements about the world (or the world-wide web) in three-word sentences of the form [subject] [verb] [direct object] e.g., [student A] [correctly answers] .
    • Because English grammar works the way it does, you can string these sentences together to form inferences, e.g., [tests knowledge of] [multiplying fractions]; therefore, [student A] [correctly answers] [a question about multiplying fractions].
    • We can define mandatory and optional details of every noun and verb e.g., it might be mandatory to know that question 13 was a multiple choice question, but it might be optional to include the actual text of the question, the correct answer, and the distractors.

    That’s it. Three-word sentences, which work the way they do in English grammar, and definitions of the “words.”

    A learning engineer could use Caliper paragraphs as a design artifact to facilitate conversations about refining the standard, the products involved, and the experimental design. I’ll share a modified version of an example I recently shared with an IMS engineer to illustrate this same point.

    Suppose you are interested in helping students become better at reflective writing. You want to do this by providing them with continuous formative assessment, i.e., in addition to the feedback that you give them as an instructor, you want to provide them an externalization of the language in their reflective writing assignments. You want to use textual analysis to help the students look at their own writing through a new lens, find the spots where they are really doing serious thought work, and also the spots where maybe they could think a little harder.

    But you have to solve a few problems in order to do give this affordance to your students. First, you have to develop the natural language analysis tool that can detect cues in the students’ writing that indicate self-reflection (or not). That’s hard enough, but the research is being conducted and progress is being made. The second problem is that you are designing a new experiment to test your latest iteration and need some sort of summative measure to test against. So maybe you design a randomized controlled trial where half the students in the class use the new feedback tool, half don’t, and all get the same human-graded final reflective writing assignment. You compare the results.

    This is an example of theory-driven learning analytics. Your theory is that student reflection improves when students become more aware of certain types of reflective language in their journaling. You think you can train a textual analysis algorithm to reliably distinguish—externalize—the kind of language that you want students to be more aware of in their writing and point it out to them. You want to test that by giving students such a tool and see if their reflective writing does, in fact, improve. Either students’ reflective writing will improve under the test condition, which will provide supporting evidence for the theory, or it won’t, which at the very least will not support the theory and might provide evidence that tends to disprove the theory, depending on the specifics. There are data science and machine learning being employed here, but they are being employed more selectively than just shotgunning an algorithm at a data set and expecting it to come up with novel insights about the mysteries of human cognition.

    Constructing theory-driven learning analytics of the sort described here is challenging enough to do in a unified system that is designed for the experiment. But now we get to the problem for which we will need the help of IMS over the next decade, which is that the various activities we need to monitor for this work often happen in different applications. Each writing assignment is in response to a reading. So the first thing you might want to do, at least for the experiment if not in the production application, is to control for students who do the reading. If they aren’t doing the reading, then their reflective writing on that reading isn’t going to tell you much. Let’s say the reading happens to take place in an ebook app. But their writing takes place in a separate notebook app. Maybe it’s whatever notebook app they normally use—Evernote, One Note, etc. Ideally, you would want them to journal in whatever they normally use for that sort of activity. And if it’s reflective writing for their own growth, it should be an app that they own and that will travel with them after they leave the class and the institution. On the other hand, the final writing assignment needs to be submittable, gradable, and maybe markable. So maybe it gets submitted through an LMS, or maybe through a specialized tool like Turnitin.

    This is an interoperability problem. But it’s a special one, because the semantics have to be preserved through all of these connections in order for (a) the researchers to conduct the study, and then (b) the formative assessment tool to have real value to the students. The people who normally write Caliper metric profiles—the technical definitions of the nouns in Caliper—would have no idea about any of this on their own. Nor would the application developers. Both groups would need to have a conversation with the researchers in order to get the clarity they need in order to define the profiles for this purpose.

    The language of Caliper could help with this if a person with the right role and expertise were facilitating the conversation. That person would start by eliciting a set of three-word sentences from the researchers. What do you need to know? The answers might include statements like the following:

    • Student A reads text 1
    • Student A writes text alpha
    • Text alpha is a learning reflection of text 1
    • Student A reads text 2
    • Text 2 is a learning reflection of texts 1 and 2
    • Etc.

    The person asking the questions of the researcher and the feature designer—let’s call that person the learning engineer—would then ask questions about the meanings and details of the words, such as the following:

    • In what system or systems is the reading activity happening?
    • Do you need to know if the student started the reading? Finished it? Anything finer grained than that?
    • What do you need to know about the student’s writing in order to perform your textual analysis? What data and metadata do you need? And how long a writing sample do you need to elicit in order to perform the kind of textual analysis you intend and get worthwhile results back?
    • What do you mean when you say that text 2 is a reflection of both text 1 and 2, and how would you make that determination?

    At some point, the data scientist and software systems engineers would join in the conversation and different concerns would start to come up, such as the following:

    • Right now, I have no way of associating Student A in the note-taking system with Student A in the reading system.
    • To do the analysis you want, you need the full text of the reflection. That’s not currently in the spec, and it has performance implications. We should discuss this.
    • The student data privacy implications are very different for an IRB-approved research study, an individual student dashboard, and an instructor- or administrator-facing dashboard. Who owns these privacy concerns and how do we expect them to be handled?

    Notice that the Caliper language has become the externalization that we manipulate socially in the design exercise. There are two aspects of Caliper that make this work: (1) the three-word sentences are linguistically generative, i.e., they can express new ideas that have never been expressed before, and (2) every human-readable expression directly maps to a machine-readable expression. These two properties together enable rich conversations among very different kinds of stakeholders to map out theory-driven analytics and the interoperability requirements that they entail.

    This is the kind of conversation by which Caliper can evolve into a standard that leads to useful insights and tools for improving learning impact. And in the early days, it will likely happen one use case at a time. Over time, the working group would learn from having enough of these conversations that design patterns would emerge, both for writing new portions of the specification itself and for the process by which the specification is modified and extended.

    Copyright Carnegie Mellon University, CC-BY
  • How and Why the IMS Failed with LTI 2.0

    How and Why the IMS Failed with LTI 2.0

    While at EDUCAUSE last week, we heard from several sources that the IMS is walking away from LTI 2.0. I’m not sure what term the organization is using to characterize what they’re doing; there’s nothing on the public web site that I can find at the moment. But the reality is that they are no longer encouraging the adoption of LTI 2.0 and are actively investing their energy in developing the 1.x branch, starting with something that they’re calling “LTI Advantage.” (More on that in a bit.)

    To be clear, I believe this is a good decision, and I also believe the fact that they felt the need to do so is not particularly scandalous. Everybody makes mistakes. The important thing is to learn from them. So, with that in mind, and with the IMS quarterly meeting coming up this week, I thought it might be useful to write a post-mortem that could potentially be of use going forward.

    Interoperability Standards as Multilateral Trade Deals

    One of the biggest problems the IMS faces is that a lot of the participants on the specification committees don’t fully understand the big picture of how standards adoption works in the real world. They are mostly technologists with a sprinkling of product managers. They usually have the best of intentions. But they do not have the right roles to see the business ramifications of spec development clearly, and they very often do not get adequate guidance from the stakeholders in their respective organizations that do have the right roles.

    First and foremost, interoperability specifications are a lot like multilateral trade agreements. They are agreements that everybody will operate according to certain rules under certain circumstances. The fact that these agreements are written in engineering language and implemented in software code doesn’t change the fact that they are multi-party treaties. One of the major implications that falls out of this circumstance is that organizations will not adopt the standard—or “sign the treaty”—unless they believe the benefits will outweigh the costs. If a specification is going to be broadly adopted, then it needs to be designed so that all the adopting parties will see direct benefit. Remember, every minute of developer time spent implementing a standard could have been spent developing a feature or fixing a bug instead. If their customers don’t care about the benefits of the standard and developers don’t see internal benefits (like reducing the time they have to spend on one-off integrations), then the rational decision for product or project owners is to not implement the standard.

    Such a decision does not mean that all progress grinds to a halt. In the trade world, there are many more bilateral trade agreements than there are multilateral trade agreements. In the software world, the equivalent to a bilateral agreement would be a proprietary integration. Many companies today that want to provide richer integration with another platform—especially with an LMS—will use the multilateral agreement that is LTI 1.x as a baseline and extend it with the bilateral agreements of proprietary integrations. This is as it should be. Not every integration needs to be standards-based. In fact, a lot of innnovation happens beyond the edge of the standard as software developers come up with new ideas about how their products can work together. They hammer out a bilateral trade agreement to get that new idea implemented and to gain a competitive edge for a while. Eventually, enough products may choose to implement the same pattern that it makes sense for everybody to adopt the same way of integrating so that developers don’t have to build slightly different versions of the same integration over and over.

    Interoperability standards, being multilateral trade agreements, are rarely innovative or sexy if they are done right. Once in a while, a working group may come up with a new spin that is generative. But this is always a risky proposition. The technologists and product designers on these working groups, being creative people, want to come up with cool stuff. But “cool” isn’t what drives their employers to adopt. There always needs to be a cost/benefit analysis for each potential adopter. If that analysis doesn’t look good, then no amount of coolness will matter.

    As far as I can tell from the outside, this is precisely what went wrong with LTI 2.0.

    Please Pass (on) the SOAP

    Back in the early days of service-oriented architecture (SOA) in software engineering, everybody was using a complex XML-based protocol called SOAP. The idea was that everybody had data to pass around, and if there were discrete services that could be called for that data, then we could have a lot more interoperability. SOAP was considered secure (or securable, anyway), so it could be used for sensitive data. SOA advocates envisioned a world in which all software provided lots of services for other software to use, creating rich opportunities for interoperability. In fact, they thought there would be so many services that it would be impractical to connect them all up by hand. So they invented something called a service bus that helped each piece of software automagically discover the services available to it from other software in the system and also to offer up its own services to any other software that needed them.

    As you might imagine, this all turned out to be horribly complex and expensive. I happened to work at Oracle at the time that all this peaked, so I had a ring-side seat. The company was selling hugely expensive service management software that was often implemented by hugely expensive consultants. Some of that stuff took off and is in use in some places—usually big IT shops like moneycenter banks. But it never got the ubiquitous adoption that the enthusiasts expected because it was too much work for not enough payoff. There just weren’t that many services to justify the management costs.

    That was roughly ten years ago. Fast forward seven years from then. What is the essence of LTI 2.0? It was essentially an attempt to implement the same idea using the more modern REST style of web programming. It was going to have a strong security model and service discovery. Predictably, it was really hard to implement. A senior development manager from a major LMS vendor—somebody who has been in the industry and involved with the standards process for a very long time—said that LTI 2.0 was by far the most complex standard he’s ever had to implement. If the major LMS providers struggled to implement it, then what are the chances that lots of small LTI-compatible tool makers will implement it? To answer that question, start from zero and subtract a significant number.

    But I don’t think LTI would have had a high chance of success even had it been less complicated, because it wasn’t designed to be a multilateral trade agreement. What problem would a secure service bus with discoverability solve for each of the treaty signatories? The LMS companies do get a clear benefit. For them, many of the customers have LMS systems administrators who spend their days hooking up tools for individual instructors. The LMS gets most of the blame for this situation, not because it’s primarily their fault but because their customers perceive it as a pattern of weakness when they use the product. Automagic integrations would make some of their key stakeholders very happy.

    On the other hand, if you’re a tool vendor, you usually only get blamed for the challenges of one integration: your tool into the customer’s LMS. And even then, the LMS vendor gets most of the heat. At the same time, hardly anybody is passing data to you, and it can be a struggle to even get the LMS vendors to accept the data that you want to give them. So LTI 2.0 offered most tool makers a lot of pain for no obvious benefit.

    The people at the negotiating table for LTI 2.0 were apparently not the right people to negotiate a trade agreement. They were the right people to think of something really cool. For a standard to work, you need the former and want the latter, but only if the people thinking of cool stuff are in close communication with their colleagues who are making the cost/benefit decisions.

    Now, I believe there is an opportunity to get to the cool world that the LTI 2.0 working group envisioned. But to get there, you need to flip the script.

    Services First

    Let’s try another analogy for a moment. In the early days of the telephone, human operators connected every call manually. There was interoperability in the sense that all telephones operated the same way over the network of wires and switchboards, but each individual connection between callers had to be hand-configured. In the early days, that was fine. It made sense. There were few enough callers that it was worth the cost of the operators. But eventually, the number of phone connections grew to the point where hiring humans to manually wire connections was no longer viable for anybody. Call connection times grew, as did service provider costs. At that point it made economic sense to create an automated switchboard.

    LTI 2.0 was an attempt to invent a very fancy automated switchboard at a time when there are still hardly any callers. Once we reach the point where many tool providers are offering multiple data services and, in some cases, also receiving them, then they will see direct benefits from a secure and automated service discovery mechanism. At that point, the negotiations can begin regarding feature richness versus cost of implementation.

    To get there, the IMS needs to flip the script and start by creating a world that is rich enough in services that people need to care more about managing them all. The IMS has made a good (re)start with LTI Advantage, which just adds roster provisioning, grade return, and deep linking to LTI. These are basic hygiene needs. Basically, this means any tool can find out from the LMS who is in the class and what their roles are, which a lot of tools need and is a step up from the way LTI passes user information along now. LTI Advantage can also enable the tool provider to give the LMS links that support single sign-on to specific places within the tool, and it can enable tools to return multiple grades to the LMS. Lots of apps would find these capabilities handy.

    I’ve heard through the grapevine that the next challenge is improving security. Obviously, securing student data is critical. There are a lot of data sharing services that shouldn’t be offered until that security can be guaranteed. (I won’t comment on the security of rostering and grade information in LTI Advantage because I don’t know anything about it, but obviously that is one very sensitive area where schools should be asking questions about security.)

    Once security is in place, the next step beyond that should be to focus on generating enough call connections to keep the switchboard operator busy. I have been arguing for some time that Caliper should be used as a data interoperability exchange standard between apps that operates through the LTI window. This could work for bilateral agreements. For example, let’s say that you want to integrate a blog with an LMS so that posts with a certain tag could be automagically submitted upon publication for a particular assignment. Let’s further suppose that the IMS has not yet developed a standard, a multilateral trade agreement, for that kind of integration. If LTI and Caliper provide tools that make developing that integration easy enough, then the two integrating parties could do the work in the style of an official integration, saving time by drawing on the LTI and Caliper infrastructure that’s already in place. If enough other tool makers become interested in the integration, then it could be submitted for ratification as an official extension of the standard. This only works if designing and implementing a new Caliper profile is easy. But if it’s done right, then we should see the number of services explode.

    At some point, it will become obvious to all parties that they need autodiscovery to manage all these integrations. Then and only then will it make sense to come up with a hopefully simpler implementation of service discovery for the next run at LTI 2.x.

    Open It Up

    To reiterate, the path to getting to the next era of interoperability is to make it easy for many developers to build bilateral integrations that draw on the building blocks of the standards and are easy to incorporate as extensions to the standards. Some of this involves prioritizing the work on LTI and Caliper to make this easy. But it also likely will require the IMS to consider changing its policies in ways that will make some stakeholders uncomfortable.

    A little history is in order here. A bit more than a decade ago, the IMS nearly died. It wasn’t generating enough revenue to sustain the work. While IMS Global is a non-profit, it still has salaries to pay, rent to cover, and so on. Operating it is not free. One of the steps that the organization made to save it from potential insolvency was to put a lot of the work behind a paywall. I don’t like it, but I get it. And it worked. The IMS appears to be much healthier now and has produced some of its best work in a very long time. Life is about trade-offs.

    It may be time to consider a new trade-off. Just like getting the telephone to take off required a network effect—the value of the network increases exponentially with the number of people on it—the value of a world of standards-based services increases exponentially with the number of services on it. To get there, it’s time to at least consider lowering the pay wall. The IMS needs to create an environment with very low barriers to creating standards-compatible integrations. A membership fee, however reasonable it may seem to the membership organization, is a barrier. The IMS leadership should think creatively about how to lower this barrier while maintaining the financial stability of the larger organization.

    The Nub of It

    To sum up, here’s what I think the IMS should do to recover from the LTI 2.0 misstep and foster a step-function change in interoperability:

    1. Promote LTI Advantage and build from there.
    2. Focus next on creating a simple security model that is trustworthy for common use cases.
    3. Focus Caliper efforts on turning it into a quick and easy to use method for creating data interoperability extensions that take advantage of the existing LTI infrastructure.
    4. Make a giant push to get developers to build their bilateral integrations using the LTI and Caliper infrastructure, whether or not they submit those integrations as extensions to the standard. This should explicitly include re-examining policies about giving non-members early access to the work being done inside the IMS in cases where such access will increase the velocity of new Caliper-compatible service generation.
    5. Wait for demand to build on both provider and consumer side before attempting another run at discoverability.
  • A Flexible, Interoperable Digital Learning Platform: Are We There Yet?

    A Flexible, Interoperable Digital Learning Platform: Are We There Yet?

    In 2005, some colleagues and I had been tasked with identifying a single LMS that could serve the needs of all 64 campuses of the State University of New York—from Adirondack Community College to SUNY Stony Brook to the two medical schools. We came to the conclusion that no single LMS at the time could meet such diverse needs. We proposed instead that SUNY should build a modular system from which each campus, and indeed each educator, could create their own fit-for-purpose digital learning environment. We called this idea the Learning Management Operating System, or LMOS.

    We were neither the first nor the last group of people to propose such an idea. We had been preceded by e-Learning Framework, which had been put forth by the UK’s Joint Information Services Committee (Jisc) a year or two earlier. In 2012, Phil wrote about the idea of  Learning Platform here on e-Literate. In 2015, the EDUCAUSE Learning Initiative (ELI) started promoting the idea of a Next-Generation Digital Learning Environment (NGDLE).

    There are some conceptual and implementation differences among these different formulations, but they share two things in common. First, they all posited that contemporary learning environments were insufficiently flexible to meet a diverse range of teaching and learning needs. Second, none of them have been implemented. An upcoming issue of EDUCAUSE Review will focus on NGDLE, in part to try to build momentum for it. I have contributed an article.

    Nearly a decade and a half after the idea of some kind of modular learning environment surfaced, why hasn’t it happened yet? How close are we? What are the barriers? Having recently returned from IMS Global’s annual Learning Impact meeting, I am convinced that we are closer than ever. But how quickly the remaining barriers fall and how well the end result works will depend heavily on what happens next in the standards-making process.

    (more…)

  • 68% of Statistics Are Meaningless, D2L Edition

    Two years ago, I wrote about how D2L’s analytics package looked serious and potentially ground-breaking, but that there were serious architectural issues with the underlying platform that were preventing the product from working properly for customers. Since then, we’ve been looking for signs that the company has dealt with these issues and is ready to deliver something interesting and powerful. And what we’ve seen is…uh…

    …uh…

    Well, the silence has ended. I didn’t get to go to FUSION this year, but I did look at the highlights of the analytics announcements, and they were…

    …they were…

    OK, I’ll be honest. They were incredibly disappointing in almost every way possible, and good examples of a really bad pattern of hype and misdirection that we’ve been seeing from D2L lately.

    (more…)

  • The EDUCAUSE NGDLE and an API of One’s Own

    I have been meaning for some time to get around to blogging about the EDUCAUSE Learning Initiative’s (ELI’s) paper on a Next-Generation Digital Learning Environment (NGDLE) and Tony Bates’ thoughtful response to it. The core concepts behind the NGDLE are that a next-generation digital learning environment should have the following characteristics:

    • Interoperability and Integration
    • Personalization
    • Analytics, Advising, and Learning Assessment
    • Collaboration
    • Accessibility and Universal Design

    The paper also suggests that the system should be modular. They draw heavily on an analogy to LEGOs and make a call for more robust standards. In response, Bates raises three concerns:

    1. He is suspicious of a potentially heavy and bureaucratic standards-making process that is vulnerable to undue corporate influence.
    2. He worries that LEGO is a poor metaphor that suggests an industrialized model.
    3. He is concerned that, taken together, the ELI requirements for an NGDLE will push us further in the direction of computer-driven rather than human-driven classes.

    As it happens, ELI’s vision for NGDLE bears a significant resemblance to a vision that some colleagues and I came up with ten years ago when we were trying to help the SUNY system find an LMS that would fit the needs of all 64 campuses, ((I understand that SUNY has since added a 65th campus)) ranging from small, rural community colleges to R1 universities to medical and ophthalmology schools to a school of fashion. We got pretty deep into thinking about the implementation details, so it’s been on my mind to write my own personal perspective on the answers to Tony’s questions, based in large part on that previous experience. In the meantime, Jim Groom, who has made a transition from working at a university to working full-time at Reclaim Hosting, has written a series of really provocative and, to me, exciting posts on the future of the digital learning environment from his own perspective. Jim shares the starting assumption of the ELI and SUNY that a learning environment should be “learner-centric,” but he has a much more fully developed (and more radical) idea of what that really means, based on his previous work with A Domain of One’s Own. He also, in contrast to the ELI and SUNY teams, does not start from the assumption that “next-generation” means evolving the LMS. Rather, the questions he seems to be asking are “What is minimum amount of technical infrastructure required to create a rich digital learning environment?” and “Of that minimal amount of infrastructure we need, what is the minimal amount that needs to be owned by the institution rather than the learner?” I see these trains of thought emerging his posts on a university API, a personal API, and a syndication bus. What’s exciting to me about these posts is that, even though Jim is starting from a very different set of assumptions, he is also converging on something like the vision we had for SUNY.

    In this post, I’m going to try to respond to both Tony and Jim. One of the challenges of this sort of conversation is that the relationship between the technical architecture and the possibilities it creates for the learners is complex. It’s easy to oversimplify or even conflate the two if we’re not very careful. So one of the things that I’m going to try to do here is untangle the technical talk from the functional talk.

    (more…)

  • LMS Is The Minivan of Education (and other thoughts from #LILI15)

    During yesterday’s K-20 learning platform panel at IMS Global’s Learning Impact Leadership Institute (the panel that replaced the LMS Smackdown of year’s past), Scott Jaschik started the discussion off by asking “what is the LMS?”. As I have recently complained about our Saturn Vue that replaced a Chrysler Town & Country, the answer I provided was that the LMS is the minivan of education. Everyone has them and needs them, but there’s a certain shame having one in the driveway.

    The Car Committee

    It’s popular to gripe about minivans, but in reality they reflect what we (the family set with kids still at home) actually are and what we do. Sure, the minivan encourages us to throw everything in the car and continue soccer mom lives, but they do offer great seating, storage, smooth rides (on boring roads at least). Likewise, the typical LMS is in actuality still a Course Management System (CMS), which reflects how courses are organized and managed in large part.

    We’re done with the boring minivan and have moved on to SUVs, but the SUV has morphed into a minivan with bad gas mileage and poor seating. It feels so nice to call it a different name, but it’s still a CMS minivan at its core.

    There are new innovations in the car market, like the Tesla. The risk we face in education is falling back on our RFP-driven habits. Great car demo, but the committee is using a family-driven process.  Item #142 includes having more than 5 seats, with a place for little Kenny’s sippy cup in each. You know what, let’s just make it taller and add a hatch in the back. Item #275 requires ethanol percentages (and we read an article that batteries are risky), so  could you add in an standard engine? Two years later . . . “dammit, the LMS”.

    Put it together, and the LMS is important and ubiquitous, but we all know we need better options. Despite this, take away the LMS and see if students like a different method to submit assignments or check grades for every class. (more…)