Editor’s Note: I am pleased to announce that Bill has agreed to continue contributing blog posts from time to time. Therefore, he is now officially a “Featured Blogger” rather than a “Guest Blogger.”
Last week, I had the privilege of speaking at a workshop on online graduate education. At that workshop, Carnegie Mellon University Provost and Executive Vice President Dr. Mark Kamlet used the words “Learning Engineering” in his keynote which I built upon in my talk. In my previous post I referenced the need of semantic data and algorithms to support learning engineers to create and iteratively improve courses and courseware (among other things). I felt it was worth taking a little time to describe just what I believe that means.
For over 10 years, the Open Learning Initiative has been bringing together teams to develop online course materials. Carnegie Mellon is an ideal place to cultivate this work due to its multi-disciplinary programs and culture aside from its expertise in the related fields. During that time we’ve built a team of experts that are critical to the building of learning environments informed by research and capable of recording data for iterative improvement as well as creating dynamic reports for stakeholders.
Discovering Learning Engineering
At OLI, we have followed a path that was outlined by CMU professor Herb Simon, Noble Laurate:
“Improvement in post secondary education will require converting teaching from a solo sport to a community based research activity.”
If you’ve seen someone from OLI speak more than once, you’ve seen this quote and might be tempted to gloss right over it. But it’s worth considering closely, particularly in this context. We have found that the best way to build effective learning environments is to regularly convene faculty, software engineers, usability specialists, learning scientists, and others.
What does it take then to be someone who can sit at the center of this kind of diverse group and produce an online learning environment that has a successful outcomes? We’ve admittedly struggled with this question as we’ve grown as a project. It turns out that part of what we were missing was trying to shoehorn people with existing skill sets into a role that is really what we’ve come to call the learning engineer.
Engineering Learning? You Bet.
Starting with the source of all knowledge, I look to how Wikipedia defines engineering:
Engineering is the application of scientific, economic, social, and practical knowledge, in order to design, build, and maintain structures, machines, devices, systems, materials and processes. It may encompass using insights to conceive, model and scale an appropriate solution to a problem or objective. The discipline of engineering is extremely broad, and encompasses a range of more specialized fields of engineering, each with a more specific emphasis on particular areas of technology and types of application.
I can’t think of a better way to describe what it is we ask our learning engineers to do. But I work with them every day. So let me draw a rudimentary comparison: Imagine a more “traditional” engineer hired to design a bridge. They don’t revisit first principles to design a new bridge. They don’t investigate gravity, nor do they ignore the lessons learned from previous bridge-building efforts (both the successes and the failures). They know about many designs and how they apply to the current bridge they’ve been asked to design. They are drawing upon understandings of many disciplines in order to design the new bridge and, if needed, can identify where the current knowledge doesn’t account for the problem at hand and know what particular deeper expertise is needed. They can then inquire about this new problem and incorporate a solution.
In this way, a learning engineer applies learning science and what is known about other relevant disciplines (user experience, for example) and pedagogy to problems developing learning environments. When designing for platforms that collect semantic data they understand the requirements of the materials they are creating and can ensure that the data collection that will be done will provide actionable results. This does not mean a learning engineer has to understand the intricacies of the algorithms that operate on data, but they need to have a sufficient understanding of the needs of that data collection.
In one way, this type of engineering is more rapid and responsive that “traditional” engineering. We can learn from the delivery of the “built bridge” just what parts are effective and what parts need improvement. (This requires semantic data in order to discern). In the comparison I’ve made, one doesn’t usually go back and make a bridge better unless something terribly wrong comes to light. Here we can monitor and continually improve our previous work as well as apply those lessons forward to new developments.
That addresses lessons learned “in the field” (practice informing sciences). In the other direction (sciences informing practice), the comparison is harder to make. If some critical flaw is discovered one might go back and “patch” a bridge. For a learning engineer, revisiting work is not a rare occurrence but an expected iterative improvement process. Thus, a learning engineer must be aware of ongoing research in related fields and stay current with our understanding of how to teach effectively. We’ve only begun to understand teaching and learning in scientific ways and cannot rest on what we know so far. Learning engineering then, as a field, is really about developing processes and methodologies to support this work.
One good point made to me by a workshop attendee after my talk: if a bridge falls down, you know about it. In the world of online education where rich evaluation is rare, we don’t even know if our bridges are falling down.
Something We’ve Needed All Along?
Although the work to advance online education has been the spark that has made obvious the need for collaborative efforts and individuals who can work in those highly interdisciplinary teams, I refer back to the quote at the opening. Simon wasn’t saying online education required the conversion of how we teach. It just so happens that it is now obvious. If we’re truly honest with ourselves, not all experts make the best teachers. This is not to say that top-tier institutions with high-caliber faculty aren’t offering a great opportunity to students by providing access to leading researchers. (“Minds rubbing against minds” as it were). But those leading researchers are not guaranteed to be the best teachers, especially when they’re often handed a course to teach as a secondary requirement to their role that they may not be interested in.
Some shared experiences of undergraduates everywhere:
- I thought I understood the lecture, but I don’t know where to start on this homework!
- That midterm came out of nowhere – I didn’t understand it.
- I read the chapter as told but then the lecture made no sense to me.
These are the result of poor alignment in objectives, practice and assessment, which is already known to be important. This is the kind of insight and experience that the most brilliant minds can benefit from when it comes to teaching the novice. (See also the expert blind spot).
A learning engineer works with content experts and guides their work and brings in other points of view as needed in order to best develop learning experiences – it just so happens that now we really need them even more for the online experience.
How to Find a Learning Engineer
The reality is that right now individuals with such skill sets are hard to find “in the wild” and it will be some time before that changes dramatically. What is required is to find talented people interested in the work who already have some of the skills needed. It could be someone with a strong learning science background who is interested in seeing immediate practical application of their work, or someone with a strong instructional design background interested in learning how to apply learning science and data analytics to what they do, and moving those groups together. That model does provide a way to find candidates and acknowledges the fact that some effort has to be made to develop the skill sets of a learning engineer upon hiring.
I do not believe this is a case of looking for what in the software world you’d refer to as a unicorn. It really is vital to all of us in education to develop a workforce of people who understand how the creation of learning material happens as well how to apply developments happening in the understanding of how to effectively develop and test those materials.
Aren’t Learning Engineering and Instructional Design the Same?
This reminds me of when I started my career as a programmer. When I started programming, I was a software developer and not a software engineer. I knew how to write code, but I wasn’t ready to architect it or account for other disciplines in my work. A similar comparison applies here. The role of a learning engineer is not a support role, but a full contributor and participant in the process of developing an online learning environment. I asked one of our learning engineers how she viewed her role, to which she said “We want to learn about learning – what makes rich, deep, meaningful and lasting impact.” She builds environments that report data so her work can be evaluated, not to ask if she did a good job, but to learn how we might improve upon what we know to better the environment.
A learning engineer is a part of the process that improves or expands the technologies they work with. An instructional designer is often handed a suite of available technologies and content and told to make something of it. A learning engineer works both pedagogically and technologically to improve, create and make a whole experience and then evaluate the effectiveness of it with data.
An Essential Field
Learning engineering is part of what drives the success at OLI and is going to drive the development of well-informed online environments going forward anywhere such work is being done in the future We believe this is an important area to define and then expand.
With that in mind, I leave you with a work in progress statement attempting to capture the key aspects of this field. (I already know it’s not easy to read, especially out loud in a talk without stopping to get your breath!) But I’m interested in hearing what others think of the content of this sentence. It doesn’t get into some of the practical implications I outline above but hopefully it captures the essence of the idea.
Learning Engineering: The development, evaluation and improvement of the processes, methodologies, and educational technologies that lead to predictable, repeatable development and improvement of learning environments which leverage learning science and the affordances of technology to address instructional challenges and create conditions that enable robust learning and effective instruction.
The way I think about it, engineers go to work after the relevant research is done. The engineer has at her disposal manuals containing the data tables and formulas she needs to do her work. The Wikipedia definition Dr. Jerome quotes from seems consistent with this. The work of science allows wide latitude for error; engineering allows for precious little. If this understanding is correct, we can, to be sure, do some of what Dr. Jerome calls learning engineering, but only with lower level cognitive functions. To talk about engineering a college curriculum seems like it’s a bridge too far.
I am all for continuing research along the lines of what Dr. Jerome is discussing, but it will be misleading to cast it as engineering until we can document proven and replicable results.
Actually, Peter, there’s a lot of evidence (from history & philosophy of technology) that it is more common for the engineering and technology to lead the science and not vice versa (but of course engineering builds off of older science and theories, it’s a reciprocal relationship). And this is definitely the case in education – we can’t wait for psychological theories to tell us how to design learning environments. We have to take the good parts of several frameworks and theories and research findings and go beyond them to make other inferences and design decisions, some of which may be wrong. I liken theories and technologies to tools in my (learning design) toolbox.
Anyway, I would say that what Jerome describes – learning engineers – is what the Learning Sciences and learning scientists are supposed to be – it’s what we strove for. Look in learning sciences journal articles and the like mentioning design-based research, Pasteur’s quadrant, and all the innovative educational technologies those (few) with funding develop (and disappear as soon as funding runs out because they weren’t open sourced). We’re supposed to be bridging the gap between research and practice. And some definitely are doing so, such as the folks at OLI. But there are several reasons many are falling short and our efforts are inadequate. I won’t go into all of them, but one thing is that in learning sciences training, no one ever learns basic computational literacy or the skills to develop technological solutions (we never learn the D in R&D). I think at Vanderbilt I was just about the only one in the whole college of education who knew how to program. I guess at Penn State it was rare, too, because someone wrote an article about his student who had computational fluency: http://www.personal.psu.edu/bks12/papers/dc178707.pdf
On the flip side, the technological/development side, one thing that is holding me back now is the high cost of hosting very interactive and scalable web applications (cloud PAAS platforms get expensive quick). I could get funding, but what about when it runs out. And unfortunately the last two universities I’ve been at have been hostile to hosting servers and supporting custom software development (I was very spoiled at Georgia Tech and Vanderbilt I now realize). I’m hopeful for more developments in the type of web applications that run entirely in the browser and only connect to a server occasionally for persistence and synchronization, through tools such as html5, pouchdb, unhosted apps, node.js, etc.
My response is that a hallmark of educational technology, since Edison at least, has been to consistently over-promise. I started an OLI course about a year ago. Very slick. Taking the assessments gave me the exhilarating sense of forward motion. But when I paused to do a self-assessment, I discovered that I had retained very little. More recently, I had the same experience with Udacity. This is evidence that is anecdotal in the extreme, I know, (N=1) but it was very persuasive to the subject. I still believe that if people in the field went out of their way to acknowledge that they are still working on first principles of complex learning, they would approach the task with more intellectual honesty, and make more progress as a result, and they would address the public with far less hype. We live in a time when there is someone who will sell you a remedy for every ailment. I don’t want to be in that crowd.
Educational Technology is indeed transforming learning processes. The article on ‘Learning Engineering’ which was very interesting. We offer a Course titled, ‘Principles and Practices of Learning Engineering’ at our University, since 2009, (NIIT University, India). Our endeavour is to strengthen learners thru identification of their learning abilities, strengths and expertise and then work on innovative pedagogical models of learning. Once, the learners establish confidence in learning, the outcomes are fantastic.
Since, Educational Technology encompasses all disciplines, Learning engineers contribute more holistically and their insights are metacognitive in nature, thus adding tremendous value to different disciplines.
regards,
Dr. Shradha Kanwar