I have some announcements and updates to share regarding the Standard of Proof webinars. All good stuff, which I’m sure we could all use more of these days. As a general reminder, Standard of Proof webinars are always about contributions that vendors have made to our general knowledge or capabilities for supporting students—i.e., contributions that don’t just help their own customers—and they always include academics who were meaningful partners in the work. Companies featured in the Standard of Proof webinars are all sponsors of the Empirical Educator Project (EEP), but sponsorship does not guarantee a webinar. Only a qualifying contribution to the educational commons, as judged by me, earns them a webinar. This is not just promotional fluff. So I’m pleased to share the latest with you, which you’ll see represents an uptick in our tempo.
First, the video archive of the webinar on the LXPathways learning architect and instructional technologist competency programs from iDesign is now live on YouTube. Or you can just watch it here, if you have the hour:
iDesign LX Pathways Standard of Proof webinar
Second, a reminder that we have a webinar tomorrow with Macmillan on their method for studying the effectiveness of their products early in the product development cycle. I serve as a (paid) member of the Macmillan Impact Research Advisory Council (IRAC), so I have had both a ring-side seat and some input on this process. The transparency they are attempting to provide regarding what they know about product impact, how they know it, and how confident they are in the results at any given moment is commendable. I’m looking forward to that conversation tomorrow, i.e., Wednesday, March 10th at 2 PM ET. Register for the Macmillan efficacy methodology webinar here.
Third, you may have seen an announcement go out earlier today about our webinar for next Wednesday, March 18th at 2 PM ET for our Driving Toward a Degree webinar with Tyton Partners. We still have a lot to learn about increasing graduation rates, particularly among first-generation, post-traditional, and underrepresented students. Tyton has made an important contribution with their finding of the value of a coordinated response that far too few colleges and universities even attempt to achieve for this sort of effort. Their methodological approach to studying the impact of different approaches is also really interesting. Register for the Driving Toward a Degree webinar here.
And finally, I have a hold-the-date pre-announcement for a follow-up webinar on the research AdmitHub has been doing. (You can see the video of our webinar with them on summer melt here.) This research has broader implications for helping students get to graduation. I’ll hold the details until we have the description nailed down with the participants—this is new research, so I always want to get the characterization right—but for now, save the date for a webinar on Thursday, April 9th at 2 PM.
I have long argued that the best path to long-term sustainability for many colleges and universities is to serve as many students within their geographic reach as possible, as well as they can, for 20 to 40 years rather than for two to four years. And that in order to do this, they need to transform from organizations with philosophical commitments to student success into ones that are operationally excellent at enabling student success. In other words, we need more student-ready colleges.
The reasons for this need are becoming increasingly clear and uncontroversial. On one hand, both real and hypothetical alternative revenue sources are not materializing. States are not dramatically increasing funding for public universities and in some cases continue to do the opposite. Demographics are reducing the number of “traditional” students who would almost automatically apply to their local schools. Grant funding is down. And distance learning reach to non-local students is proving to be limited. On the other hand, today’s workers are in constant need of more education as their jobs continually change. We expect that trend to accelerate. All of this argues that the way forward for many colleges and universities is (a) to increase the enrollment and success rates of students who haven’t traditionally applied to or completed college, and (b) maintain close, continual, long-term relationships with graduates by offering them “small bite” opportunities at education that fit with what they need to stay current and get ahead in their career paths.
Academic institutions were not designed for this (to the extent that they were designed at all). And I think that we have been singularly bad at talking about change management. There’s a lot of finger-wagging and angst but not a lot of clarity about how to get the front-line folks from here to there. For all our talk about the future of work for the whole world, we seem to have a blind spot for the future of work in academia. We’re not thinking about it, never mind planning for it.
One way to think about both the Empirical Educator Project and the broader new mission of e-Literate is as just that: helping to clarify the future of work for academia. To that end, I am calling out two e-Literate events happening this week that are thematically related to this idea.
Developing a growth mindset
Carol Dweck’s idea of a growth mindset is a deceptively simple one. In contrast to a fixed mindset, where we believe that we are who we are, fundamentally limited by the talents and abilities we are born with, the growth mindset is the disposition that talents can be developed and that we can become more than who we are now. As Dweck often writes, it is easy to oversimplify the idea in practice. But this summary is at least a good place to start.
If academic institutions are going to be resilient in the face of the massive changes they face in their environment, then academic workers will need to cultivate a growth mindset. On an individual level, they will have to define their professional identities not just on what they have already been trained to do but what they could learn to do on the path forward from where they already are. And on a cultural level, they will need to engage in positive conversations about how they could grow together to meet the future. Strategic planning in the context of shared governance and an environment of rapid change is all about cultivating a growth mindset. It only works when the participants are willing to let go of their customary roles and explore new ways that they can constructively participate in defining both the future of the institution and their roles in it.
I’m proud to announce that Susan Baldridge will be writing (at least) a couple of guest posts on this topic. I met Susan at Middlebury College where, as Provost, she created a new role of Associate Provost for Digital Learning as one important element for fostering new thinking among the faculty about how Middlebury could continue to thrive as recognizably Middlebury in the 21st Century.
Since leaving that role and starting her own consulting business, she has co-authored a forthcoming book with Bob Zemsky and Susan Shaman called The College Stress Test: Tracking Institutional Futures Across a Crowded Market. Because the authors developed a quantitative method for identifying sustainability risk levels for colleges, the academic press wants to write stories about the output of that method. They want to write about which schools are most in danger of going bankrupt. But focusing on the output misses the point of the book. Rather, the authors’ goal is to provide academics with new analytic tools which, when coupled with a growth mindset in a functional shared governance process, will enable institutional communities to steer clear of rocky shoals and chart a course toward safer waters.
I have offered Susan space here on e-Literate to frame the conversation she would like to encourage us to have. She has so far said that she will write two posts, the first of which will go live later this week. If we are lucky, she won’t stop there.
Guided pathways
The second “happening” I want to talk about is not new but rather a reminder: On Thursday, February 20th at 2 PM, I’ll be hosting the e-Literate Standard of Proof webinar, Competency-based Credentials for Educational Professionals: LX Pathways from iDesign. I had the pleasure of attending the IMS’s summit on digital credentials last week, which put this webinar in a new perspective for me. What are digital credentials, anyway? What’s the point? What are badges, how do they tie into guided pathways, and how do those, in turn, connect with a growth mindset or the future of work?
One of my realizations of the week is that these things are all tied together. A badge is just a way of signifying that something that you did has some value to somebody. A digital badge is designed to do this in a way that hopefully makes it possible for people (and machines) to see some evidence that connects the thing you did to the value being assigned. And it’s a small step, which is the point. If you are trying to move somebody from a fixed mindset to a growth mindset, you want them to have the experience of feeling that something they did counts. And you want them to have that feeling often because it is addictive. Achievement is addictive. We get a little rush of dopamine. My Apple Watch told me today that I am on a “streak” for my standing and moving goals. It also gave me a badge for my “longest move streak.” We sometimes call that strategy of frequent reward through recognition “gamification.” When the person internalizes their ability to earn those little rewards, that represents movement toward a growth mindset.
At some point under the right circumstances, those rewards can add up to something. The participant can “level up.” In modern academic parlance, this is called a “guided pathway.” It both cultivates a growth mindset and channels that impulse toward the participant’s real-world goals rather than just satisfying the immediate need for a sense of accomplishment. And micro-credentials, like turns in a game that can be saved and returned to later, are durable. Part-time learners who are squeezing their further education in around the rest of their lives typically need to “save the game” and come back to it often. They need an easy way to save their place and pick up where they left off, but also to choose a different path forward from there depending on how they have changed since the last time they were engaged in the coursework. Maybe they thought they wanted to be an instructional designer but since learned that they have an affinity for—and maybe a job opportunity as—a learning architect. The badges they earned that are relevant toward both paths should help them evaluate and accelerate their progress on their revised path.
One of the aspects that I like about iDesign’s LX Pathways competency-based program is that it has all of these features. Learners work toward one of several certificates of 21st-Century academic work skills, like instructional technologist or learning architect. They do it one competency at a time. Competencies that apply to multiple credentials give learners the flexibility to make different choices at a fairly granular level. This is a model that we need to think about providing for academic workers if we want to support the future of academic work. This, in turn, is something we need to think about if we want our academic institutions to continue serving society well in the next millennium.
These are not the first two e-Literate/EEP efforts in the “future of work for academic workers” theme, but they are the first two that I have explicitly labeled as such. I will be developing this theme going forward.
In a recent post, I wrote about my experience on the EDUCAUSE exhibition hall looking for vendors with proof that their products actually help students and how this is an example of the more general problem:
Right now, higher education has very poor signals to quickly distinguish between those vendors who can prove that their products are effective and those that can’t. Between those vendors who are contributing to the general state of knowledge about how to improve student success and those that aren’t. Until we can improve the signal-to-noise ratio, we are doomed to ride the roller coaster from hell that is the hype cycle until we pass out from exhaustion.
I’m going to do something about improving the signal-to-noise ratio, and I will be announcing what that something is within the next week.
Today I am pleased to announce the Standard of Proof ongoing webinar series, which attempts to do just that. The first is scheduled for November. But the plan is to have at least one a month starting in late January, after folks have returned from winter break.
How webinar topics are selected
For many years now, I have turned down requests to facilitate paid webinars because it’s very hard to do them without sliding down a slippery slope and either appearing to be a shill or actually crossing over the line and becoming one. And yet, I would like to give webinars that highlight genuinely good work, and I would like to get paid for my time so that I can afford to do this work.
The Empirical Educator Project (EEP) has given me a novel way to thread the needle. Vendors don’t pay me directly for a webinar. Instead, they sponsor EEP. I vet prospective EEP sponsors—they have to have something to contribute beyond just money—and I also price sponsorship to incentivize active participation and contribution. Standard of Proof webinars are now another incentive for vendors to contribute.
A Standard of Proof webinar is not an automatic benefit of EEP sponsorship. Each webinar is about not a company but a contribution to the work of empirical education. It can be, and often is, novel research conducted by the company that (a) contributes to our knowledge of how to help students succeed, and (b) adds to our knowledge in a way that is generalizable beyond their product or service. In other words, it is a contribution to the commons. But the contribution can also be a project that helps disseminate established research-backed practices or generally promote the culture of empirical education. A contribution should either be free if it has no maintenance costs or non-profit if it has does have a cost to maintain.
Contributions also have to have been developed with some meaningful collaboration with or peer review from academics. As with the definition of “contribution” above, I am more interested in the spirit of meaningful collaboration than in setting up narrow rules. But there has to be an academic who was involved enough in the project that she or he is willing to speak publicly about their participation and their views of the project.
While EEP sponsors are not guaranteed a minimum number of webinars in return for their sponsorship, neither are they limited to a maximum number. If they have made multiple contributions and I have time to cover them in the Standard of Proof schedule, I will. If that means running more than one webinar a month, I would be happy to have that problem.
The end goal here is to help vendors that meaningfully contribute to education differentiate based on those contributions. Providing credible evidence that their solutions support student success is part of that work, but the bar is higher than that. The webinar series title, “Standard of Proof,” refers not only to the products but to the vendors. The series highlights acts of good citizenship that promote empirical education. By raising the profile of these acts and the vendors who perform them, I hope to influence the criteria by which prospective vendors are selected and therefore their incentives for behavior.
Folks, this only works if you and your institutions decide that you will prioritize good behavior—including but not limited to credible proof that a product is effective—in your vendor selection. The most that I can do is help make that good behavior a little more visible.
The first webinar
As I wrote earlier in this post, the first webinar coming up later this month. I have two more lined up for early next year that I will be announcing at a later date and more in the pipeline.
Summer melt: A randomized controlled trial
We’ll be kicking off the series a randomized controlled trial by AdmitHub and its academic partners to advance our understanding of a critical phenomenon that influences the success of first-generation students.
Folks involved with admissions at access-oriented schools may be familiar with the term “summer melt.” It’s when students are admitted to a university, say they are coming, and then never show up. And it tends to be a first-generation student problem. There are all kinds of logistical and bureaucratic hurdles in getting from admissions to first day of classes that may be harder for first-generation students, either because the processes are not designed to accommodate their situations or because they don’t have anybody in their lives who can help them. Maybe they need an immunization certification and they don’t know how to get it. Maybe they need the signature of a parent who they don’t know how to contact. This is a problem that is often caused by a number of small contributing factors, and it is hard to diagnose because the people who know what kept them from coming to college…never came to college. So having a dialogue with them after the fact is difficult to arrange (not to mention unhelpful for the student, who has already missed an opportunity).
There are a number of products on the market, particularly machine learning-enhanced chatbots, that purport to address the summer melt problem. AdmitHub is one of them. But rather than just claim that they can help, they collaborated with their major customer, Georgia State University (GSU), and an academic researcher from the University of Pittsburgh who studies summer melt, to conduct a rigorous study of the impact of the AdmitHub-enabled interventions on summer melt at GSU. The results of the study? GSU was able to reduce summer melt by 20% using AdmitHub.
In addition to providing credible evidence for the product’s effectiveness, this study’s results and experimental design both contribute to our general knowledge of summer melt. We’ll be talking about that as part of the webinar, which will include the PI Lindsay Page from Pitt as well as Tim Renick from GSU, who happens to be a personal hero of mine.
We just finished the third of our three-part webinar series on“The Modern Learning Platform: Not Your Mother’s Old LMS”, hosted by our friends at NobleStream. In the first discussion, Michael and I laid out our vision for why we are analysts for the LMS market and what we hope to achieve, capped off by Michael’s comment “Depending on how you define it, I have a feeling that, for better or for worse, long after humanity is dead and gone, cockroaches will still be using the LMS.” In the second discussion, we took the institutional view and had Pat James and Anna Stirling, from the California Community Colleges, talking about how they approached LMS selection and in particular how they focused on the academic and strategic needs first, treating the LMS as plumbing to enable the house to be built. In the third discussion, we took the vendor view and had legendary Ray Henderson take a break from his well-deserved steelhead fishing excursions and describe how vendors understand and manage the LMS vendor selection process. This conversation was capped off by my mention of “the miniature elephant in the room”, where strategically-beneficial LMS decisions are typically made in spite of the RFP process, not because of it.
Michael and I thoroughly enjoyed these discussions and we hope you will, too.
We have now had 2 of our 3 webinars (or colloquiums) with NobleStream organized around “The Modern Learning Platform: Not Your Mother’s Old LMS”. In our first webinar, Howard Weiner from NobleStream interviewed Michael and me about the work we do as consultants and market analysts. Video available here. In the second webinar, we had Pat James, executive director of OEI, and Anna Stirling, director of @ONE training and professional development. OEI is the Online Education Initiative from the California Community Colleges, and @ONE is providing key elements of OEI’s faculty and student support services. This Wednesday (Nov 16th), we have Ray Henderson to discuss the vendor perspective, but more on that at the bottom of this post.
This was a fun discussion with Pat and Anna, as we discussed the LMS decision in context of overall academic strategy, using OEI as a case study. We seemed to end on the metaphor of the LMS as plumbing that needs to be in place to build the house you need. For OEI, the house is the course exchange as well as a shared set of services to help faculty develop online courses throughout the system. Resources, quality rubrics and design support, professional development, support students, etc.
Coming next: we have Ray Henderson as our guest on Wednesday, Nov 16th, at 2pm EST to discuss the vendor perspective. Ray has a long history on the vendor side, working with eCollege, ANGEL, and Blackboard as a top executive, and he current sits on the board of several ed tech companies. This should be a great discussion. Sign up here for the third webinar.
Thanks to the good folks at NobleStream, Michael and I are participating in a series of three colloquiums (or colloquia or webinars, you choose) hosted by Howard Weiner. This past Wednesday, the three of us discussed the vision that Michael and I have for e-Literate and our consulting practice with a focus on learning platforms. In fact, the title of the series is “The Modern Learning Platform: Not Your Mother’s Old LMS”.
At one point Howard asked us about why the LMS is so important and why we started our subscription service around the LMS, to which Michael responded in Feldstein fashion (question starting at 39:00):
Depending on how you define it, I have a feeling that, for better or for worse, long after humanity is dead and gone, cockroaches will still be using the LMS. It’s just this durable category. People talk about it dying or disappearing or morphing, and what happens is that it slowly evolves. It’s become this critical piece of infrastructure.
Michael and I will be participating in two upcoming EDUCAUSE webinars.
Massive and Open: A Flipped Webinar about What We Are Learning
On Thursday, December 4th from 1:00–2:00 p.m. ET we will be joined by George Siemens for an EDUCAUSE Live! webinar:
In 2012, MOOCs burst into public consciousness with course rosters large enough to fill a stadium and grand promises that they would disrupt higher education. Two years later, after some disappointments, setbacks, and not a small amount of schadenfreude, MOOCs seem almost passé. And yet, away from the sound and the fury, researchers and teachers have been busy finding answers to some basic questions: What are MOOCs good for, and what can we learn from them? Phil Hill and Michael Feldstein will talk about what we’re learning so far. (more…)
