WGU Is Not Off the Hook

In Phil’s first piece on the Department of Education’s Office of the Inspector General (OIG) finding the Western Governors University (WGU) should be considered a correspondence provider rather than a distance education provider, he wrote,

This audit is a travesty in my opinion. Even though it is likely to be rejected by the ED itself, it will have an impact, and the internal review of the audit will likely take years.

The problem, in a nutshell, is that the OIG decided that WGU’s unbundled instructor’s role, with multiple staff roles supporting students in a (largely) self-paced environment, does not count as “regular and substantive interaction between students and teachers,” which is a requirement for classification as a distance learning provider.

Phil believes that this assessment by the OIG was arbitrary and, based on my admittedly limited understanding of their assessment process, I tend to agree. But that doesn’t mean that the OIG is wrong. It means we don’t know whether the OIG is wrong. And the heart of the problem—the definition and test for “regular and substantive interaction between students and instructors”—is a real challenge. While feel fairly confident that the OIG applied too narrow an interpretation of a standard that problably needs to be revised anyway, coming up with a better evidence-based standard is tough. And if we don’t have one, we can’t tell if WGU’s programs should be considered equivalent to more traditional distance learning programs.

There are two positions that one could take in arguing against the OIG finding: (1) that it is possible to deliver the equal of a traditional education without regular and substantive interaction between students and teachers, or (2) that this interaction is necessary but we need a different, perhaps more flexible definition of it.

Let’s look at each of these in turn.

Position 1: The Standard is Unnecessary

The more radical of the two positions is that “regular and substantive interactions between students and teachers” is outdated in the sense that such interaction is not necessary for a quality distance education program. In this view, good design and good technology provide enough support for self-paced students. People who take this position tend to have a high opinion of the impact of technology, a low opinion of the impact of the average instructor, or both.

I’m not aware of any research that definitively settles this particular debate and would be surprised if there were any. In fact, I’m not sure it’s possible to produce such evidence in principle, because there are too many contextual factors to come up with just one answer. Some students in some programs studying some subjects to some level of achievement may do as well (or better) in a self-paced, largely self-guided competency-based program as they would in a traditional instructor-led setting. There would need to be an enormous amount of research, including some foundational research that we don’t have yet, to sort out all of the many “ifs” that determine the circumstances under which such a program would be equivalent in effectiveness.

I think it’s dangerous to assume that “regular and substantive interaction between students and instructors” is an obsolete standard, and I do think there are at least three strands of research with results that should give us pause about being too aggressive about taking human teachers out of the equation.

First, there’s Benjamin Bloom’s research on the Two Sigma Problem. Since I recently wrote about this in some detail as part of a longer post, I’ll give you the short version. Bloom found that by using tutors in a mastery learning context, he could achieve two standard deviations of improvement over standard instruction. One could argue that WGU’s model of self-paced learning with periodic assessments and support from course mentors attempts to imitate Bloom’s approach (although one would have to look closely to see whether the degree to which they are actually doing so). The relevant detail for our current purpose is that Bloom could never isolate exactly what it was about the tutors that delivered that second sigma. Without understanding the reasons why having a human tutor involved improves student outcomes by as much as a full course grade, it seems imprudent to assume it can be removed or replaced.

Second, there’s the research conducted by Gallup and Purdue University showing that college graduates were 1.7 times more likely to thrive in all five of Gallup’s measures of wellbeing—physical, financial, community, career, and social—if they agreed with the statement “My professors at [college] cared about me as a person.” They were 1.5 more likely to thrive on those measures if they answered agreed with the statement “I had at least one professor at [College] who made me excited about learning.” Those are pretty compelling results, and it’s hard to see how one would replicate them without some form of regular and substantial interaction between students and instructors. For more on this study, see my post on it.

In a follow-up piece to that post I just referenced, I talked about the third strand of research from Vincent Tinto. He showed that students are more likely to persist at college if they feel a sense of belonging. “[S]Students have to come to see themselves as a member of a community of other students, faculty and staff who value their membership.” Yet again, there is evidence of impact for a human factor that argues in favor of regular and substantial interaction between students and teachers.

To be clear, I’m not suggesting that one could not create an educational system that provides real value without such interaction. But it would probably be a different kind of education that provides different kinds and levels of value. The OIG is concerned with classification and equivalence: Is WGU providing educational value that’s similar enough to more traditional distance learning programs that it can be classified as the same type of degree? I don’t think we can let the university off the hook by dismissing the “regular and substantial interaction” requirement as obsolete.

Position 2: The Standard Needs Revision

The more conservative argument against the OIG’s evaluation of WGU’s courses is that we still need a standard for “regular and substantial interaction between students and teachers,” but that our interpretation of that standard should be more flexible than the one that the OIG applied. Ideally, there would be some sort of evidence-based test. Let’s see if we can imagine what such a test might look like, based on the three research strands I mentioned above.

It would be hard (and probably pointless) to try to replicate Bloom’s highly controlled laboratory experiments which took place in a very different schooling context. But we might get something from the spirit of the experiment. Simply put, can we come up with some sort of rough measure of the impact of the instructors (or the various folks who individually or collectively fulfill the instructor’s function) on mastery of materials? Can we find evidence of impact? One place might be to look at variance in student performance between instructors teaching the same material. If there is substantial variance that can be reliably attributed to instructors, then WGU could argue that their courses have enough student/instructor interaction to make a difference.

The Tinto and Gallup/Purdue research would be relatively straightforward to draw upon, since they both use student attitude surveys. But only relatively, because I haven’t seen studies applying any of these instruments specifically to distance learning programs. (If anybody knows of such research, please let me know.) One would need to establish a baseline. But that seems like a good idea anyway.

So there are probably a number of ways that the OIG could establish an empirical test to find evidence of student/instructor interaction that is regular and substantive enough to pass an equivalence threshold. It would probably be crude, but a crude test is better than no test at all, which appears to be what we have now.

I don’t know if the OIG assessment of WGU’s courses was wrong. I feel fairly confident that it was made arbitrarily. But the fact that we have no reason to believe that it is right is not the same as saying we have reason to believe that it is wrong. The reason that bears repeating is that, defined this way, the problem exists not only for WGU but for every assessment that the OIG makes. If the standard is completely subjective and therefore inherently arbitrary in its application, then it is meaningless.