WGU Audit Findings: Interpretations of “regular and substantive” and “self-paced”

The big news this week was the Office of the Inspector General (OIG) at the Department of Education (ED) finding that Western Governors University (WGU) should be considered a correspondence provider instead of a distance education provider, and the school should return more than $700 million in Title IV federal funding programs.1 In short, being ruled a correspondence provider would mean that most student loans and Pell grants would not apply for WGU students and would all but shut down the institution or make it irrelevant. While the ED itself is unlikely to follow these non-binding recommendations, this ruling will have a big impact for years. More on that in tomorrow’s post. For now, the title loans for college students are the best option right now.

It would be useful to first review the actual audit findings, especially since most media reporting focused mostly or solely on the issue of “regular and substantive interaction”, but the findings are broader and also encompass issues are self-paced vs. credit-hour / term-based education.

The audit started over four years ago and primarily focused on 102 courses (out of 980) offered in the 2013 – 14 academic year. No one seems to know why the OIG started this audit, but the audit report itself makes it clear that quality was not the issue (page 6):

we did not assess whether the school’s model was improving educational quality or expanding access to higher education.

WGU’s regional accreditor,  Northwest Commission on Colleges and Universities, has accredited WGU as a term-based distance education provider, including reaffirming the accreditation in February of this year. The Department of Education explicitly allowed WGU to be classified as a distance education provider as part of the Distance Education Demonstration Program from 1999 – 2005 and granted further waivers and agreements in April 2005.

By my reading, the audit is a model of hyper-literal translation of ambiguous regulations, leading to three findings.

• Finding 1) Course Offerings Met the Title IV Definition of a Correspondence Course, Not the Title IV Definition of Distance Education – This finding was centered on reviews of course materials for 69 courses as well as a mapping of WGU’s unbundled faculty role to traditional instructor definitions. The OIG found that using the courses did not meet the interpretation of regular and substantive interaction required of distance education courses.
• Finding 2) Western Governors University Disbursed Title IV Funds to Students Before the Students Were Eligible to Receive the Funds – This finding was based on additional mapping of WGU’s CBE model to traditional term-based model. WGU itself decided to consider itself a term-based institution, mapping one competency unit to one credit hour over a 26-week academic term, and the ED recognized this classification in the 2005 agreement. The OIG, however, found that WGU should have been classified as a non-term school using self-paced programs. There are a different set of regulations for non-term programs.
• Finding 3) Western Governors University Did Not Always Comply With the Requirements Governing the Return of Title IV Funds – This finding is important, but it deals with detailed bureaucratic rules upon student withdrawals. I’ll let someone else look at this finding.

It is the combination of findings 1) and 2) that are important not just to WGU but to any school developing an online or hybrid non-lecture-based approach.

Regular and Substantive Interaction

WGU has been at the forefront of breaking apart the traditional faculty role, instead using mentors, evaluators, and other interdependent roles. The audit acknowledged that (page 15):

Northwest Commission recognized Western Governors University’s student mentors, course mentors, evaluators, product managers, and council members as members of the school’s faculty. The accrediting agency also distinguished between the roles of student mentors and course mentors, characterizing student mentors as serving in academic advisory roles and course mentors serving in instructional roles.

The OIG used a binary role-based approach (you are an instructor or you are not) leading to conclusion that only course mentors and evaluators could be considered as instructors, however. The basis of this determination was an instructor must “provide instruction on course content” – clearly a content-dissemination view that rejects alternative pedagogies. And this interpretation that the OIG treats as unambiguous is not based on law, regulations, or commonly-accepted educational terminology.

The OIG looked at the ambiguous regulations and chose their own, very literal, interpretations (page 14):

Because the HEA and Title IV regulations did not define instructor, substantive, or regular, we considered the ordinary meaning of those terms when assessing whether the school designed the 102 courses to offer regular and substantive interaction between students and instructors. We reviewed the school’s course design materials for evidence of interaction that was not primarily initiated by the student and was (1) with someone who instructs or provides knowledge about the subject matter of the course (instructor), (2) relevant to the subject matter (substantive), and (3) occurring with some reasonable frequency considering the school-suggested length of the course (regular).

This is why I call the audit methodology as hyper-literal. Somehow the OIG thinks they can determine – without any disagreement or ambiguity – the “ordinary meaning of those terms” based on their own interpretations.

Also note that the determination was entirely based on course design materials – think syllabus and course outlines. The OIG did not look at interactions arising during the course of actual course work, just whether there were pre-defined webinars, meetings, and student-instructor interactions. The OIG did eliminate many interactions as not being “substantive” (page 16):

After identifying the employees who could reasonably be considered instructors, we determined what type of interactions could reasonably be considered substantive. We considered an interaction to be substantive if the course design materials described student interaction with a course mentor or required an individual submission of a performance task for which an evaluator provided the student feedback. We did not consider the following to be instances of substantive interactions between students and instructors:

• Objective assessments that students submitted for evaluation because feedback on objective assessments was computer-generated, was not provided by instructors, and did not facilitate synchronous or asynchronous interaction between students and instructors.
• Recorded webinars, videos, and reading materials if the course design materials did not require the students to watch the webinars or videos and then interact with an instructor. Many course outlines stated only that course mentors were available to students for assistance if the student wanted to contact the course mentor. Had the course design materials indicated that the recorded webinars, videos, and reading materials facilitated synchronous or asynchronous interactions, such as requiring the student to contact an instructor or participate in an online discussion moderated by an instructor, we would have considered those instances to be substantive interaction.
• Contact with student mentors because the accrediting agency’s recognition, the school’s description of the student mentor’s role, and our interviews with six student mentors disclosed that student mentors did not provide instruction on the subject matter of the courses that students were taking.

To be “regular”, the OIG required that all interactions be pre-planned, with the right people, in the course design materials (page 16).

We did not find any evidence in the course design materials for 69 courses that would provide a reasonable basis for concluding that planned student interactions with course mentors and evaluators could be considered as occurring with some reasonable frequency (regular). The only evidence of regular interaction was student contact with student mentors. However, student mentors did not provide instruction.

In its comment on the draft findings, WGU complained that OIG (page 20 and 26):

did not count, as regular and substantive interaction, significant interactions not described in course outlines.

OIG’s response basically agreed with this complaint:

We considered all events described in course outlines and pacing guides, along with calendars of live events referenced in those materials. If substantive interactions were not described in any of these course design materials, we had no reasonable basis to conclude that such interactions were part of the design of the courses and did not consider them as planned course requirements. [snip]

Course mentors might have identified students who were struggling, and many course outlines instructed students to contact course mentors if the students needed assistance. However, if course design materials did not describe the interaction, there was no reasonable assurance that students had any regular and substantive interaction with course mentors.

These views essentially reject not just WGU’s approach to CBE but also the broader movement of faculty from “sage on the stage to guide on the side”. Instructors, from the OIG view, must provide instruction on course content and interactions must be pre-planned in the course design materials, at least for online courses.

Self-Paced

While I assume there were good reasons for WGU to want to avoid being classified as a non-term school, their 2005 declaration that they were not self-paced but rather term-based now appears to be a self-inflicted wound (page 34).

Western Governors University Comments
Western Governors University stated that its courses were not self-paced. Students were given a pacing guide and were expected to complete a certain number of competency units each term. In addition, a key role of the student mentor was to guide the pace of academic progress with individual students to ensure course completion by a certain date. Students had some flexibility in the pacing and moved through the content at different rates to allow for their individual competency development. However, that does not mean that the courses were self-paced.

OIG Response
Western Governors University’s statement that courses were not self-paced is contrary to its advertising materials, pacing guide descriptions, and statements from school officials we interviewed. According to the school’s web site, students could complete a degree program as soon as they successfully completed all of the necessary assessments. Students who completed assessments quicker could complete their degree quicker. The school’s web site also stated that students could complete assessments as soon as they were ready; if they were already competent in a subject area, they could prove it faster and complete their degree faster. If students completed their programs in less time, they paid only the tuition for the number of student terms in which they needed to enroll to complete the program. Course outlines stated the following about the pacing guides: “The pacing guide suggests a weekly structure to pace your completion of learning activities. It is provided as a suggestion and does not represent a mandatory schedule.” Western Governors University’s Program Development Operations Manager, Director of Assessment Design and Development, and Associate Provost for Academic Services confirmed to us during the audit that courses were self-paced.

Ouch. It is far different for OIG to arbitrarily pick their own interpretations of regular and substantive interactions than it is for OIG to use WGU’s own descriptions and interviews. This seems to be a strong argument by OIG.

WGU Response and Web Site

WGU’s response was dated May 22 of this year, so they have had time to prepare for the audit findings. They have a web site that explains the situation, answers basic questions, and highlights their arguments against the findings.

WGU strongly disagrees with the Inspector General’s audit report, which challenges our innovative, results-proven faculty model.

With this key arguments:

WGU has complied with the higher education laws and Department of Education guidance since our founding 20 years ago. Students, graduates, and employers of our graduates can rest assured that WGU’s accreditation and financial aid eligibility are intact.

Our accreditor, the Northwest Commission on Colleges and Universities, has approved our faculty model and reaffirmed our accreditation in February 2017. Accreditors are responsible for determining whether a university is eligible for federal financial aid.

Fixes Needed

I agree with both Russ Poulin and Amy Laitinen regarding the need to fix but not remove the “regular and substantive interaction” regulations. From Inside Higher Ed’s coverage:

Russell Poulin, director of policy and analysis at the WICHE Cooperative for Educational Technologies, said the department has done a “horrible” job of informing colleges about its expectations of how to comply with the regular-and-substantive requirements, which he said have changed over time.

In addition, he criticized the inspector general’s decision to base its compliance position on disagreement about the mode of teaching at WGU when there is no evidence of any harm to students.

“I totally agree with the intention of proponents of the ‘regular-and-substantive interaction’ rule, which is to avoid fraud. But it is an outdated method of reaching that goal,” he said via email, comparing it to a hypothetical decision by regulators to remove all ATM card readers because of the risk of credit card skimmers. [snip]

Amy Laitinen, director of higher education policy for the group and a former Obama administration Education Department official, said the law was a response to rampant fraud and abuse.

“We need to carefully fix (not gut) the now-outdated law to ensure that students are getting the academic and other supports that they need,” she said via email. “If we don’t do it carefully, it will be a fast race to the bottom, which would be bad for students and bad for the competency-based education community.”

The Worst Part

To me the worst part of the audit is the language used by the OIG that tries to make any disagreement with a hyper-literal translation of ambiguous regulations seem to be invalid (page 3).

None of these 69 courses could reasonably be considered as providing regular and substantive interaction between students and instructors.

Who holds these “unreasonable” views? The Department of Education and the Northwest Commission are two groups, among others. It’s bad enough that the OIG took the out-of-context, hyper-literal approach to the audit, but to not acknowledge the ambiguity and lack of clear guidance about this requirement is disingenuous.

This audit is a travesty in my opinion. Even though it is likely to be rejected by the ED itself, it will have an impact, and the internal review of the audit will likely take years. I’ll write more about potential impact of the audit tomorrow.

Additional Reading

• OIG Audit
• WGU Response Site
• IHE Coverage (best news article)
• Chronicle Coverage
• EdSurge Coverage
• WCET Analysis (best summary of issues)