Tag: Audit

One issue that almost all observers seem to agree upon is that the Department of Education is unlikely to accept the Office of the Inspector General’s (OIG’s) recommendations to declare Western Governors University (WGU) a provider of correspondence courses and to force the school to pay back more than $700 million in Title IV funds. It would be a mistake, however, to dismiss the audit findings that attempted to add new interpretations of distance education requirements for “regular and substantive interaction”, a topic I mentioned on Friday.

Ah – where we differ is 'CBE juggernaut'. I think movement is fragile and this audit, even if unimplemented, will affect other decisions.

— Phil Hill (@PhilOnEdTech) September 22, 2017

One important variable in this equation is timing – how long will the ED take to review the audit findings and make official decisions on adopting or rejecting the recommendations? Based on history, the decision could take years, as described by Michael Goldstein, lawyer at Cooley LLP in a WCET post by Russ Poulin:

The IG’s report and recommendations go to Federal Student Aid, which decides what, if any, action should be taken. (The “if any” is directly from the IG transmittal.) That involves a further, and often lengthy, review process. The ultimate decision authority is the Secretary.

I’m not counting on institutional leaders using the argument “I’m not worried, Betsy DeVos has my back”. The longer the review process drags on, the bigger the impact.

Another important variable is the extent to which the ED will reject the findings – will they reject the audit in its entirety and take no action, or will they accept some of the findings? Keep in mind that the argument about WGU providing self-paced courses and rejecting the institution’s claims to be a term-based institution are much stronger than the argument behind the OIG’s arbitrary interpretation of regular and substantive interaction.

Impact on Big CBE Programs

The impact on WGU itself is most likely a matter of whether perception drives enrollment down. WGU clearly spent a lot of time and effort this summer preparing their web site to deal with the fall out. The central theme is to attempt to reassure current and prospective students that WGU “students, graduates, and employers of our graduates can rest assured that WGU’s accreditation and financial aid eligibility are intact”.

Students don’t follow all the details of political maneuvering and and even less of OIG audit reviews, but the perception that WGU’s status as a qualified distance education provider is at risk, therefore making the value of the degrees and likelihood of financial aid at risk, will add a barrier to enrollment decisions. I don’t suspect WGU can sweep this under the rug – they will have to go public as they have done with web site and hit this topic head on.

In my estimation, however, this will be a matter of a reduction in WGU’s enrollment growth, or possibly a drop, and not an existential problem for them. WGU has a 20-year history, more than 83,000 students, full institutional commitment to fight this audit in public, and a lot of clout in Washington with bipartisan support of their model – a rarity these days.

Southern New Hampshire University (SNHU) is also known for CBE, as their College for America got a lot of press over the past several years. What is not as well-known is that College for America, with its 8,000+ students, has been folded into the main operations of SNHU and its 100,000+ students, using the description of Workforce Partnerships. SNHU has less exposure than WGU to the audit for three reasons:

• Obviously the audit was specifically on WGU and not SNHU.
• CBE is all that WGU does, whereas SNHU has a fast-growing traditional online set of programs that provides the majority of its enrollment.
• SNHU’s College for America is a business-to-business model, working directly with employers rather than being a consumer program targeting individual students as WGU does. It will be a lot easier to control the message and reassure partners in the B2B model.

There could be some impact to SNHU and College for America, but again this is likely a manageable problem for these large schools.

Impact on Everyone Else

The bigger impact in my estimation will be on the other CBE programs in operation or in consideration, and this gets to the fragile movement comment.

Both WGU and SNHU have gone all-in on CBE. They invested heavily in developing the models, they have worked with employers to understand needs, they already have thousands and thousands of students and are growing enrollment, and they are fully committed as an institution to the concept and implementation of full-fledged CBE and not just CBE-lite. But both schools are outliers in the broader CBE movement.

A great deal of the perception of CBE is that there are “hundreds of schools” developing programs, often driven by a 2015 Public Agenda survey. What is happening in reality is that hundreds of schools are very cautiously dipping their toes in the CBE waters with no real commitment to make the model work, and they are only playing with courses and individual programs. Carl Straumsheim from Inside Higher Ed covered this situation in the spring, triggered by e-Literate news that Ellucian was dropping its CBE platform due to weak market demand:

Last year, Ellucian partnered with the consulting and research firm Eduventures and the American Council on Education to survey 251 colleges on their competency-based education strategies. The survey identified one major reason why the competency-based education market may be a tricky one for vendors to build a profitable business model in: most colleges aren’t ready to go all in yet.

The study found that only 7 percent of the colleges surveyed said they delivered most of their education using a competency-based model. Many more colleges said they were at the point of testing competency-based education in individual programs (18 percent) or courses (37 percent).

For most of the CBE programs that I have seen, we’re dealing with dozens, or perhaps hundreds of students. The University of Wisconsin’s UW Flex program – one of the best known outside of WGU and SNHU – has only grown to 5 degree programs, 3 certificate programs, 1300 students in total, and $2.8 million in annual gross revenue. And the vast majority of programs are much smaller than that. Read the Eduventures 2016 report (the same one referenced by Ellucian in the IHE article) for additional perspective.

Rather than a single, dominant version of CBE, our 2016 survey data reveals a diversity of practice across a spectrum of schools, each deploying and experimenting with CBE in order to meet specific institutional challenges. A portrait of CBE emerges as a menu of tools and practices, rather than a monolithic approach or linear path. These findings underscore the need for institutions to carefully weigh the pros and cons of CBE implementation, and to proactively select the CBE components that make the most sense for their students and mission.

The challenge is that CBE calls for a new organizational model and a new pedagogical design in order to make it work, at least in a self-sustaining manner. It is great to see hundreds of schools experimenting with new methods to reach non-traditional students, but none of these programs will last if they do not get sufficient scale to justify the costs. But very few schools are even looking at how to get to thousands of students and determining what investment and organizational setup will be required to get there. This isn’t as simple as flipping a classroom or two.

When you add in the WGU audit results, this challenge gets much harder to address. It will be much more difficult to justify investing in CBE programs or partnerships, or expanding beyond a pilot, when the audit provides marketing copy for those would resist CBE at traditional schools.

California

From the annals of bad timing comes this news from a week ago:

The California Community Colleges’ Board of Governors today approved a new partnership with Western Governors University (WGU) that allows graduates of California’s 114 community colleges to transfer and seek their bachelor’s degree at a discount from the fully accredited, online institution.

The agreement goes beyond pre-setting up transfers. It also is meant to encourage CCC students to move to WGU for their bachelor’s degree, where appropriate. There are terms for a 5% discount for CCC students, and there is a section on joint marketing.

The CCCCO supports and will encourage CCCs to collaborate with WGU so that partnership
information is available to students, faculty, and staff. WGU will also work in collaboration
with the CCCCO to assist in dissemination of information to students, faculty, and staff at
CCCs. CCCs will be encouraged to publicize locally the Chancellor’s Office support of this
agreement, to inform students, faculty, and staff of the agreement’s benefits. Information will
be encouraged through established internal CCC communication channels (such as
student/employee newsletters, web sites pages, and listservs)

This news comes at the same time as the active debate on the CCC system being directed to create a fully-online college serving non-traditional working adult students.

These are dramatic proposals for the largest higher education system in the US, and there is bound to be plenty of opposition to such plans from faculty unions and other groups skeptical about changes in model.

I suspect that the political climate in California just got a lot more difficult in terms of implementing both initiatives, and I suspect that the option lists available for setting up the new online college is shorter than it was last week.

Reminder

To be clear for those who have not read the first post, I consider the audit a travesty. The CBE movement and individual programs deserve scrutiny to ensure quality education for students, but this audit and its impact will do nothing useful to protect students or ensure quality.

((Full disclosure and Update: WGU is a past client of MindWires. Although we have no recent or pending relationship with WGU, since the topic was raised at Hack Education, I have added this disclosure. SNHU is a customer of our LMS market analysis service.))

The big news this week was the Office of the Inspector General (OIG) at the Department of Education (ED) finding that Western Governors University (WGU) should be considered a correspondence provider instead of a distance education provider, and the school should return more than $700 million in Title IV federal funding programs. ((Full disclosure and Update: WGU is a past client of MindWires. Although we have no recent or pending relationship with WGU, since the topic was raised at Hack Education, I have added this disclosure.)) In short, being ruled a correspondence provider would mean that most student loans and Pell grants would not apply for WGU students and would all but shut down the institution or make it irrelevant. While the ED itself is unlikely to follow these non-binding recommendations, this ruling will have a big impact for years. More on that in tomorrow’s post. For now, the title loans for college students are the best option right now.

It would be useful to first review the actual audit findings, especially since most media reporting focused mostly or solely on the issue of “regular and substantive interaction”, but the findings are broader and also encompass issues are self-paced vs. credit-hour / term-based education.

The audit started over four years ago and primarily focused on 102 courses (out of 980) offered in the 2013 – 14 academic year. No one seems to know why the OIG started this audit, but the audit report itself makes it clear that quality was not the issue (page 6):

we did not assess whether the school’s model was improving educational quality or expanding access to higher education.

WGU’s regional accreditor,  Northwest Commission on Colleges and Universities, has accredited WGU as a term-based distance education provider, including reaffirming the accreditation in February of this year. The Department of Education explicitly allowed WGU to be classified as a distance education provider as part of the Distance Education Demonstration Program from 1999 – 2005 and granted further waivers and agreements in April 2005.

By my reading, the audit is a model of hyper-literal translation of ambiguous regulations, leading to three findings.

• Finding 1) Course Offerings Met the Title IV Definition of a Correspondence Course, Not the Title IV Definition of Distance Education – This finding was centered on reviews of course materials for 69 courses as well as a mapping of WGU’s unbundled faculty role to traditional instructor definitions. The OIG found that using the courses did not meet the interpretation of regular and substantive interaction required of distance education courses.
• Finding 2) Western Governors University Disbursed Title IV Funds to Students Before the Students Were Eligible to Receive the Funds – This finding was based on additional mapping of WGU’s CBE model to traditional term-based model. WGU itself decided to consider itself a term-based institution, mapping one competency unit to one credit hour over a 26-week academic term, and the ED recognized this classification in the 2005 agreement. The OIG, however, found that WGU should have been classified as a non-term school using self-paced programs. There are a different set of regulations for non-term programs.
• Finding 3) Western Governors University Did Not Always Comply With the Requirements Governing the Return of Title IV Funds – This finding is important, but it deals with detailed bureaucratic rules upon student withdrawals. I’ll let someone else look at this finding.

It is the combination of findings 1) and 2) that are important not just to WGU but to any school developing an online or hybrid non-lecture-based approach.

Regular and Substantive Interaction

WGU has been at the forefront of breaking apart the traditional faculty role, instead using mentors, evaluators, and other interdependent roles. The audit acknowledged that (page 15):

Northwest Commission recognized Western Governors University’s student mentors, course mentors, evaluators, product managers, and council members as members of the school’s faculty. The accrediting agency also distinguished between the roles of student mentors and course mentors, characterizing student mentors as serving in academic advisory roles and course mentors serving in instructional roles.

The OIG used a binary role-based approach (you are an instructor or you are not) leading to conclusion that only course mentors and evaluators could be considered as instructors, however. The basis of this determination was an instructor must “provide instruction on course content” – clearly a content-dissemination view that rejects alternative pedagogies. And this interpretation that the OIG treats as unambiguous is not based on law, regulations, or commonly-accepted educational terminology.

The OIG looked at the ambiguous regulations and chose their own, very literal, interpretations (page 14):

Because the HEA and Title IV regulations did not define instructor, substantive, or regular, we considered the ordinary meaning of those terms when assessing whether the school designed the 102 courses to offer regular and substantive interaction between students and instructors. We reviewed the school’s course design materials for evidence of interaction that was not primarily initiated by the student and was (1) with someone who instructs or provides knowledge about the subject matter of the course (instructor), (2) relevant to the subject matter (substantive), and (3) occurring with some reasonable frequency considering the school-suggested length of the course (regular).

This is why I call the audit methodology as hyper-literal. Somehow the OIG thinks they can determine – without any disagreement or ambiguity – the “ordinary meaning of those terms” based on their own interpretations.

Also note that the determination was entirely based on course design materials – think syllabus and course outlines. The OIG did not look at interactions arising during the course of actual course work, just whether there were pre-defined webinars, meetings, and student-instructor interactions. The OIG did eliminate many interactions as not being “substantive” (page 16):

After identifying the employees who could reasonably be considered instructors, we determined what type of interactions could reasonably be considered substantive. We considered an interaction to be substantive if the course design materials described student interaction with a course mentor or required an individual submission of a performance task for which an evaluator provided the student feedback. We did not consider the following to be instances of substantive interactions between students and instructors:

• Objective assessments that students submitted for evaluation because feedback on objective assessments was computer-generated, was not provided by instructors, and did not facilitate synchronous or asynchronous interaction between students and instructors.
• Recorded webinars, videos, and reading materials if the course design materials did not require the students to watch the webinars or videos and then interact with an instructor. Many course outlines stated only that course mentors were available to students for assistance if the student wanted to contact the course mentor. Had the course design materials indicated that the recorded webinars, videos, and reading materials facilitated synchronous or asynchronous interactions, such as requiring the student to contact an instructor or participate in an online discussion moderated by an instructor, we would have considered those instances to be substantive interaction.
• Contact with student mentors because the accrediting agency’s recognition, the school’s description of the student mentor’s role, and our interviews with six student mentors disclosed that student mentors did not provide instruction on the subject matter of the courses that students were taking.

To be “regular”, the OIG required that all interactions be pre-planned, with the right people, in the course design materials (page 16).

We did not find any evidence in the course design materials for 69 courses that would provide a reasonable basis for concluding that planned student interactions with course mentors and evaluators could be considered as occurring with some reasonable frequency (regular). The only evidence of regular interaction was student contact with student mentors. However, student mentors did not provide instruction.

In its comment on the draft findings, WGU complained that OIG (page 20 and 26):

did not count, as regular and substantive interaction, significant interactions not described in course outlines.

OIG’s response basically agreed with this complaint:

We considered all events described in course outlines and pacing guides, along with calendars of live events referenced in those materials. If substantive interactions were not described in any of these course design materials, we had no reasonable basis to conclude that such interactions were part of the design of the courses and did not consider them as planned course requirements. [snip]

Course mentors might have identified students who were struggling, and many course outlines instructed students to contact course mentors if the students needed assistance. However, if course design materials did not describe the interaction, there was no reasonable assurance that students had any regular and substantive interaction with course mentors.

These views essentially reject not just WGU’s approach to CBE but also the broader movement of faculty from “sage on the stage to guide on the side”. Instructors, from the OIG view, must provide instruction on course content and interactions must be pre-planned in the course design materials, at least for online courses.

Self-Paced

While I assume there were good reasons for WGU to want to avoid being classified as a non-term school, their 2005 declaration that they were not self-paced but rather term-based now appears to be a self-inflicted wound (page 34).

Western Governors University Comments
Western Governors University stated that its courses were not self-paced. Students were given a pacing guide and were expected to complete a certain number of competency units each term. In addition, a key role of the student mentor was to guide the pace of academic progress with individual students to ensure course completion by a certain date. Students had some flexibility in the pacing and moved through the content at different rates to allow for their individual competency development. However, that does not mean that the courses were self-paced.

OIG Response
Western Governors University’s statement that courses were not self-paced is contrary to its advertising materials, pacing guide descriptions, and statements from school officials we interviewed. According to the school’s web site, students could complete a degree program as soon as they successfully completed all of the necessary assessments. Students who completed assessments quicker could complete their degree quicker. The school’s web site also stated that students could complete assessments as soon as they were ready; if they were already competent in a subject area, they could prove it faster and complete their degree faster. If students completed their programs in less time, they paid only the tuition for the number of student terms in which they needed to enroll to complete the program. Course outlines stated the following about the pacing guides: “The pacing guide suggests a weekly structure to pace your completion of learning activities. It is provided as a suggestion and does not represent a mandatory schedule.” Western Governors University’s Program Development Operations Manager, Director of Assessment Design and Development, and Associate Provost for Academic Services confirmed to us during the audit that courses were self-paced.

Ouch. It is far different for OIG to arbitrarily pick their own interpretations of regular and substantive interactions than it is for OIG to use WGU’s own descriptions and interviews. This seems to be a strong argument by OIG.

WGU Response and Web Site

WGU’s response was dated May 22 of this year, so they have had time to prepare for the audit findings. They have a web site that explains the situation, answers basic questions, and highlights their arguments against the findings.

WGU strongly disagrees with the Inspector General’s audit report, which challenges our innovative, results-proven faculty model.

With this key arguments:

WGU has complied with the higher education laws and Department of Education guidance since our founding 20 years ago. Students, graduates, and employers of our graduates can rest assured that WGU’s accreditation and financial aid eligibility are intact.

Our accreditor, the Northwest Commission on Colleges and Universities, has approved our faculty model and reaffirmed our accreditation in February 2017. Accreditors are responsible for determining whether a university is eligible for federal financial aid.

Fixes Needed

I agree with both Russ Poulin and Amy Laitinen regarding the need to fix but not remove the “regular and substantive interaction” regulations. From Inside Higher Ed’s coverage:

Russell Poulin, director of policy and analysis at the WICHE Cooperative for Educational Technologies, said the department has done a “horrible” job of informing colleges about its expectations of how to comply with the regular-and-substantive requirements, which he said have changed over time.

In addition, he criticized the inspector general’s decision to base its compliance position on disagreement about the mode of teaching at WGU when there is no evidence of any harm to students.

“I totally agree with the intention of proponents of the ‘regular-and-substantive interaction’ rule, which is to avoid fraud. But it is an outdated method of reaching that goal,” he said via email, comparing it to a hypothetical decision by regulators to remove all ATM card readers because of the risk of credit card skimmers. [snip]

Amy Laitinen, director of higher education policy for the group and a former Obama administration Education Department official, said the law was a response to rampant fraud and abuse.

“We need to carefully fix (not gut) the now-outdated law to ensure that students are getting the academic and other supports that they need,” she said via email. “If we don’t do it carefully, it will be a fast race to the bottom, which would be bad for students and bad for the competency-based education community.”

The Worst Part

To me the worst part of the audit is the language used by the OIG that tries to make any disagreement with a hyper-literal translation of ambiguous regulations seem to be invalid (page 3).

None of these 69 courses could reasonably be considered as providing regular and substantive interaction between students and instructors.

Who holds these “unreasonable” views? The Department of Education and the Northwest Commission are two groups, among others. It’s bad enough that the OIG took the out-of-context, hyper-literal approach to the audit, but to not acknowledge the ambiguity and lack of clear guidance about this requirement is disingenuous.

This audit is a travesty in my opinion. Even though it is likely to be rejected by the ED itself, it will have an impact, and the internal review of the audit will likely take years. I’ll write more about potential impact of the audit tomorrow.

Additional Reading

• OIG Audit
• WGU Response Site
• IHE Coverage (best news article)
• Chronicle Coverage
• EdSurge Coverage
• WCET Analysis (best summary of issues)