Mind the Gaps: WCET survey adds valuable context

This week the WICHE Cooperative for Educational Technologies (WCET) released its Managing Online Education survey results that were previewed at the WCET13 conference in November. Despite all of the talk about the potential data-driven decision-making in higher education, it is remarkable how little we know.

Course Completion

Based on media coverage, the WCET survey results that are grabbing people’s attention:

• For institutions reporting both completion rates, the on-campus completion rate was better than the online rate by an average of less than 5 percent
• Institutions had trouble providing completion rates with 65 percent not being able to provide an on-campus rate and 55 percent not reporting an online rate

These factors were covered quite well at Inside Higher Ed:

Some respondents blamed the lack of data on course catalogs that don’t specify if a particular section of a course is online or not. Distance education providers have for years fought to eliminate the stigma of online courses’ implied lack of quality, and the shift toward an equal billing makes it difficult to distinguish between different forms of course delivery.

“If institutions wish to improve retention, they will need to collect these statistics,” the report reads. “It’s hard to improve what is not measured.”

Russell Poulin, deputy director of research and analysis for the WICHE Cooperative for Educational Technologies, offered another theory: What if institutions are intentionally withholding low completion rates?

Additional news reports can be found at the Chronicle and Campus Technology.

As the WCET survey points out, “There is considerable mythology around completion rates for online courses.” Where do we get this information? To the best of my knowledge, the most well-known studies were targeted specifically at community and technical colleges. The WCET survey may actually be the first cross-institution study looking at online vs. on-campus completion rates across 2-year, 4-year, masters and research institutions.

The Community College Research Center (CCRC), which is part of Teachers College at Columbia University, performed two studies in the mid 2000s.

• They first looked at the 2004 cohort across the Virginia Community College System, finding a “13 percentage point difference in completion between face-to-face and online courses”, as reported in 2010.
• The same research team (Shanna Smith Jaggars and Di Xu) then looked at the 2004 and 2008 cohorts at Washington State Community and Technical Colleges, finding “online course completion rates were 8 percentage points lower than face-to-face completion rates” for the 2004 cohort and 6 percentage points lower in the 2008 cohort, as reported in 2011.

The Instructional Technology Council has captured data since 2004 in their Distance Education Survey results, also targeted at community and technical colleges [emphasis added]

During the early years of distance education, retention and completion rates could easily fall below 50 percent. Studies consistently report that colleges have positively addressed this challenge, despite continued misconceptions. The ITC Survey participants reported that the gap between online and face-to-face student retention now averages only eight percent. In nine years of data, the trend in online retention continues to improve, but challenges remain, and addressing the gap is a major priority for many programs.

While many individual schools and even statewide systems have internally published their online and on-campus course completion rates, the WCET is a valuable addition for several reasons.

• The study looks across a broad array of institution types; of the 225 responding schools, 43% have associates as the highest degree, 9% bachelors, 18% masters and 31% doctorate;
• The study adds the important context that more than half of schools either cannot or will not share their course completion data; and
• The study is based on the 2011 – 12 academic year, helping to provide more up-to-date information.

There is a huge gap between the discussions of the power of data to improve education and the on-the-ground reality of missing or inconsistent data, even for the most basic of measurements such as course completion. This is true both at the institutional level, as evidenced by the high percentage of schools that could not provide the completion data for both online and on-campus courses, and at the cross-institutional level, as evidenced by the very small number of studies available.

Practice What We Preach, Not What We Do

There is a lot more to the WCET survey (and the CCRC and ITC studies, as well) than just course completion data, and it is well worth reading the whole report. One section that got my attention was on academic and student support services.

• Only about one quarter (22 percent) of respondents require their online students to take an orientation prior to their first online course, even though research suggests that experience aids in online course success.
• The vast majority of institutions offer library services and advising to online students. Fewer, but still a majority, offer tutoring services.
• More than three quarters of institutions have a policy on “academic integrity” (preventing cheating on assessments) for online learners. Only about 40 percent use technologies to authenticate the identity of online learners.
• Only about one-third (30 percent) of institutions offer 24/7 technical support for students. Given that students work all hours on online courses, the lack of support could hamper their success in the course.
• In meeting the needs of those with disabilities, it is alarming that sixteen percent have no policy on this subject and another thirty-six percent rely on the faculty to provide support. Therefore, at least half of the responding institutions have no systematic way to assure that students with disabilities are well-served.

It is fairly well known that students perform better in online courses once they have experience with the medium. The importance of student support services increases for low-income and underprepared students. Based on the survey, too many institutions are not providing the needed services.

The ITC survey documented what e-learning professionals identified as the “greatest challenges for students enrolled in distance education classes”, and the list reads like the flip side of the WCET findings:

1. (biggest challenge) Orientation/preparation for taking distance education classes
2. Providing equivalent student services virtually
3. Assessing student learning and performance in distance education classes
4. Computer problems/technical support
5. Low student completion rate
6. Completion of student evaluations
7. Cheating

Here is another huge gap, this time between what we know is needed and what is provided. It makes you wonder what the online course completion rate would be if institutions could provide the needed academics and student support services.