Open Data: A case study using IPEDS for online education

At e-Literate we’re planning to do more data journalism posts. Think of this along the lines of Harper Reed’s keynote last year as described in the Chronicle.

Harper Reed, who served as chief technology officer in President Obama’s 2012 campaign, offered those people what he jokingly called “an intervention.”

“Big Data is bullshit,” Mr. Reed said in a keynote speech on Tuesday.

His point is not that there is no value in the large amounts of data available to the community, but that data analysis is getting democratized and doesn’t require huge investments.

“The exciting thing is you can get a lot of this stuff done just in Excel,” he said. “You don’t need these big platforms. You don’t need all this big fancy stuff. If anyone says ‘big’ in front of it, you should look at them very skeptically … You can tell charlatans when they say ‘big’ in front of everything.”

I would add one other aspect along with the democratization of data analysis – open data. Inspired by Audrey Watters and even her recent tweet, we are looking for ways to not only explore the open data available to the education community but also to share the data we use in our analyses.

https://twitter.com/audreywatters/status/426541112527192064

I’m currently working on a blog post on the new University of Florida Online (UF Online) program that starts this spring. In preparation for this post I wanted to explore the current state of exclusively online education (where students take all their courses online) for public 4-year institutions based on the new data from IPEDS. This data gives an opportunity to look at one example of exploring and sharing open data.

Basic Institutional Data for Public 4-year Institutions from IPEDS

The first task is to narrow down the IPEDS distance education data (which is roughly equivalent to online courses, which just a few exceptions) to just the public 4-year institutions in the US. What I did was the following:

• Go to the IPEDS Data Center and selected the ‘Compare Individual Institutions’ option.
• Once there I used ‘Provisional Release Data’ and filtered the institutions By Groups.
  • For the groups, I selected ‘First Look Universe’ and ‘US Only’; and
  • Under ‘Sector’ I selected ‘Public, 4-year or above’.
• This led to 690 institutions.
• For the variables to download, I selected ‘Fall Enrollment’ > ‘Distance education’ and then selected ‘Fall 2012’ and ‘Degree/certificate-seeking total’.
• Under this option I selected the following variables:

I then downloaded a CSV file and cleaned up the data by replacing blanks with zeros.

The next challenge is to display on WordPress allowing user access to the data. For this I chose two methods:

• Upload to Google GDrive for direct sharing or download of the spreadsheet data; and
• Use TablePress plugin to display tables in an interactive format.

The TablePress plugin required the DataTables Sorting plugin Extension to allow it to handle a , thousands separator. Here is the result of the institutional data analysis:

[table id=1 /]

Here is the data table on GDrive for sharing / downloading.

Summary by State

The next task was to summarize the data by each state. I created the summaries mostly by the SUMIF() function in MS Excel.

=SUMIF($B$2:$B$691,”=”&$K2,$F$2:$F$691)

where column B is the state code of each institution, column K is the state code in the summary table (one row per state), and column F is the data for each time of enrollment.

Once this was done, I saved a csv file with the data only (no formulas) as well as the percentages, uploaded to GDrive and published using TablePress:

[table id=2 /]

[table id=4 /]

Here is the data table on GDrive for sharing / downloading.

Listing of Florida Public Institutions

At this point it seemed useful to filter down to the specific Florida public 4-year institutions, and share both the raw enrollment data as well as the percentages:

[table id=3 /]

[table id=5 /]

Here is the data table on GDrive for sharing / downloading.

Feedback

As I mentioned, we plan on doing more open data exploration on e-Literate, so let me know if you have any problems accessing, viewing or downloading the data, or if you have any suggestions.

Update: I highlighted the Florida and University of Florida rows based on the story on UF Online.