July 28, 2014

WCET’s analysis of U.S. statistics on distance education

Listen with webReader

IPEDS 2

U.S.Department of Education (2014) Web Tables: Enrollment in Distance Education Courses, by State: Fall 2012 Washington DC: U.S.Department of Education National Center for Education Statistics

Hill, P. and Poulin, R. (2014) A response to new NCES report on distance education e-Literate, June 11

The U.S. Department of Education’s Institute of Education Sciences operates a National Center for Education Statistics which in turn runs the Integrated Postsecondary Education Data System (IPEDS). IPEDS is:

a system of interrelated surveys conducted annually by the U.S. Department’s National Center for Education Statistics (NCES). IPEDS gathers information from every college, university, and technical and vocational institution that participates in the federal student financial aid programs. The Higher Education Act of 1965, as amended, requires that institutions that participate in federal student aid programs report data on enrollments, program completions, graduation rates, faculty and staff, finances, institutional prices, and student financial aid. These data are made available to students and parents through the College Navigator college search Web site and to researchers and others through the IPEDS Data Center

Recently IPEDS released “Web Tables” containing results from their Fall Enrollment 2012 survey. This was the first survey in over a decade to include institutional enrollment counts for distance education students. In the article above, Phil Hill of e-Literate and Russell Poulin of WCET have co-written a short analysis of the Web Tables released by IPEDS.

The Hill and Poulin analysis

The main points they make are as follows:

  • overall the publication of the web tables in the form of a pdf is most welcome, in particular by providing a breakdown of IPEDS data by different variables such as state jurisdiction, control of institution, sector and student level
  • according to the IPEDS report there were just over 5.4 million students enrolled in distance education courses in the fall semester 2012 (NOTE: this number refers to students, NOT course enrollments).
  • roughly a quarter of all post-secondary students in the USA are enrolled in a distance education course.
  • the bulk of students in the USA taking distance education courses are in publicly funded institutions (85% of those taking at least some DE courses), although about one third of those taking all their classes at a distance are in private, for-profit institutions (e.g. University of Phoenix)
  • these figures do NOT include MOOC enrollments
  • as previously identified by Phil Hill in e-Literate, there is major discrepancy in the number of students taking at least one online course between the IPEDS study and the regular annual surveys conducted by Allen and Seaman at Babson College – 7.1 million for Babson and 5.5 million for IPEDS. Jeff Seaman, one of the two Babson authors, is also quoted in e-Literate on his interpretation of the differences. Hill and Poulin comment that the NCES report would have done well to at least refer to the significant differences.
  • Hill and Poulin claim that there has been confusion over which students get counted in IPEDS reporting and which do not. They suspect that there is undercounting in the hundreds of thousands, independent of distance education status.

Comment

There are lies, damned lies and statistics. Nevertheless, although the IPEDS data may not be perfect, it does a pretty good job of collecting data on distance education students across the whole of the USA. However, it does not distinguish between mode of delivery of distance education (are there still mainly print-based courses around)?

So we now have two totally independent analyses of distance education students in the USA, with a minimum number of 5.5 million and a maximum number of 7.1 million, i.e. between roughly a quarter and a third of all post-secondary students. From the Allen and Seaman longitudinal studies, we can also reasonably safely assume that online enrollments have been increasing between 10-20% per annum over the last 10 years, compared with overall enrollments of 2-5% per annum.

By contrast, in Canada we have no national data on either online or distance education students. It’s hard to see how Canadian governments or institutions can take evidence-based policy decisions about online or distance education without such basic information.

Lastly, thank you, Phil and Russ, for a very helpful analysis of the IPEDs report.

Update

For a more detailed analysis, see also:

Haynie, D. (2014) New Government Data Sheds Light on Online Learners US News, June 13

 

The success or otherwise of online students in the California Community College system

Listen with webReader

 Online offerings vary widely across subject

Johnson, H. and Mejia, M. (2014) Online learning and student outcomes in California’s community colleges San Francisco CA: Public Policy Institute of California, 20 pp

I’m not a great fan of studies into completion rates in online learning, because most studies fail to take into account a whole range of factors outside of the mode of delivery that influence student outcomes. However, this study is an exception. Conducted by researchers at the highly influential PPIC, it takes a very careful look at how well students across the whole California community college system (CCCS) do in online learning, and there are some very interesting findings that may not come as a surprise to experienced observers of online learning, but will certainly provide fodder for both supporters and skeptics of online learning.

Why the study is important

Several reasons:

  • California’s community colleges offer more online credit courses than any other public higher education institution in the country. By 2012, online course enrollment in the state’s community colleges totaled almost one million, representing about 11 percent of total enrollment
  • Over the past ten years, online course enrollment has increased by almost 850,000, while traditional course enrollment has declined by almost 285,000.
  • Community colleges are more likely than other institutions of higher education [in the USA] to serve nontraditional students. These students often have employment and family obligations and therefore may potentially benefit the most from online learning.
  • The state of California is investing $57 million over the next five to six years for online learning initiatives within the California Community College system
  • The California Community Colleges Chancellor’s Office (CCCCO) provided … access to unique longitudinal student- and course-level data from all of the state’s 112 community colleges

Main findings

  • Between 2008–09 and 2011–12, total credit enrollment at California’s community colleges declined by almost a million. The scarcity of traditional courses has been a factor in the huge increase in online enrollments. With the state cutting support to community colleges by more than $1.5 billion between 2007–08 and 2011–12, community colleges experienced an unprecedented falloff in enrollment 
  • online course success rates are between 11 and 14 percentage points lower than traditional course success rates.
  • in the long term, students who take online classes tend to be more successful than those who enroll only in traditional courses…students who take at least some online courses are more likely than those who take only traditional courses to earn an associate’s degree or to transfer to a four-year institution.
  • for students juggling school, family and work obligations, the ability to maintain a full-time load by mixing in one or two online courses per term may outweigh the lower chances of succeeding in each particular online course.
  • if a student’s choice is between taking an online course or waiting for the course to be offered in a classroom at a convenient time, taking the online course can help expedite completion or transfer
  • participation in online courses has increased for each of the state’s largest ethnic groups—and online enrollment rates for African American students, an underrepresented group in higher education in California, are particularly high. However, these rates are much lower among Latino students.

Main recommendations

  • move from ad hoc offerings to more strategic planning of online courses
  • improve the ability to transfer credits between community colleges and between colleges and the state’s universities
  • improve the design and provide more consistency in the quality of online courses between institutions
  • adopt a standardized learning management system across all colleges
  • collect systematic information on the cost of developing and maintaining online courses

My comments

This is another excellent and succinct research report on online learning, with a very strong methodology and important results, even if I am not at all surprised by the outcomes. I would expect online completion rates for individual courses to be lower than for traditional courses as students taking online courses often have a wider range of other commitments to manage than full-time, on campus students.

Similarly, I’m not surprised that online course success is lightly lower for community colleges than for universities (if we take both the figures from Ontario and my own experience as a DE director) and for certain ethnic groups who suffer from a range of socio-economic disadvantages. Online learning is more demanding and requires more experience in studying. Post-graduate students tend to do better at online learning than undergraduate students, and final year undergraduate students tend to do better than first year undergraduate students. Nevertheless, as the study clearly indicates, over the long term online learning provides not only increased access but also a greater chance of success for certain kinds of students.

I am worried though that online learning in California has ‘succeeded’ because of the massive cuts to campus-based education. It is better than nothing, but online learning deserves to be considered in its own right, not as a cheaper alternative to campus-based education. Online learning is not a panacea. Different students have different needs, and a successful public post secondary education system should cater to all needs. In the meantime, this is one of the most useful studies on online completion rates.

 

 

 

A balanced research report on the hopes and realities of MOOCs

Listen with webReader

Columbia MOOCs 2

Hollands, F. and Tirthali, D. (2014) MOOCs: Expectations and Reality New York: Columbia University Teachers’ College, Center for Benefit-Cost Studies of Education, 211 pp

We are now beginning to see a number of new research publications on MOOCs. The journal Distance Education will be publishing a series of research articles on MOOCs in June, but now Hollands and Tirthali have produced a comprehensive research analysis of MOOCs.

What the study is about

We have been watching for evidence that MOOCs are cost-effective in producing desirable educational outcomes compared to face-to-face experiences or other online interventions. While the MOOC phenomenon is not mature enough to afford conclusions on the question of long-term cost-effectiveness, this study serves as an exploration of the goals of institutions creating or adopting MOOCs and how these institutions define effectiveness of their MOOC initiatives. We assess the current evidence regarding whether and how these goals are being achieved and at what cost, and we review expectations regarding the role of MOOCs in education over the next five years. 

The authors used interviews with over 80 individuals covering 62 institutions ‘active in the MOOCspace’, cost analysis, and analysis of other research on MOOCs to support their findings. They identified six goals from the 29 institutions in the study that offered MOOCs, with following analysis of success or otherwise in accomplishing such goals:

1. Extending reach (65% 0f the 29 institutions)

Data from MOOC platforms indicate that MOOCs are providing educational opportunities to millions of individuals across the world. However, most MOOC participants are already well-educated and employed, and only a small fraction of them fully engages with the courses. Overall, the evidence suggests that MOOCs are currently falling far short of “democratizing” education and may, for now, be doing more to increase gaps in access to education than to diminish them. 

2. Building and maintaining brand (41%)

While many institutions have received significant media attention as a result of their MOOC activities, isolating and measuring impact of any new initiative on brand is a difficult exercise. Most institutions are only just beginning to think about how to capture and quantify branding-related benefits.

3. Reducing costs or increasing revenues (38%)

….revenue streams for MOOCs are slowly materializing but we do not expect the costs of MOOC production to fall significantly given the highly labor-intensive nature of the process. While these costs may be amortized across multiple uses and multiple years, they will still be additive costs to the institutions creating MOOCs. Free, non-credit bearing MOOCs are likely to remain available only from the wealthiest institutions that can subsidize the costs from other sources of funds. For most institutions, ongoing participation in the current MOOC experimentation will be unaffordable unless they can offer credentials of economic value to attract fee-paying participants, or can use MOOCs to replace traditional offerings more efficiently, most likely by reducing expensive personnel. 

4. Improving educational outcomes (38%)

for the most part, actual impact on educational outcomes has not been documented in any rigorous fashion. Consequently, in most cases, it is unclear whether the goal of improving educational outcomes has been achieved . However, there were two exceptions, providing evidence of improvement in student performance as a result of adopting MOOC strategies in on-campus courses

5. Innovation in teaching and learning (38%)

It is abundantly clear that MOOCs have prompted many institutions and faculty members to engage in new educational activities. The strategies employed online such as frequent assessments and short lectures interspersed with questions are being taken back on-campus. It is less clear what has been gained by these new initiatives because the value of innovation is hard to measure unless it can be tied to a further, more tangible objective. We …. conclude that most institutions are not yet making any rigorous attempt to assess whether MOOCs are more or less effective than other strategies to achieve these goals. 

6. Research on teaching and learning (28%)

A great deal of effort is being expended on trying to improve participant engagement and completion of MOOCs and less effort on determining whether participants actually gain skills or knowledge from the courses ….While the potential for MOOCs to contribute significantly to the development of personalized and adaptive learning is high, the reality is far from being achieved. 

Cost analysis

The report investigates the costs of developing MOOCs compared to those for credit-based online courses, but found wide variations and lack of reliable data.

Conclusions from the report

The authors came to the following conclusions:

1. there is no doubt that online and hybrid learning is here to stay and that MOOCs have catalyzed a shift in stance by some of the most strongly branded institutions in the United States and abroad.

2. MOOCs could potentially affect higher education in more revolutionary ways by:

  • offering participants credentials of economic value

  • catalyzing the development of true adaptive learning experiences

However, either of these developments face substantial barriers and will require major changes in the status quo.

My comments on the report

First this is an excellent, comprehensive and thoughtful analysis of the expectations and realities of MOOCs. It is balanced, but where necessary critical of the unjustified claims often made about MOOCs. This report should be required reading for anyone contemplating offering MOOCs.

Different people will take away different conclusions from this report, as one would expect from a balanced study. From my perspective, though, it has done little to change my views about MOOCs. MOOC providers to date have made little effort to identify the actual learning that takes place. It seems to be enough for many MOOC proponents to just offer a course, on the assumption that if people participate they will learn.

Nevertheless, MOOCs are evolving. Some of the best practices that have been used in credit-based online courses are now being gradually adopted as more MOOC players enter the market with experience of credit-based online learning. MOOCs will eventually occupy a small but important niche as an alternative form of non-formal, continuing and open education. They have proved valuable in making online learning more acceptable within traditional institutions that have resisted online learning previously. But no-one should fear them as a threat to credit-based education, either campus-based or online.

More developments in online labs

Listen with webReader
Embry-Riddle's crash simulation

Embry-Riddle’s crash simulation

Zalaznik, M. (2014) Online labs evolve University Business, May

What’s happening

This article reports on the following developments in online labs:

  • Embry-Riddle Aeronautical University has created an online ‘crash lab’ for students taking a Master of Aeronautical Science program. Through the online lab, students can examine ‘all the procedures leading up and through [an] accident.’ The lab, developed in partnership with Unity, uses high quality graphics and elements of video-gaming, to develop a range of skills. The next project will be to develop a virtual robotics lab where students will design and fly unmanned aircraft
  • Arizona State University is offering a fully online Bachelor of Science in Electrical Engineering for approximately 200 students. Students will use low cost hardware kits that enable them to build and test circuits, in conjunction with software on their computers that enable measurements to be made on a simulated oscilloscope. Online students take the same exams as the students on campus. The online program is taken mainly by older, more experienced students.
  • Henderson Community College, Kentucky, combines home kits and real materials such as  sheep hearts and bacteria with a learning management system to do experiments remotely. The kits are produced by eScience Labs and Pearson provides a biology lab  that contains a virtual microscope
  • The NANSLO project, which was reported in detail here: Can you teach lab science via remote labs?

Why is the use of online labs increasing?

The article suggests the following reasons for the growth of online labs:

  • a shortage of real lab equipment and facilities, especially for general science students
  • online labs require fewer instructors and can make greater use f teaching assistants
  • its most suitable for non-science majors who don’t need to know how to run a lab or how to operate all the equipment in a lab
  • online labs give more leeway to design their own experiments
  • students can make and learn from mistakes in virtual labs
  • online labs are being used in conjunction with on campus facilities, enabling more productive use of limited physical resources.

Resources

The article provides a very useful set of links to these projects.

 

What I learned from the Open Textbook Summit

Listen with webReader
Paul Stacey MC-ing the Open Textbook Summit

Paul Stacey MC-ing the Open Textbook Summit

BCcampus (2014) Five lessons learned at the Open Textbooks Summit Vancouver BC: BCcampus

BCcampus organized an open textbook summit again this year (the first one was last year). I attended, because I’m writing my own open textbook on ‘Teaching in a Digital Age.’ BCcampus has published its own blog post on the lessons learned, but I came away with something different, from a potential author’s perspective.

1. Open textbooks are gaining momentum.

There were two Ministers of Advanced Education present, one from BC and one from Saskatchewan. This is because the three western Canadian provinces (BC, Alberta, and Saskatchewan) have signed a New West Partnership which includes collaboration on and sharing of open textbooks (Saskatchewan’s participation interestingly was initially driven by pressure from students.)

Last year there were 30 participants at the Open Textbook Summit, this year 130, including David Wiley, representatives from Open Stax, librarians, Barbara Illowsky, an author of an open textbook on comparative statistics, and senior university administrators and faculty in BC who were incorporating open textbooks in their teaching.

Currently, BC has 19 open textbooks available for large enrollment courses, with another 28 being ready in September this year, and another 20 by September 2015. So the supply side is really ramping up in western Canada and government is getting behind it in a big way.

2. There is a clear need for open textbooks.

Kim Thanos from LumenLearning pointed out that textbook costs have increased by 6.8% compared with a cost of living increase of 3.8%. 60% of students at some point during their program do not buy a recommended textbook because of cost, and 31% of students avoid certain courses because of the high cost of textbooks. Open Stax with just 11 open textbooks in 18 months has reached 600 schools/institutions, almost 100,000 students and saved students  $9.3 million in textbook costs.

3. The supply and the demand from students is coming – but where is the adoption by faculty?

Adoption by faculty and instructors remains a major challenge. Diane Salter from Kwantlen Polytechnic University stated that there needs to be an institutional strategy for open textbooks and open educational resources, to raise awareness and get buy-in from faculty. Takashi Soto, an instructor also from Kwantlen, pointed out that with the ability to edit, remix and delete, he can move an open textbook that initially gives him 85% of what he wants to 95%.

But still many faculty are suspicious of the quality of open textbooks or are just not aware that there are suitable open textbooks available for their courses. Open textbooks do not have the marketing clout of commercial textbook publishers. But I also have to say that there is still a certain evangelicism around open textbooks and OERs which I think puts off many faculty. Faculty need to take some ownership of the process of selection, adaptation and implementation if open textbooks are to be adopted on a larger scale. (See Dr. Rajiv Jhangiani’s excellent post on some of the reasons for the slow adoption by faculty.)

4. Open textbooks have their own pedagogy.

Most open textbooks today remind me of the movies at the turn of the century. Movies then mainly looked like recorded music hall acts. Cinema needed a D.W. Griffith to recognize the potential of the medium. Most open textbooks look just like commercial printed textbooks; static, lots of print, some graphics, but no animation, video, audio, learner activities or feedback built in.

David Wiley, as always, was very interesting on this topic. He pointed out that opening up student activities beyond the classroom or campus and sharing and collaborating with students on the development and production of content enables quality improvements and more transparency in the teaching (which may explain some of the resistance by many faculty).

I am still struggling, as I write my own open textbook, with the issue of when an open textbook moves from being a ‘book’ to a ‘course’, as one builds in more opportunities for ‘expert’ and ‘student’ contributions to the content, and more links and activities around the content.

5. The technology is still crude

Because the current technology ‘model’ for open textbooks is still based on printed books, the functions that enable more open collaboration, remix and re-use are still very crude. PressBook is a useful adaptation of WordPress, but it lacks many features that I feel I need as an author.  BCcampus has developed a plug-in called PressBook Textbook that has or will have features such as enabling better quality tables and math equations to be easily incorporated, but I’m still trying to work out how to download/add it to my version of PressBook (this is probably due more my technological naivity). Trying to manipulate graphics or images is also very clunky. So all the features that an author needs to create an open textbook that goes beyond a simple text still need more work.

More fundamentally, I’m still struggling with how someone else can take what I’ve written and incorporate it in their own work in an easy and transparent manner, without destroying the integrity of the original. How do I track the changes and variations that others have made? How can I keep the book dynamic – even after I’m dead? How many versions of the book should there be, and how will readers be able to judge which is ‘authentic’ or reliable?

These are interesting questions that I will continue to explore as I develop my open textbook. In the meantime, the Open Textbook Summit was very helpful as I start out on this journey.