From the abstract:

Proponents of postsecondary online education were recently buoyed by a meta-analysis sponsored by the U.S. Department of [...]]]> Smith Jaggars, S. and Bailey, T. (2010) Effectiveness of Fully Online Courses for College Students: Response to a Department of Education Meta-Analysis New York: Community College Research Center, Teachers College, Columbia University.

From the abstract:

Proponents of postsecondary online education were recently buoyed by a meta-analysis sponsored by the U.S. Department of Education suggesting that, in many cases, student learning outcomes in online courses are superior to those in traditional face-to-face courses. This finding does not hold, however, for the studies included in the meta-analysis that pertain to fully online, semester-length college courses; among these studies, there is no trend in favor of the online course mode. What is more, these studies consider courses that were taken by relatively well-prepared university students, so their results may not generalize to traditionally underserved populations. Therefore, while advocates argue that online learning is a promising means to increase access to college and to improve student progression through higher education programs, the Department of Education report does not present evidence that fully online delivery produces superior learning outcomes for typical college courses, particularly among low-income and academically underprepared students. Indeed some evidence beyond the meta-analysis suggests that, without additional supports, online learning may even undercut progression among low-income and academically underprepared students.

Comment

This is a study that should be read by all those proposing to research the differences between online and face-to-face teaching. It is a very nice dissection of the U.S. Department of Education study:

Means, B. et al. (2009) Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies Washington, DC: US Department of Education

The Columbia University researchers found that of the 28 ‘rigorous, scientific studies’ that compared online vs face-to-face teaching in the Department of Education study, only seven looked at ‘typical’ semester-long courses, and these were all in universities. Close examination of these seven studies by the Columbia University group found (p.10) that ‘Overall, then, the online courses showed no strong advantage or disadvantage in terms of learning outcomes among the samples of students under study.’

More interestingly, Smith Jaggers and Bailey argued that the U.S. Department of Education study produced no evidence to indicate that online learning is widening access to low-income or other disadvantaged learners, nor is there anything else in the literature to support this argument for online learning (a dangerous conclusion, because it only needs one study to come to light to challenge the Columbia University conclusion – in other words, never say never in research. Does anyone know of such a study that looks at whether online learning widens access to disadvantaged groups?).

None of this is surprising. Anyone wanting to do comparative research between face-to-face and technology delivered teaching should say over and over again what Wilbur Schramm said as long ago as 1974: we should not ask whether one means of delivery is better than another, but what are the conditions that determine the appropriate choice of technology.

My view of meta-analysis research is that it should be dumped in the garbage, as far as education is concerned. The in-condition variables are always greater than the between-condition variables, and these are lost in meta-analyses. In other words, it depends on how well the teaching is done, whatever the medium.

So, coming back to the Smith Jaggers and Bailey conclusion: can online learning widen access to disadvantaged groups? Well, first, this may be a bit of a straw man. I don’t recall much of the rationale for online learning in the literature being that it can recruit students from high school who can’t get into college – or don’t want to go. This is a tough market to reach, by any means, and often these students do not have the confidence or independent learning skills to study wholly online. The main justification from my perspective for fully online learning (apart from the skills it develops) is that it suits best adult and lifelong learners who cannot otherwise access college, but have pretty good learning skills already (and may well already have a conventional community college or university education). Online learning may be used to help some kinds of high school drop-outs, but it would need to be combined with a whole range of other strategies, such as personal counselling and one-on-one learner support.

So can online learning widen access to disadvantaged groups? Yes, probably, if the right conditions are there, such as quality learner support, subsidized programs, etc. However, the same conditions could equally apply to face-to-face programs – or different conditions. Neither the U.S. Department of Education nor the Columbia University study provides any evidence on this question, and trying to make this sort of comparison is a waste of time. So focus on the conditions that support teaching and learning, accepting that there likely to be some conditions where online teaching is more or less appropriate than other conditions.

Now try to explain this to your professor who wants a simple answer to the question as to whether online teaching is ‘as good as’ face-to-face teaching. The answer is of course, ‘Yes, provided that…..’ or ‘it all depends….’

See also:

Wojciechowska, I. (2010) Continuing debate over online education Inside Higher Education, July 16

From the Abstract

A systematic search of the research literature from 1996 through July 2008 identified more than a thousand empirical studies of online learning. Analysts screened these studies [...]]]> Means, B. et al. (2009) Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies Washington, DC: US Department of Education

From the Abstract

A systematic search of the research literature from 1996 through July 2008 identified more than a thousand empirical studies of online learning. Analysts screened these studies to find those that (a) contrasted an online to a face-to-face condition, (b) measured student learning outcomes, (c) used a rigorous research design, and (d) provided adequate information to calculate an effect size. As a result of this screening, 51 independent effects were identified that could be subjected to meta-analysis. The meta-analysis found that, on average, students in online learning conditions performed better than those receiving face-to-face instruction.

That seems pretty conclusive, for a research report, doesn’t it? Well, note the following ‘caveat’ in the report:

“In many of the studies showing an advantage for online learning, the online and classroom conditions differed in terms of time spent, curriculum and pedagogy. It was the combination of elements in the treatment conditions (which was likely to have included additional learning time and materials as well as additional opportunities for collaboration) that produced the observed learning advantages. At the same time, one should note that online learning is much more conducive to the expansion of learning time than is face-to-face instruction.”

The report also noted that blended learning (a combination of face-to-face and online learning) produced the best results, but this may be mainly due to the extra ‘time-on-task’ of students in such a context.

For further comments on this report, see:

Jaschik, S. (2009) The Evidence on Online Education Inside Higher Education, June 29

Terry Anderson’s blog: http://terrya.edublogs.org/2009/07/06/past-the-no-significant-difference/