December 20, 2014

Review of ‘Online Distance Education: Towards a Research Agenda.’

Listen with webReader
Drop-out: the elephant in the DE room that no-one wants to talk about

Drop-out: the elephant in the DE room that no-one wants to talk about

Zawacki-Richter, O. and Anderson, T. (eds.) (2014) Online Distance Education: Towards a Research Agenda Athabasca AB: AU Press, pp. 508

It is somewhat daunting to review a book of over 500 pages of research on any topic. I doubt if few other than the editors are likely to read this book from cover to cover. It is more likely to be kept on one’s bookshelf (if these still exist in a digital age) for reference whenever needed. Nevertheless, this is an important work that anyone working in online learning needs to be aware of, so I will do my best to cover it as comprehensively as I can.

Structure of the book

The book is a collection of about 20 chapters by a variety of different authors (more on the choice of authors later). Based on a Delphi study and analysis of ‘key research journals’ in the field, the editors have organized the topic into three sections, with a set of chapters on each sub-section, as follows:

1. Macro-level research: distance education systems and theories

  • access, equity and ethics
  • globalization and cross-cultural issues
  • distance teaching systems and institutions
  • theories and models
  • research methods and knowledge transfer

2. Meso-level research: management, organization and technology

  • management and organization
  • costs and benefits
  • educational technology
  • innovation and change
  • professional development and faculty support
  • learner support services
  • quality assurance

3. Micro-level: teaching and learning in distance education

  • instructional/learning design
  • interaction and communication
  • learner characteristics.

In addition, there is a very useful preface from Otto Peters, an introductory chapter by the editors where they justify their structural organization of research, and a short conclusion that calls for a systematic research agenda in online distance education research.

More importantly, perhaps, Terry Anderson and Olaf Zawacki-Richter demonstrate empirically that research in this field has been skewed towards micro-level research (about half of all publications).  Interestingly, and somewhat surprisingly given its importance, costs and benefits of online distance education is the least researched area.

What I liked

It is somewhat invidious to pick out particular chapters, because different people will have different interests from such a wide-ranging list of topics. I have tended to choose those that I found were new and/or particularly enlightening for me, but other readers’ choices will be different. However, by selecting a few excellent chapters, I hope to give some idea of the quality of the book.

1. The structuring/organization of research

Anderson and Zawacki-Richter have done an excellent job in providing a structural framework for research in this field. This will be useful both for those teaching about online and distance education but in particular for potential Ph.D. students wondering what to study. This book will provide an essential starting point.

2. Summary of the issues in each area of research

Again, the editors have done an excellent job in their introductory chapter in summarizing the content of each of the chapters that follows, and in so doing pulling out the key themes and issues within each area of research. This alone makes the book worthwhile.

3. Globalization, Culture and Online Distance Education

Charlotte (Lani) Gunawardena of the University of New Mexico has written the most comprehensive and deep analysis of this issue that I have seen, and it is an area in which I have a great deal of interest, since most of the online teaching I have done has been with students from around the world and sometimes multi-lingual.

After a general discussion of the issue of globalization and education, she reviews research in the following areas:

  • diverse educational expectations
  • learners and preferred ways of learning
  • socio-cultural environment and online interaction
  • help-seeking behaviours
  • silence
  • language learning
  • researching culture and online distance learning

This chapter should be required reading for anyone contemplating teaching online.

4. Quality assurance in Online Distance Education

I picked this chapter by Colin Latchem because he is so deeply expert in this field that he is able to make what can be a numbingly boring but immensely important topic a fun read, while at the same time ending with some critical questions about quality assurance. In particular Latchem looks at QA from the following perspectives:

  • definitions of quality
  • accreditation
  • online distance education vs campus-based teaching
  • quality standards
  • transnational online distance education
  • open educational resources
  • costs of QA
  • is online distance education yet good enough?
  • an outcomes approach to QA.

This chapter definitely showcases a master at the top of his game.

5. The elephant in the room: student drop-out

This is a wonderfully funny but ultimately serious argument between Ormond Simpson and Alan Woodley about the elephant in the distance education room that no-one wants to mention. Here they start poking the elephant with some sticks (which they note is not likely to be a career-enhancing move.) The basic argument is that institutions should and could do more to reduce drop-out/increase course completion. This chapter also stunned me with providing hard data about really low completion rates for most open university students. I couldn’t help comparing these with the high completion rates for online credit courses at dual-mode (campus-based) institutions, at least in Canada (which of course are not ‘open’ institutions in that students must have good high school qualifications.)

Woodley’s solution to reducing drop-out is quite interesting (and later well argued):

  • make it harder to get in
  • make it harder to get out

In both cases, really practical and not too costly solutions are offered that nevertheless are consistent with open access and high quality teaching.

In summary

The book contains a number of really good chapters that lay out the issues in researching online distance education.

What I disliked

I have to say that I groaned when I first saw the list of contributors. The same old, same old list of distance education experts with a heavy bias towards open universities. Sure, they are nearly all well-seasoned experts, and there’s nothing wrong with that per se (after all, I see myself as one of them.)

But where are the young researchers here, and especially the researchers in open educational resources, MOOCs, social media applications in online learning, and above all researchers from the many campus-based universities now mainstreaming online learning? There is almost nothing in the book about research into blended learning, and flipped classrooms are not even mentioned. OK, the book is about online distance learning but the barriers or distinctions are coming down with a vengeance. This book will never reach those who most need it, the many campus-based instructors now venturing for the first time into online learning in one way or another. They don’t see themselves as primarily distance educators.

And a few of the articles were more like lessons in history than an up-to-date review of research in the field. Readers of this blog will know that I strongly value the history of educational technology and distance learning. But these lessons need to be embedded in the here and now. In particular, the lessons need to be spelled out. It is not enough to know that Stanford University researchers as long ago as 1974 were researching the costs and benefits of educational broadcasting in developing countries, but what lessons does this have for some of the outrageous claims being made about MOOCs? A great deal in fact, but this needs explaining in the context of MOOCs today.

Also the book is solely focused on post-secondary university education. Where is the research on online distance education in the k-12/school sector or the two-year college/vocational sector? Maybe they are topics for other books, but this is where the real gap exists in research publications in online learning.

Lastly, although the book is reasonably priced for its size (C$40), and is available as an e-text as well as the fully printed version, what a pity it is not an open textbook that could then be up-dated and crowd-sourced over time.

Conclusion

This is essential reading for anyone who wants to take a professional, evidence-based approach to online learning (distance or otherwise). It will be particularly valuable for students wanting to do research in this area. The editors have done an incredibly good job of presenting a hugely diverse and scattered area in a clear and structured manner. Many of the chapters are gems of insight and knowledge in the field.

However, we have a huge challenge of knowledge transfer in this field. Repeatedly authors in the book lamented that many of the new entrants to online learning are woefully ignorant of the research previously done in this field. We need a better way to disseminate this research than a 500 page printed text that only those already expert in the field are likely to access. On the other hand, the book does provide a strong foundation from which to find better ways to disseminate this knowledge. Knowledge dissemination in a digital world then is where the research agenda for online learning needs to focus.

 

What UBC has learned about doing MOOCs

Listen with webReader

Coursera certificate 2

Engle, W. (2104) UBC MOOC Pilot: Design and Delivery Vancouver BC: University of British Columbia

The University of British Columbia, a premier public research university in Canada, successfully delivered five MOOCs in the spring and summer of 2013, using the Coursera platform. This report is an evaluation of the experience.

The report is particularly valuable because it provides details of course development and delivery, including media used and costs. Also UBC has been developing online courses for credit for almost 20 years, so it is interesting to see how this has impacted on the design of their MOOCs.

The MOOCs

1. Game Theory I: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

2. Game Theory II: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

3. Useful Genetics: R. Redfield, UBC

4. Climate Literacy: S. Harris and S. Burch, UBC

5. Introduction to Systematic Program Design: G. Kizcales, UBC

In terms of comparability I’m going to treat Game Theory I and II as one MOOC, as combined they were about the same length as the other MOOCs (between 8-12 weeks)

Basic statistics

330,150 signed up (82,500 on average per course)

164,935 logged in at least once (41,000 per course)

12,031 took final exam (3,000 per course)

8,174 earned course certificate (2,000 per course)

60-70% already had a post-secondary degree

30-40% were North American, with participants from nearly every country in the world.

Course development

None of the instructors had taught an online course before, but were supported by instructional designers, media development staff, and academic assistants (graduate and undergraduate students).

One major difference between UBC MOOCs and its online credit courses (which are primarily LMS-based) was the extensive use of video, the main component of the MOOC pilot courses.

Video production

305 videos constituting a total of 65 hours were produced. Each MOOC used a different method of production:

  • Intensive studio (Climate Literacy)
  • Hybrid studio plus instructor desktop (Systematic Program Design)
  • Light studio production (Game Theory I and II)
  • Instructor desktop (Useful Genetics)

Web pages

All the MOOCs except Games Theory also included weekly modules as HTML-based web pages, which is a variation of the Coursera design default model. Altogether 98 HTML module pages were developed. The weekly modules were used to provide guidance to students on learning goals, amount of work expected, an overview of activities, and additional quiz or assignment help. (All standard practice in UBC’s LMS-based credit courses.)

Assessment

1,049 quiz questions were developed, of which just over half were graded.

There were four peer assessments in total across all the MOOCs.

Course delivery

As well as the faculty member responsible for each MOOC, graduate and undergraduate academic assistants were a crucial component of all courses, with the following responsibilities:

  • directly assisting learners
  • troubleshooting technical problems
  • conducting quality assurance activities

There was very little one-on-one interaction between the main instructor and learners, but academic assistants monitored and moderated the online forum discussions.

Costs

As always, costing is a difficult exercise. Appendix B of the report gives a pilot total of $217,657, but this excludes academic assistance or, perhaps the most significant cost, instructor time.

Working from the video production costs ($95,350) and the proportion of costs (44%) devoted to video production in Figure 1 in the report, I estimate the direct cost at $216,700, or approximately $54,000 per MOOC, excluding faculty time and co-ordination support, but including academic assistance.

However, the range of cost is almost as important. The video production costs for Climate Literacy, which used intensive studio production, were more than six times the video production costs of Systematic Program Design (hybrid studio + desktop).

MOOCs as OERs

  • the UBC instructors are using their MOOC materials in their own on-campus, for-credit classes in a flipped classroom model
  • courses are left open and active on Coursera for self-paced learning
  • porting of video materials as open access YouTube videos
  • two courses (Climate Literacy and Useful Genetics) added Creative Commons licenses for re-use

Challenges

  • copyright clearance (Coursera owns the copyright so third party copyright needs to be cleared)
  • higher than expected time demands on all involved
  • iterative upgrades to the Coursera platform
  • partner relationship management (UBC + Coursera + Stanford University) was time-consuming.
  • training and managing academic assistants, especially time management
  • the Coursera platform limited instructors’ ability to develop desired course activities
  • Coursera’s peer assessment functionality in particular was limiting

Lessons

  • UBC’s prior experience in credit-based online learning led to better-designed, more interactive and more engaging MOOCs
  • learners always responded positively to instructor ‘presence’ in forums or course announcements
  • MOOC students were motivated by grades
  • MOOC students were willing to critically engage in critiquing instructors’ expertise and teaching
  • open publishing via MOOCs is a strong motivator for instructors
  • MOOCs require significant investment.

Conclusion

All the MOOCs received positive feedback and comments from students. UBC was able to gain direct experience in and knowledge of MOOCs and look at how this might inform both their for-credit on-campus and online teaching. UBC was also able to bring its experience in for-credit online learning to strengthening the design of MOOCs. Lastly it was able to make much more widely known the quality of UBC instructors and course materials.

Comment

First, congratulations to UBC for

  • experimenting with MOOCs
  • conducting the evaluation
  • making the report publicly available.

It is clear from the comments of participants in the appendices that at least some of the participants (we don’t know how many) were very pleased with the courses. As usual though with evaluation reports on MOOCs, there is no assessment of learning other than the end of course quiz-based tests. I don’t care too much about completion rates, but some measurement of student satisfaction would have been helpful.

It is also significant that UBC has now decided to move from Coursera to edX as its platform for MOOCs. edX, which is open source and allows partners to modify and adapt the platform, provides the flexibility that Coursera lacked, despite its many iterative ‘improvements’.

This also demonstrates the hubris of MOOC platform developers in ignoring best design principles in online learning when they designed their platforms. It is clear that UBC designers were able to improve the design of their MOOCs by drawing on prior for-credit online experience, but also that the MOOC platforms are still very limited in enabling the kind of learning activities that lead to student engagement and success.

The UBC report also highlighted the importance (and cost) of providing some form of learner support in course delivery. The use of academic assistants in particular clearly made the MOOCs more interactive and engaging, as well as limited but effective interventions from the instructors themselves, once again supported by (and confirming) prior research on the importance of instructor presence for successful for-credit online learning.

I very much appreciate the cost data provided by UBC, and the breakdown of production and delivery costs is extremely valuable, but I have to challenge the idea of providing any costs that exclude the time of the instructors. This is by far the largest and most important cost in MOOCs and the notion that MOOCs are free of instructor cost is to fly in the face of any respectable form of economics.

It is clear that MOOCs are more expensive to date per hour of study time than LMS-based for-credit online courses. We still do not have enough data to give a precise figure, and in any case, as the UBC study shows, cost is very much a factor of design. However, even without instructors costs, the UBC MOOCs at $54,000 each for between 8-12 weeks are already more than the average cost of a 13 week for-credit LMS-based online course, including instructor time.

This is partly due to the increased instructor time in preparation/production, but also to the higher cost of video production.  I am not against the use of video in principle, but it must add value. Using it for content transmission when this can be done so much more cheaply textually and/or by audio is a waste of the medium’s potential (although perhaps more motivating for the instructor).

More importantly, every institution contemplating MOOCs needs to do a cost-benefit exercise. Is it better to invest in MOOCs or credit-based online learning or both? If MOOCs are more expensive, what are the added benefits they provide and does this more than make up for not only the extra cost, but the lost opportunity of investing in (more) credit-based online learning or other forms of campus-based learning? I know what my answer would be.

 

Is the ADDIE model appropriate for teaching in a digital age?

Listen with webReader
© Flexible Learning Australia, 2014

Click on the graphic for the interactive version © Flexible Learning Australia, 2014

Chapter 5 of my open textbook, ‘Teaching in a Digital Age‘, is now published. In Chapter 5, I developed the concept of a learning environment.

I am now working on Chapter 6, ‘Models for Designing Teaching and Learning.’ In my last post I discussed the appropriateness of the classroom design model for a digital age. In this post, I explore the same issue for the ADDIE model.

What is ADDIE?

ADDIE stands for:

Analyse

  • identify all the variables that need to be considered when designing the course, such as learner characteristics, learners’ prior knowledge, resources available, etc.  This stage is similar to describing the learning environment outlined in Chapter 5.

Design

  • this stage focuses on identifying the learning objectives for the course and how materials will be created and designed (for instance, it may include describing what content areas are to be covered and a storyboard outlining what will be covered in text, audio and video and in what order), and deciding on the selection and use of technology, such as an LMS, video or social media

Develop

  • the creation of content, including whether to develop in-house or outsource, copyright clearance for third party materials, recording videos or audio, loading of content into a web site or LMS, etc.

Implement

  • this is the actual delivery of the course, including any prior training or briefing of learner support staff, and student assessment

Evaluate

  • feedback and data is collected in order to identify areas that require improvement and this feeds into the design, development and implementation of the next iteration of the course.

The interactive infographic above provides an in-depth, step-by-step approach to the design of learning, with lots of online resources to draw on. There have been many books written about the ADDIE model (see for instance, Morrison, 2010; Dick and Carey, 2004).

Where is ADDIE used?

This is a design model used by many professional instructional designers for technology-based teaching. ADDIE has been almost a standard for professionally developed, high quality distance education programs, whether print-based or online. It is also heavily used in corporate e-learning and training. There are many variations on this model (my favourite is ‘PADDIE’, where planning and/or preparation are added at the start). The model is mainly applied on an iterative basis, with evaluation leading to re-analysis and further design and development modifications.

One reason for the widespread use of the ADDIE model is that it is extremely valuable for large and complex teaching designs. ADDIE ‘s roots go back to the Second World War and derive from system design, which was developed to manage the hugely complex Normandy landings.

The Open University in the United Kingdom heavily uses ADDIE to manage the design of complex multi-media distance education courses. When the OU opened in 1971 with an initial intake of 20,000, it used radio, television, specially designed printed modules, text books, reproduced research articles in the form of selected readings that were mailed to students, and regional study groups, with teams of often 20 academics, media producers and technology support staff developing courses, and with delivery and learner support provided by an army of regional tutors and senior counsellors. Creating and delivering its first courses without systematic instructional design model would have been impossible, and in 2014, with over 200,000 students, the OU still employs a strong instructional design model based on ADDIE.

Although ADDIE and instructional design in general originated in the USA, the Open University’s success in developing high quality learning materials influenced many more institutions that were offering distance education on a much smaller scale to adopt the ADDIE model, if on a more modest scale. As distance education courses became increasingly developed as online courses, the ADDIE model continued, and is now being used by instructional designers in many institutions for the re-design of large lecture classes, hybrid learning, and for fully online courses.

What are the benefits of ADDIE?

One reason it has been so successful is that it is heavily associated with good quality design, with clear learning objectives, carefully structured content, controlled workloads for faculty and students, integrated media, relevant student activities, and assessment strongly tied to desired learning outcomes. Although these good design principles can be applied with or without the ADDIE model, it is a model that allows these design principles to be identified and implemented on a systematic and thorough basis. It is also a very useful management tool, allowing for the design and development of large numbers of courses to a standard high quality.

What are the limitations of ADDIE?

The ADDIE approach can be used with any size of teaching project, but works best with large and complex projects. Applied to courses with small student numbers and a deliberately simple or traditional classroom design, it becomes expensive and possibly redundant, although there is nothing to stop an individual teacher following this strategy when designing and delivering a course.

A second criticism is that the ADDIE model is what might be called ‘front-end loaded’ in that it focuses heavily on content design and development, but does not pay as much attention to the interaction between instructors and students during course delivery. It has been criticised by constructivists for not paying enough attention to learner-instructor interaction, and for privileging more behaviourist approaches to teaching.

Another criticism is that while the five stages are reasonably well described in most descriptions of the model, it does not provide guidance on how to make decisions within that framework. For instance, it does not provide guidelines or procedures for deciding how to choose between different technologies, or what assessment strategies to use. Instructors have to go beyond the ADDIE framework to make these decisions.

The over-enthusiastic application of the ADDIE model can and has resulted in overly complex design stages, with many different categories of workers (faculty, instructional designers, editors, web designers) and consequently a strong division of labour, resulting in courses taking up to two years from initial approval to actual delivery. The more complex the design and management infrastructure, the more opportunities there are for cost over-runs and very expensive programming.

My main criticism though is that the model is too inflexible for the digital age. Adamson (2012) states:

The systems under which the world operates and the ways that individual businesses operate are vast and complex – interconnected to the point of confusion and uncertainty. The linear process of cause and effect becomes increasingly irrelevant, and it is necessary for knowledge workers to begin thinking in new ways and exploring new solutions.

In particular knowledge workers must deal with situations and contexts that are volatile, uncertain, complex and ambiguous (what Adamson calls a VUCA environment). This certainly applies to teachers working with ever changing technologies, very diverse students, a rapidly changing external world that puts pressure on institutions to change.

If we look at course design, how does a teacher respond to rapidly developing new content, new technologies or apps being launched on a daily basis, to a constantly changing student base? For instance, even setting prior learning outcomes is fraught in a VUCA environment, unless you set them at an abstract ‘skill’ level such as thinking flexibly, networking, and information retrieval and analysis. Students need to develop the key knowledge management skills of knowing where to find relevant information, how to assess, evaluate and appropriately apply such information. This means exposing them to less than certain knowledge and providing them with the skills, practice and feedback to assess and evaluate such knowledge then apply that to solving real world problems.

This means designing learning environments that are rich and constantly changing, but enable students to develop and practice the skills and acquire the knowledge they will need in a VUCA world. I would argue that while the ADDIE model has served us well in the past, it is too pre-determined, linear and inflexible to handle this type of learning. I will discuss more flexible models later in this chapter.

Over to you

1. Have I given enough information about what ADDIE is, by using the infographic, or do I need to cover this more fully in the text? Do I need to say something about rapid course development here?

2. What are your views on the ADDIE model? Is it a useful model for designing teaching in a digital age? Do you agree with my criticisms of the model?

3. Any suggestions about other, more flexible models that could be used?

What’s next

So far I have done drafts of the following (as blogs)

  • What is a design model?
  • The classroom model
  • Classroom models in online learning
    • LMSs
    • lecture capture
  • ADDIE

Still to come:

  • Competency-based learning,
  • Connectivist models, including Communities of practice and cMOOCs
  • Flexible design models
  • PLEs
  • AI approaches.
  • Conclusion

My next post in this series then will be on the appropriateness of competency-based learning for teaching in a digital age.

References

Adamson, C. (2012) Learning in a VUCA world, Online Educa Berlin News Portal, November 13

Dick, W., and Carey, L. (2004). The Systematic Design of Instruction. Allyn & Bacon; 6 edition Allyn & Bacon

Morrison, Gary R. (2010) Designing Effective Instruction, 6th Edition. New York: John Wiley & Sons 

Special edition on research on MOOCs in the journal ‘Distance Education’

Listen with webReader
The University of Toronto is one of a number of institutions conducting research on MOOCs

The University of Toronto is one of a number of institutions conducting research on MOOCs; their results are still to come

The August 2014 edition of the Australian-based journal, Distance Education (Vol.35, No. 2.), is devoted to new research on MOOCs. There is a guest editor, Kemi Jona, from Northwestern University, Illinois, as well as the regular editor, Som Naidu.

The six articles in this edition are fascinating, both in terms of their content, but even more so in their diversity. There are also three commentaries, by Jon Baggaley, Gerhard Fischer and myself.

My commentary provides my personal analysis of the six articles.

MOOCs are a changing concept

In most of the literature and discussion about MOOCs, there is a tendency to talk about ‘instructionist’ MOOCs (i.e. Coursera, edX, Udacity, xMOOCs) or ‘connectivist’ MOOCs (i.e. Downes, Siemens, Cormier, cMOOCs). Although this is still a useful distinction, representing very different pedagogies and approaches, the articles in this edition show that MOOCs come in all sizes and varieties.

Indeed, it is clear that the design of MOOCs is undergoing rapid development, partly as a result of more players coming in to the market, partly because of the kinds of research now being conducted on MOOCs themselves, and, sadly much more slowly, a recognition by some of the newer players that much is already known about open and online education that needs to be applied to the design of MOOCs, while accepting that there are certain aspects, in particular the scale, that make MOOCs unique.

The diversity of MOOC designs

These articles illustrate clearly such developments. The MOOCs covered by the articles range from

  • MOOC video recorded lectures watched in isolation by learners (Adams et al.)
  • MOOC video lectures watched in co-located groups in a flipped classroom mode without instructor or tutorial support (Nan Li et al.)
  • MOOCs integrated into regular campus-based programs with some learner support (Firmin et al.)
  • MOOCs using participatory and/or connectivist pedagogy (Anderson, Knox)

Also the size of the different MOOC populations studied here differed enormously, from 54 students per course to 42,000.

It is also clear that MOOC material is being increasingly extracted from the ‘massive’, open context and used in very specific ‘closed’ contexts, such as flipped classrooms, at which point one questions the difference between such use of MOOCs and regular for-credit online programming, which in many cases also use recorded video lectures or online discussion and increasingly other sources of open educational materials. I would expect in such campus-based contexts the same quality standards to apply to the MOOC+ course designs as are already applied to credit-based online learning. Some of the research findings in these articles indirectly support the need for this.

The diversity of research questions on MOOCs

Almost as interesting is the range of questions covered by these articles, which include:

  • capturing the lived experience of being in a MOOC (Adams et al.; Knox)
  • the extent to which learners can/should create their own content, and the challenges around that (Knox; Andersen)
  • how watching video lectures in a group affects learner satisfaction (Nan Li et al.)
  • what ‘massive’ means in terms of a unique pedagogy (Knox)
  • the ethical implications of MOOCs (Marshall)
  • reasons for academic success and failure in ‘flipped’ MOOCs (Firmin et al.; Knox)

What is clear from the articles is that MOOCs raise some fundamental questions about the nature of learning in digital environments. In particular, the question of the extent to which learners need guidance and support in MOOCs, and how this can best be provided, were common themes across several of the papers, with no definitive answers.

The diversity of methodology in MOOC research

Not surprisingly, given the range of research questions, there is also a very wide range of methodologies used in the articles in this edition, ranging from

  • phenomenology (Adams),
  • heuristics (Marshall)
  • virtual ethnography (Knox; Andersen)
  • quasi-experimental comparisons (Nan Li et al.)
  • data and learning analytics (Firmin et al.)

The massiveness of MOOCs, their accessibility, and the wide range of questions they raise make the topic a very fertile area for research, and this is likely to generate new methods of research and analysis in the educational field.

Lessons learned

Readers are likely to draw a variety of conclusions from these studies. Here are mine:

  • the social aspect of learning is extremely important, and MOOCs offer great potential for exploiting this kind of learning, but organizing and managing social learning on a massive scale, without losing the potential advantages of collaboration at scale, is a major challenge that still remains to be adequately addressed. The Knox article in particular describes in graphic detail the sense of being overwhelmed by information in open connectivist MOOCs. We still lack methods or designs that properly support participants in such environments. This is a critical area for further research and development.
  • a lecture on video is still a lecture, whether watched in isolation or in groups. The more we attempt to support this transmissive model through organized group work, ‘facilitators’, or ‘advisors’ the closer we move towards conventional (and traditional) education and the further away from the core concept of a MOOC.
  • MOOCs have a unique place in the educational ecology. MOOCs are primarily instruments for non-formal learning. Trying to adapt MOOCs to the campus not only undermines their primary purpose, but risks moving institutions in the wrong direction. We would be better re-designing our large lecture classes from scratch, using criteria, methods and standards appropriate to the goals of formal higher education. My view is that in the long run, we will learn more from MOOCs about handling social learning at scale than about transmitting information at scale. We already know about that. It’s called broadcasting.
  • lastly, there was surprisingly little in the articles about what actual learning took place. In some cases, it was a deliberate research strategy not to enquire into this, relying more on student or instructor feelings and perceptions. While other potential benefits, such as institutional branding, stimulating interest, providing a network of connections, and so on, are important, the question remains: what are participants actually learning from MOOCs, and does this justify the hype and investment (both institutionally and in participants’ time) that surrounds them?

Cultural and ethical issues

The Marshall paper provides an excellent overview of ethical issues, but there is almost no representation of perspectives on MOOCs from outside Western contexts. I would have liked to have seen more on cultural and ethical issues arising from the globalization of MOOCs, based on actual cases or examples. Given the global implications of MOOCs, other perspectives are needed. Perhaps this is a topic for another issue.

Happy reading

I am sure you will be as fascinated and stimulated by these articles as I am. I am also sure you will come away with different conclusions from mine. I am sure we will see a flood of other articles soon on this topic. Nevertheless, these articles are important in setting the research agenda, and should be essential reading for MOOC designers as well as future researchers on this topic.

How to get the articles

To obtain access to these articles, go to: http://www.tandfonline.com/toc/cdie20/current#.U-1WqrxdWh1

The success or otherwise of online students in the California Community College system

Listen with webReader

 Online offerings vary widely across subject

Johnson, H. and Mejia, M. (2014) Online learning and student outcomes in California’s community colleges San Francisco CA: Public Policy Institute of California, 20 pp

I’m not a great fan of studies into completion rates in online learning, because most studies fail to take into account a whole range of factors outside of the mode of delivery that influence student outcomes. However, this study is an exception. Conducted by researchers at the highly influential PPIC, it takes a very careful look at how well students across the whole California community college system (CCCS) do in online learning, and there are some very interesting findings that may not come as a surprise to experienced observers of online learning, but will certainly provide fodder for both supporters and skeptics of online learning.

Why the study is important

Several reasons:

  • California’s community colleges offer more online credit courses than any other public higher education institution in the country. By 2012, online course enrollment in the state’s community colleges totaled almost one million, representing about 11 percent of total enrollment
  • Over the past ten years, online course enrollment has increased by almost 850,000, while traditional course enrollment has declined by almost 285,000.
  • Community colleges are more likely than other institutions of higher education [in the USA] to serve nontraditional students. These students often have employment and family obligations and therefore may potentially benefit the most from online learning.
  • The state of California is investing $57 million over the next five to six years for online learning initiatives within the California Community College system
  • The California Community Colleges Chancellor’s Office (CCCCO) provided … access to unique longitudinal student- and course-level data from all of the state’s 112 community colleges

Main findings

  • Between 2008–09 and 2011–12, total credit enrollment at California’s community colleges declined by almost a million. The scarcity of traditional courses has been a factor in the huge increase in online enrollments. With the state cutting support to community colleges by more than $1.5 billion between 2007–08 and 2011–12, community colleges experienced an unprecedented falloff in enrollment 
  • online course success rates are between 11 and 14 percentage points lower than traditional course success rates.
  • in the long term, students who take online classes tend to be more successful than those who enroll only in traditional courses…students who take at least some online courses are more likely than those who take only traditional courses to earn an associate’s degree or to transfer to a four-year institution.
  • for students juggling school, family and work obligations, the ability to maintain a full-time load by mixing in one or two online courses per term may outweigh the lower chances of succeeding in each particular online course.
  • if a student’s choice is between taking an online course or waiting for the course to be offered in a classroom at a convenient time, taking the online course can help expedite completion or transfer
  • participation in online courses has increased for each of the state’s largest ethnic groups—and online enrollment rates for African American students, an underrepresented group in higher education in California, are particularly high. However, these rates are much lower among Latino students.

Main recommendations

  • move from ad hoc offerings to more strategic planning of online courses
  • improve the ability to transfer credits between community colleges and between colleges and the state’s universities
  • improve the design and provide more consistency in the quality of online courses between institutions
  • adopt a standardized learning management system across all colleges
  • collect systematic information on the cost of developing and maintaining online courses

My comments

This is another excellent and succinct research report on online learning, with a very strong methodology and important results, even if I am not at all surprised by the outcomes. I would expect online completion rates for individual courses to be lower than for traditional courses as students taking online courses often have a wider range of other commitments to manage than full-time, on campus students.

Similarly, I’m not surprised that online course success is lightly lower for community colleges than for universities (if we take both the figures from Ontario and my own experience as a DE director) and for certain ethnic groups who suffer from a range of socio-economic disadvantages. Online learning is more demanding and requires more experience in studying. Post-graduate students tend to do better at online learning than undergraduate students, and final year undergraduate students tend to do better than first year undergraduate students. Nevertheless, as the study clearly indicates, over the long term online learning provides not only increased access but also a greater chance of success for certain kinds of students.

I am worried though that online learning in California has ‘succeeded’ because of the massive cuts to campus-based education. It is better than nothing, but online learning deserves to be considered in its own right, not as a cheaper alternative to campus-based education. Online learning is not a panacea. Different students have different needs, and a successful public post secondary education system should cater to all needs. In the meantime, this is one of the most useful studies on online completion rates.