November 23, 2014

A ‘starter’ bibliography on MOOCs

Listen with webReader
Image: © educatorstechnology.com, 2014

Image: © educatorstechnology.com, 2014

For the increasing number of students doing Masters’ dissertations or Ph.D’s on MOOCs I have collected together for convenience all the references made in my chapter on MOOCs for my open textbook, ‘Teaching in a Digital World.’ However, there are many other publications – this cannot be considered a comprehensive list. Also note the date of this blog post: anything published after this will not be here, unless you let me know about it.

In return, I would really appreciate other suggestions for references that you have found to be valuable or influential. I’m now less interested in ‘opinion pieces’ but I am looking for more papers that reflect actual experience or research on MOOCs.

Balfour, S. P. (2013). Assessing writing in MOOCs: Automated essay scoring and calibrated peer review. Research & Practice in Assessment, Vol. 8.

Bates, A. (1985) Broadcasting in Education: An Evaluation London: Constables

Bates, A. and Sangrà, A. (2011) Managing Technology in Higher Education San Francisco: Jossey-Bass/John Wiley and Co

Bates, T. (2012) What’s right and what’s wrong with Coursera-style MOOCs Online Learning and Distance Education Resources, August 5

Bayne, S. (2014) Teaching, Research and the More-than-Human in Digital Education Oxford UK: EDEN Research Workshop (keynote: no printed record available)

Book, P. (2103) ACE as Academic Credit Reviewer–Adjustment, Accommodation, and Acceptance WCET Learn, July 25

Chauhan, A. (2014) Massive Open Online Courses (MOOCS): Emerging Trends in Assessment and Accreditation Digital Education Review, No. 25

Christensen, C. (2010) Disrupting Class, Expanded Edition: How Disruptive Innovation Will Change the Way the World Learns New York: McGraw-Hill

Christensen, C. and Eyring, H. (2011), The Innovative University: Changing the DNA of Higher Education, New York, New York, USA: John Wiley & Sons,

Christensen, C. and Weise, M. (2014) MOOCs disruption is only beginning, The Boston Globe, May 9

Collins, E. (2013) SJSU Plus Augmented Online Learning Environment Pilot Project Report San Jose CA: The Research and Planning Group for California Colleges

Colvin, K. et al. (2014) Learning an Introductory Physics MOOC: All Cohorts Learn Equally, Including On-Campus Class, IRRODL, Vol. 15, No. 4

Daniel, J. (2012) Making sense of MOOCs: Musings in a maze of myth, paradox and possibility Seoul: Korean National Open University

Dillenbourg, P. (ed.) (1999) Collaborative-learning: Cognitive and Computational Approaches. Oxford: Elsevier

Dillenbourg, P. (2014) MOOCs: Two Years Later, Oxford UK: EDEN Research Workshop (keynote: no printed record available)

Downes, S. (2012) Massively Open Online Courses are here to stay, Stephen’s Web, July 20

Downes, S. (2014) The MOOC of One, Valencia, Spain, March 10

Engle, W. (2104) UBC MOOC Pilot: Design and Delivery Vancouver BC: University of British Columbia

Falchikov, N. and Goldfinch, J. (2000) Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks Review of Educational Research, Vol. 70, No. 3

Firmin, R. et al. (2014) Case study: using MOOCs for conventional college coursework Distance Education, Vol. 35, No. 2

Friedland, T. (2013) Revolution hits the universities, New York Times, January 26

Harasim, L. (2012) Learning Theory and Online Technologies New York/London: Routledge

Haynie, D. (2014). State Department hosts ‘MOOC Camp’ for online learners. US News,January 20

Hernandez, R. et al. (2014) Promoting engagement in MOOCs through social collaboration Oxford UK: Proceedings of the 8th EDEN Research Workshop

Hill, P. (2012) Four Barriers that MOOCs Must Overcome to Build a Sustainable Model e-Literate, July 24

Ho, A. et al. (2014) HarvardX and MITx: The First Year of Open Online Courses Fall 2012-Summer 2013 (HarvardX and MITx Working Paper No. 1), January 21

Hollands, F. and Tirthali, D. (2014) MOOCs: Expectations and Reality New York: Columbia University Teachers’ College, Center for Benefit-Cost Studies of Education

Hülsmann, T. (2003) Costs without camouflage: a cost analysis of Oldenburg University’s  two graduate certificate programs offered  as part of the online Master of Distance Education (MDE): a case study, in Bernath, U. and Rubin, E., (eds.) Reflections on Teaching in an Online Program: A Case Study Oldenburg, Germany: Bibliothecks-und Informationssystem der Carl von Ossietsky Universität Oldenburg

Jaschik, S. (2013) MOOC Mess, Inside Higher Education, February 4

Knox, J. (2014) Digital culture clash: ‘massive’ education in the e-Learning and Digital Cultures Distance Education, Vol. 35, No. 2

Kop, R. (2011) The Challenges to Connectivist Learning on Open Online Networks: Learning Experiences during a Massive Open Online Course International Review of Research into Open and Distance Learning, Vol. 12, No. 3

Lave, J. and Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press

Lyotard, J-J. (1979) La Condition postmoderne: rapport sur le savoir: Paris: Minuit

Mackness, J. (2013) cMOOCs and xMOOCs – key differences, Jenny Mackness, October 22

Milligan, C., Littlejohn, A. and Margaryan, A. (2013) Patterns of engagement in connectivist MOOCs, Merlot Journal of Online Learning and Teaching, Vol. 9, No. 2

Piech, C., Huang, J., Chen, Z., Do, C., Ng, A., & Koller, D. (2013). Tuned models of peer assessment in MOOCs. Palo Alto, CA: Stanford University.

Rumble, G. (2001) The costs and costing of networked learning, Journal of Asynchronous Learning Networks, Vol. 5, No. 2

Suen, H. (2104) Peer assessment for massive open online courses (MOOCs) International Review of Research into Open and Distance Learning, Vol. 15, No. 3

Tapscott, D. (undated) The transformation of education dontapscott.com

University of Ottawa (2013) Report of the e-Learning Working Group Ottawa ON: The University of Ottawa

van Zundert, M., Sluijsmans, D., van Merriënboer, J. (2010). Effective peer assessment processes: Research findings and future directions. Learning and Instruction, 20, 270-279

Watters, A. (2012) Top 10 Ed-Tech Trends of 2012: MOOCs Hack Education, December 3

Yousef, A. et al. (2014) MOOCs: A Review of the State-of-the-Art Proceedings of 6th International Conference on Computer Supported Education – CSEDU 2014, Barcelona, Spain

A New Zealand analysis of MOOCs

Listen with webReader

NZ MOOCs 2

Shrivastava, A. and Guiney, P. (2014) Technological Development and Tertiary Education Delivery Models: The Arrival of MOOCs  Wellington NZ: Tertiary Education Commission/Te Amorangi Mātauranga Matua

Why this paper?

Another report for the record on MOOCs, this time from the New Zealand Tertiary Education Commission. The reasoning behind this report:

The paper focuses on MOOCs [rather than doing a general overview of emerging technologies] because of their potential to disrupt tertiary education and the significant opportunities, challenges and risks that they present. MOOCs are also the sole focus of this paper because of their scale and the involvement of the elite United States universities.

What’s in the paper?

The paper provides a fairly standard, balanced analysis of developments in MOOCs, first by describing the different MOOC delivery models, their business models and the drivers behind MOOCs, then by following up with a broad discussion of the possible implications of MOOCs for New Zealand, such as unbundling of services, possible economies of scale, globalization of tertiary (higher) education, adaptability to learners’ and employers’ needs, and the possible impact on New Zealand’s tertiary education workforce.

There is also a good summary of MOOCs being offered by New Zealand institutions.

At the end of the paper some interesting questions for further discussion are raised:

  • What will tertiary education delivery look like in 2030?

  • What kinds of opportunities and challenges do technological developments, including MOOCs, present to the current policy, regulatory and operational arrangements for tertiary teaching and learning in New Zealand?

  • How can New Zealand make the most of the opportunities and manage any associated risks and challenges?

  • Do MOOCs undermine the central value of higher education, or are they just a helpful ‘updating’ that reflects its new mass nature?

  • Where do MOOCs fit within the New Zealand education and qualifications systems?

  • Who values the knowledge and skills gained from a MOOC programme and why?

  • Can economies of scale be achieved through MOOCs without loss of quality?

  • Can MOOCs lead to better learning outcomes at the same or less cost than traditional classroom-based teaching? If so, how might the Government go about funding institutions that want to deliver MOOCs to a mix of domestic and international learners?

  • What kinds of MOOC accreditation models might make sense in the context of New Zealand’s quality-assurance system?

Answers on a postcard, please, to the NZ Tertiary Education Commission.

Comment

Am I alone in wondering what has happened to for-credit online education in government thinking about the future? It is as if 20 years of development of undergraduate and graduate online courses and programs never existed. Surely a critical question for institutions and government planners is:

  • what are the relative advantages and disadvantages of MOOCs over other forms of online learning? What can MOOCs learn from our prior experience with credit-based online learning?

There are several reasons for considering this, but one of the most important is the huge investment many institutions, and, indirectly, governments. have already made in credit-based online learning.

By and large, online learning in publicly funded universities, both in New Zealand and in Canada, has been very successful in terms of both increasing access and in student learning. It is also important to be clear about the differences and some of the similarities between credit-based online learning and MOOCs.

Some of the implications laid out in this paper, such as possibilities of consortia and institutional collaboration, apply just as much to credit-based online learning as to MOOCs, and many of the negative criticisms of MOOCs, such as difficulties of assessment and lack of learner support, disappear when applied to credit-based online learning.

Please, policy-makers, realise that MOOCs are not your only option for innovation through online learning. There are more established and well tested solutions already available.

Review of ‘Online Distance Education: Towards a Research Agenda.’

Listen with webReader
Drop-out: the elephant in the DE room that no-one wants to talk about

Drop-out: the elephant in the DE room that no-one wants to talk about

Zawacki-Richter, O. and Anderson, T. (eds.) (2014) Online Distance Education: Towards a Research Agenda Athabasca AB: AU Press, pp. 508

It is somewhat daunting to review a book of over 500 pages of research on any topic. I doubt if few other than the editors are likely to read this book from cover to cover. It is more likely to be kept on one’s bookshelf (if these still exist in a digital age) for reference whenever needed. Nevertheless, this is an important work that anyone working in online learning needs to be aware of, so I will do my best to cover it as comprehensively as I can.

Structure of the book

The book is a collection of about 20 chapters by a variety of different authors (more on the choice of authors later). Based on a Delphi study and analysis of ‘key research journals’ in the field, the editors have organized the topic into three sections, with a set of chapters on each sub-section, as follows:

1. Macro-level research: distance education systems and theories

  • access, equity and ethics
  • globalization and cross-cultural issues
  • distance teaching systems and institutions
  • theories and models
  • research methods and knowledge transfer

2. Meso-level research: management, organization and technology

  • management and organization
  • costs and benefits
  • educational technology
  • innovation and change
  • professional development and faculty support
  • learner support services
  • quality assurance

3. Micro-level: teaching and learning in distance education

  • instructional/learning design
  • interaction and communication
  • learner characteristics.

In addition, there is a very useful preface from Otto Peters, an introductory chapter by the editors where they justify their structural organization of research, and a short conclusion that calls for a systematic research agenda in online distance education research.

More importantly, perhaps, Terry Anderson and Olaf Zawacki-Richter demonstrate empirically that research in this field has been skewed towards micro-level research (about half of all publications).  Interestingly, and somewhat surprisingly given its importance, costs and benefits of online distance education is the least researched area.

What I liked

It is somewhat invidious to pick out particular chapters, because different people will have different interests from such a wide-ranging list of topics. I have tended to choose those that I found were new and/or particularly enlightening for me, but other readers’ choices will be different. However, by selecting a few excellent chapters, I hope to give some idea of the quality of the book.

1. The structuring/organization of research

Anderson and Zawacki-Richter have done an excellent job in providing a structural framework for research in this field. This will be useful both for those teaching about online and distance education but in particular for potential Ph.D. students wondering what to study. This book will provide an essential starting point.

2. Summary of the issues in each area of research

Again, the editors have done an excellent job in their introductory chapter in summarizing the content of each of the chapters that follows, and in so doing pulling out the key themes and issues within each area of research. This alone makes the book worthwhile.

3. Globalization, Culture and Online Distance Education

Charlotte (Lani) Gunawardena of the University of New Mexico has written the most comprehensive and deep analysis of this issue that I have seen, and it is an area in which I have a great deal of interest, since most of the online teaching I have done has been with students from around the world and sometimes multi-lingual.

After a general discussion of the issue of globalization and education, she reviews research in the following areas:

  • diverse educational expectations
  • learners and preferred ways of learning
  • socio-cultural environment and online interaction
  • help-seeking behaviours
  • silence
  • language learning
  • researching culture and online distance learning

This chapter should be required reading for anyone contemplating teaching online.

4. Quality assurance in Online Distance Education

I picked this chapter by Colin Latchem because he is so deeply expert in this field that he is able to make what can be a numbingly boring but immensely important topic a fun read, while at the same time ending with some critical questions about quality assurance. In particular Latchem looks at QA from the following perspectives:

  • definitions of quality
  • accreditation
  • online distance education vs campus-based teaching
  • quality standards
  • transnational online distance education
  • open educational resources
  • costs of QA
  • is online distance education yet good enough?
  • an outcomes approach to QA.

This chapter definitely showcases a master at the top of his game.

5. The elephant in the room: student drop-out

This is a wonderfully funny but ultimately serious argument between Ormond Simpson and Alan Woodley about the elephant in the distance education room that no-one wants to mention. Here they start poking the elephant with some sticks (which they note is not likely to be a career-enhancing move.) The basic argument is that institutions should and could do more to reduce drop-out/increase course completion. This chapter also stunned me with providing hard data about really low completion rates for most open university students. I couldn’t help comparing these with the high completion rates for online credit courses at dual-mode (campus-based) institutions, at least in Canada (which of course are not ‘open’ institutions in that students must have good high school qualifications.)

Woodley’s solution to reducing drop-out is quite interesting (and later well argued):

  • make it harder to get in
  • make it harder to get out

In both cases, really practical and not too costly solutions are offered that nevertheless are consistent with open access and high quality teaching.

In summary

The book contains a number of really good chapters that lay out the issues in researching online distance education.

What I disliked

I have to say that I groaned when I first saw the list of contributors. The same old, same old list of distance education experts with a heavy bias towards open universities. Sure, they are nearly all well-seasoned experts, and there’s nothing wrong with that per se (after all, I see myself as one of them.)

But where are the young researchers here, and especially the researchers in open educational resources, MOOCs, social media applications in online learning, and above all researchers from the many campus-based universities now mainstreaming online learning? There is almost nothing in the book about research into blended learning, and flipped classrooms are not even mentioned. OK, the book is about online distance learning but the barriers or distinctions are coming down with a vengeance. This book will never reach those who most need it, the many campus-based instructors now venturing for the first time into online learning in one way or another. They don’t see themselves as primarily distance educators.

And a few of the articles were more like lessons in history than an up-to-date review of research in the field. Readers of this blog will know that I strongly value the history of educational technology and distance learning. But these lessons need to be embedded in the here and now. In particular, the lessons need to be spelled out. It is not enough to know that Stanford University researchers as long ago as 1974 were researching the costs and benefits of educational broadcasting in developing countries, but what lessons does this have for some of the outrageous claims being made about MOOCs? A great deal in fact, but this needs explaining in the context of MOOCs today.

Also the book is solely focused on post-secondary university education. Where is the research on online distance education in the k-12/school sector or the two-year college/vocational sector? Maybe they are topics for other books, but this is where the real gap exists in research publications in online learning.

Lastly, although the book is reasonably priced for its size (C$40), and is available as an e-text as well as the fully printed version, what a pity it is not an open textbook that could then be up-dated and crowd-sourced over time.

Conclusion

This is essential reading for anyone who wants to take a professional, evidence-based approach to online learning (distance or otherwise). It will be particularly valuable for students wanting to do research in this area. The editors have done an incredibly good job of presenting a hugely diverse and scattered area in a clear and structured manner. Many of the chapters are gems of insight and knowledge in the field.

However, we have a huge challenge of knowledge transfer in this field. Repeatedly authors in the book lamented that many of the new entrants to online learning are woefully ignorant of the research previously done in this field. We need a better way to disseminate this research than a 500 page printed text that only those already expert in the field are likely to access. On the other hand, the book does provide a strong foundation from which to find better ways to disseminate this knowledge. Knowledge dissemination in a digital world then is where the research agenda for online learning needs to focus.

 

WCET’s analysis of U.S. statistics on distance education

Listen with webReader

IPEDS 2

U.S.Department of Education (2014) Web Tables: Enrollment in Distance Education Courses, by State: Fall 2012 Washington DC: U.S.Department of Education National Center for Education Statistics

Hill, P. and Poulin, R. (2014) A response to new NCES report on distance education e-Literate, June 11

The U.S. Department of Education’s Institute of Education Sciences operates a National Center for Education Statistics which in turn runs the Integrated Postsecondary Education Data System (IPEDS). IPEDS is:

a system of interrelated surveys conducted annually by the U.S. Department’s National Center for Education Statistics (NCES). IPEDS gathers information from every college, university, and technical and vocational institution that participates in the federal student financial aid programs. The Higher Education Act of 1965, as amended, requires that institutions that participate in federal student aid programs report data on enrollments, program completions, graduation rates, faculty and staff, finances, institutional prices, and student financial aid. These data are made available to students and parents through the College Navigator college search Web site and to researchers and others through the IPEDS Data Center

Recently IPEDS released “Web Tables” containing results from their Fall Enrollment 2012 survey. This was the first survey in over a decade to include institutional enrollment counts for distance education students. In the article above, Phil Hill of e-Literate and Russell Poulin of WCET have co-written a short analysis of the Web Tables released by IPEDS.

The Hill and Poulin analysis

The main points they make are as follows:

  • overall the publication of the web tables in the form of a pdf is most welcome, in particular by providing a breakdown of IPEDS data by different variables such as state jurisdiction, control of institution, sector and student level
  • according to the IPEDS report there were just over 5.4 million students enrolled in distance education courses in the fall semester 2012 (NOTE: this number refers to students, NOT course enrollments).
  • roughly a quarter of all post-secondary students in the USA are enrolled in a distance education course.
  • the bulk of students in the USA taking distance education courses are in publicly funded institutions (85% of those taking at least some DE courses), although about one third of those taking all their classes at a distance are in private, for-profit institutions (e.g. University of Phoenix)
  • these figures do NOT include MOOC enrollments
  • as previously identified by Phil Hill in e-Literate, there is major discrepancy in the number of students taking at least one online course between the IPEDS study and the regular annual surveys conducted by Allen and Seaman at Babson College – 7.1 million for Babson and 5.5 million for IPEDS. Jeff Seaman, one of the two Babson authors, is also quoted in e-Literate on his interpretation of the differences. Hill and Poulin comment that the NCES report would have done well to at least refer to the significant differences.
  • Hill and Poulin claim that there has been confusion over which students get counted in IPEDS reporting and which do not. They suspect that there is undercounting in the hundreds of thousands, independent of distance education status.

Comment

There are lies, damned lies and statistics. Nevertheless, although the IPEDS data may not be perfect, it does a pretty good job of collecting data on distance education students across the whole of the USA. However, it does not distinguish between mode of delivery of distance education (are there still mainly print-based courses around)?

So we now have two totally independent analyses of distance education students in the USA, with a minimum number of 5.5 million and a maximum number of 7.1 million, i.e. between roughly a quarter and a third of all post-secondary students. From the Allen and Seaman longitudinal studies, we can also reasonably safely assume that online enrollments have been increasing between 10-20% per annum over the last 10 years, compared with overall enrollments of 2-5% per annum.

By contrast, in Canada we have no national data on either online or distance education students. It’s hard to see how Canadian governments or institutions can take evidence-based policy decisions about online or distance education without such basic information.

Lastly, thank you, Phil and Russ, for a very helpful analysis of the IPEDs report.

Update

For a more detailed analysis, see also:

Haynie, D. (2014) New Government Data Sheds Light on Online Learners US News, June 13

 

Opening up: chapter one of Teaching in a Digital Age

Listen with webReader
The view when I was writing Chapter 1, from the Island of Braç, Croatia

The view when I was writing Chapter 1, from the Island of Braç, Croatia

I’ve not been blogging much recently, because (a) I’ve been on holiday for a month in the Mediterranean and (b) I’ve been writing my book.

Teaching in a Digital World

As you are probably aware, I’m doing this as an open textbook, which means learning to adapt to a new publishing environment. As well as writing a darned good book for instructors on teaching in in a digital age, my aim is to push the boundaries a little with open publishing, to move it out of the traditional publishing mode into a a truly open textbook, with the help of the good folks at BCcampus who are running their open textbook project.

You will see that there’s still a long way to go before we can really exploit all the virtues of openness in publishing, and I’m hoping you can help me – and BCcampus- along the way with this.

What I’d like you to do

What I’m hoping you will do is find the time to browse the content list and preface (which is not yet finalized) and read more carefully Chapter 1, Fundamental Change in Higher Education, then give me some feedback. To do this, just go to: http://opentextbc.ca/teachinginadigitalage/

The first thing you will realise is that there is nowhere to comment on the published version. (Ideally I would like to have a comment section after every section of each chapter.) I will be publishing another post about some of the technical features I feel are still needed within PressBooks, but in the meantime, please use the comment page on this post (in which case your comment will be public), or use the e-mail facility  at the bottom of the chapter or preface (in which case your comment will be private). Send to tony.bates@ubc.ca .

What kind of feedback?

At this stage, I’m looking more for comments on the substance of the book, rather than the openness (my next post will deal with the technical issues). To help you with feedback, here are some of the questions I’m looking for answers to:

  1. Market: from what you’ve read so far, does there appear to be a need for this type of book? Are there other books that already do what I’m trying to do?
  2. Structure: does Chapter 1 have the right structure? Does it flow and is it logically organized?How could it be improved?
  3. Content: is there anything missing, dubious or just plain wrong? References that I have missed that support (or challenge) the content would also be useful.
  4. Do the activities work for you? Are there more interesting activities you can think of? How best to provide feedback? (e.g. does the use of ‘Parts’ work for this?)
  5. Presentation: are there other media/better images I could use? Is the balance between text and media right?

What’s in it for you?

First, I hope the content will be useful. Chapter 1 is probably the least useful of all the chapters to come for readers of this blog, because it’s aimed at instructors who are not comfortable with using technology, but if the material is useful to you, you are free to use it in whatever way you wish, within the constraints of a Creative Commons license.

Second, the whole point of open education is to share and collaborate. I’m opening up my book and the process; in return can I get some help and advice? In anticipation and with a degree of nervousness I look forward to your comments.