August 29, 2014

Special edition on research on MOOCs in the journal ‘Distance Education’

Listen with webReader
The University of Toronto is one of a number of institutions conducting research on MOOCs

The University of Toronto is one of a number of institutions conducting research on MOOCs; their results are still to come

The August 2014 edition of the Australian-based journal, Distance Education (Vol.35, No. 2.), is devoted to new research on MOOCs. There is a guest editor, Kemi Jona, from Northwestern University, Illinois, as well as the regular editor, Som Naidu.

The six articles in this edition are fascinating, both in terms of their content, but even more so in their diversity. There are also three commentaries, by Jon Baggaley, Gerhard Fischer and myself.

My commentary provides my personal analysis of the six articles.

MOOCs are a changing concept

In most of the literature and discussion about MOOCs, there is a tendency to talk about ‘instructionist’ MOOCs (i.e. Coursera, edX, Udacity, xMOOCs) or ‘connectivist’ MOOCs (i.e. Downes, Siemens, Cormier, cMOOCs). Although this is still a useful distinction, representing very different pedagogies and approaches, the articles in this edition show that MOOCs come in all sizes and varieties.

Indeed, it is clear that the design of MOOCs is undergoing rapid development, partly as a result of more players coming in to the market, partly because of the kinds of research now being conducted on MOOCs themselves, and, sadly much more slowly, a recognition by some of the newer players that much is already known about open and online education that needs to be applied to the design of MOOCs, while accepting that there are certain aspects, in particular the scale, that make MOOCs unique.

The diversity of MOOC designs

These articles illustrate clearly such developments. The MOOCs covered by the articles range from

  • MOOC video recorded lectures watched in isolation by learners (Adams et al.)
  • MOOC video lectures watched in co-located groups in a flipped classroom mode without instructor or tutorial support (Nan Li et al.)
  • MOOCs integrated into regular campus-based programs with some learner support (Firmin et al.)
  • MOOCs using participatory and/or connectivist pedagogy (Anderson, Knox)

Also the size of the different MOOC populations studied here differed enormously, from 54 students per course to 42,000.

It is also clear that MOOC material is being increasingly extracted from the ‘massive’, open context and used in very specific ‘closed’ contexts, such as flipped classrooms, at which point one questions the difference between such use of MOOCs and regular for-credit online programming, which in many cases also use recorded video lectures or online discussion and increasingly other sources of open educational materials. I would expect in such campus-based contexts the same quality standards to apply to the MOOC+ course designs as are already applied to credit-based online learning. Some of the research findings in these articles indirectly support the need for this.

The diversity of research questions on MOOCs

Almost as interesting is the range of questions covered by these articles, which include:

  • capturing the lived experience of being in a MOOC (Adams et al.; Knox)
  • the extent to which learners can/should create their own content, and the challenges around that (Knox; Andersen)
  • how watching video lectures in a group affects learner satisfaction (Nan Li et al.)
  • what ‘massive’ means in terms of a unique pedagogy (Knox)
  • the ethical implications of MOOCs (Marshall)
  • reasons for academic success and failure in ‘flipped’ MOOCs (Firmin et al.; Knox)

What is clear from the articles is that MOOCs raise some fundamental questions about the nature of learning in digital environments. In particular, the question of the extent to which learners need guidance and support in MOOCs, and how this can best be provided, were common themes across several of the papers, with no definitive answers.

The diversity of methodology in MOOC research

Not surprisingly, given the range of research questions, there is also a very wide range of methodologies used in the articles in this edition, ranging from

  • phenomenology (Adams),
  • heuristics (Marshall)
  • virtual ethnography (Knox; Andersen)
  • quasi-experimental comparisons (Nan Li et al.)
  • data and learning analytics (Firmin et al.)

The massiveness of MOOCs, their accessibility, and the wide range of questions they raise make the topic a very fertile area for research, and this is likely to generate new methods of research and analysis in the educational field.

Lessons learned

Readers are likely to draw a variety of conclusions from these studies. Here are mine:

  • the social aspect of learning is extremely important, and MOOCs offer great potential for exploiting this kind of learning, but organizing and managing social learning on a massive scale, without losing the potential advantages of collaboration at scale, is a major challenge that still remains to be adequately addressed. The Knox article in particular describes in graphic detail the sense of being overwhelmed by information in open connectivist MOOCs. We still lack methods or designs that properly support participants in such environments. This is a critical area for further research and development.
  • a lecture on video is still a lecture, whether watched in isolation or in groups. The more we attempt to support this transmissive model through organized group work, ‘facilitators’, or ‘advisors’ the closer we move towards conventional (and traditional) education and the further away from the core concept of a MOOC.
  • MOOCs have a unique place in the educational ecology. MOOCs are primarily instruments for non-formal learning. Trying to adapt MOOCs to the campus not only undermines their primary purpose, but risks moving institutions in the wrong direction. We would be better re-designing our large lecture classes from scratch, using criteria, methods and standards appropriate to the goals of formal higher education. My view is that in the long run, we will learn more from MOOCs about handling social learning at scale than about transmitting information at scale. We already know about that. It’s called broadcasting.
  • lastly, there was surprisingly little in the articles about what actual learning took place. In some cases, it was a deliberate research strategy not to enquire into this, relying more on student or instructor feelings and perceptions. While other potential benefits, such as institutional branding, stimulating interest, providing a network of connections, and so on, are important, the question remains: what are participants actually learning from MOOCs, and does this justify the hype and investment (both institutionally and in participants’ time) that surrounds them?

Cultural and ethical issues

The Marshall paper provides an excellent overview of ethical issues, but there is almost no representation of perspectives on MOOCs from outside Western contexts. I would have liked to have seen more on cultural and ethical issues arising from the globalization of MOOCs, based on actual cases or examples. Given the global implications of MOOCs, other perspectives are needed. Perhaps this is a topic for another issue.

Happy reading

I am sure you will be as fascinated and stimulated by these articles as I am. I am also sure you will come away with different conclusions from mine. I am sure we will see a flood of other articles soon on this topic. Nevertheless, these articles are important in setting the research agenda, and should be essential reading for MOOC designers as well as future researchers on this topic.

How to get the articles

To obtain access to these articles, go to: http://www.tandfonline.com/toc/cdie20/current#.U-1WqrxdWh1

Dialogue and discussion: critical for 21st century skills development

Listen with webReader

Tutorial 2

Introduction

First of all, thanks to the numerous people who commented on  my earlier posts on Why Lectures are Dead, and on Learning Theories and Online Learning. These were previews of chapters for my open textbook, Teaching in a Digital Age.

This feedback was particularly helpful, because several people commented that there are lots of different kinds of lectures. I fully accept that criticism, and although I did define ‘lecture’ quite narrowly in the actual post (some of the comments picked up quotes from the article in the form of tweets or LinkedIn comments that did not include the narrow definition I used.) That definition was really about lectures that were focused primarily or entirely on the transmission of knowledge. I have therefore changed the heading of that section in the book to ‘Transmissive Lectures.’

In the next section, I discuss another important method of teaching based on discussion and argument that reflects a more constructivist approach to teaching and learning. Here is the first draft.

Interactive lectures, seminars, tutorials and MOOCs

In this section, I will examine a number of different ways in which teaching can help develop conceptual knowledge. There is a particular emphasis on conceptual learning at a post-secondary level, but in recent years conceptual learning has become an increasing focus in the school or k-12 systems in many jurisdictions.

The theoretical and research basis for social learning

In the previous section, I wrote that research on lectures showed that:

‘in order to understand, analyze, apply, and commit information to long-term memory, the learner must actively engage with the material. In order for a lecture to be effective, it must include activities that compel the student to mentally manipulate the information.’

This is a cognitive approach to learning, but constructivists go beyond interaction between student and learning materials. They believe, as I wrote earlier, that:

‘individuals consciously strive for meaning to make sense of their environment in terms of past experience and their present state. It is an attempt to create order in their minds out of disorder, to resolve incongruities, and to reconcile external realities with prior experience. Problems are resolved, and incongruities sorted out, through strategies such as seeking relationships between what was known and what is new, identifying similarities and differences, and testing hypotheses or assumptions…knowledge is mainly acquired through social processes or institutions that are socially constructed.’

Researchers have identified a distinction, often intuitively recognised by instructors, between meaningful and rote learning (Asubel et al, 1978). Meaningful learning involves the learner going beyond memorization or surface comprehension of facts, ideas or principles, to a deeper understanding of what those facts, ideas or principles mean to them. Marton and Saljö, who have conducted a number of studies that examined how university students actually go about their learning, make the distinction between deep and surface approaches to learning (see, for instance, Marton and Saljö, 1997).

Students who adopt a deep approach to learning tend to have a prior intrinsic interest in the subject. Their motivation is to learn because they want to know more about a topic. Students with a surface approach to learning are more instrumental. Their interest is primarily driven by the need to get a pass grade or qualification.

Subsequent research (e.g. Entwistle and Peterson, 2004) showed that as well as students’ initial motivation for study, a variety of other factors also influence students’ approaches to learning. In particular, certain learning environments, such as an emphasis in the teaching on information transmission, tests that rely mainly on memory, and a lack of interaction and discussion, encourage surface approaches to learning, while a focus on analytical or critical thinking or problem-solving, in-class discussion, and assessment based on analysis, synthesis, comparison and evaluation tends to drive students more to a deeper approach to learning. It should also be noted that approaches to learning are not always consistent or stable, even for the same student in the same course. Nevertheless, the teaching environment is critical in establishing expectations and methods that are more likely to engage students and hence lead to more conceptual and deeper learning

In addition, others, such as Laurillard (2001) and Harasim (2010), have emphasised that academic knowledge requires students to move constantly from the concrete to the abstract and back again, and to build or construct knowledge based on academic criteria such as logic, evidence and argument. This in turn, it is argued, requires a strong teacher presence within a dialectical environment, in which argument and discussion within the rules and criteria of the subject discipline are encouraged and developed by the instructor or teacher. Laurillard calls this a rhetorical exercise, an attempt to get learners to think about the world differently.

Lastly, connectivist approaches to learning place heavy emphasis on networking learners, with all participants learning through interaction and discussion between each other, driven both by their individual interests and the extent to which these interests connect to the interests of other participants. The very large numbers participating means that there is a high probability of converging interests for all participants, although those interests may vary considerably over the whole group.

The combination of theory and research here suggests the need for frequent interaction between students, and between teacher and students, for the kinds of learning needed in a digital age. This interaction usually takes the form of semi-structured discussion. I will now examine the very wide range of ways in which this kind of learning is facilitated by educators.

Interactive lectures

Definition: An interactive lecture is a lecture where at least 25% of the time is taken up with questions and discussion from students and responses from the lecturer to points raised by students

Many lecturers deliberately design a large lecture experience to encourage interaction with and between students, even though the focus of the lecture is still mainly on information transmission, such as facts, concepts, procedures and ideas. The most common format is to allow at least 15 to 20 minutes at the end of the presentation for questions and discussion, where the instructor may well take the lead in putting questions to students and requiring particular students to provide a response, to get discussion going. However, the research suggests that a better way to ensure comprehension and the development of conceptual thinking is to break the session into small chunks of perhaps 10 minutes presentation, followed by five to ten minutes of questions and discussion.

© Center for Teaching and Learning, University of Washington, 2014

© Center for Teaching and Learning, University of Washington, 2014

More recently, instructors have started to record their lectures through lecture capture then use class time for discussion of the contents of the lecture. This model is called the ‘flipped’ classroom. Again, this is still a mainly transmissive way of teaching requiring students to respond to instructor-led presentation, and there are sometimes problems in getting students to view the lecture in advance of the class time. Clickers and Twitter back-chat channels are other ways in which technology has been used to increase student interaction.

Even interactive lectures can be criticised as being mainly behaviouristic, with a defined body of knowledge to be learned and assimilated by the student. Often discussion is cut short because ‘there is too much content to get through’ to cover the curriculum, and consequently students adopt surface rather than deep approaches to learning. Problems often appear later, when for instance students who need mathematical concepts and procedures in later engineering courses struggle because they have forgotten or been unable to conceptualise fully concepts, formulae and methods taught in earlier courses.

The main reason though why the interactive lecture is still so common is because it is one way to build some form of interaction into very large classes with 200 students or more. It should be noted though that even in interactive classes, it is unlikely that over the whole length of a thirteen week semester, more than ten per cent of the students will have the chance to ask a question or make a comment if the class size is large. (Research again has indicated that it tends to be the same ten or so students who always ask unprompted questions.)

Seminars and tutorials

Definitions:

seminar is a group meeting (either face-to-face or online) where a number of students participate at least as actively as the teacher, although the teacher may be responsible for the design of the group experience, such as choosing topics and assigning tasks to individual students.

tutorial is either a one-on-one session between a teacher and a student, or a very small group (five or less) of students and an instructor, where the learners are at least as active in discussion and presentation of ideas as the teacher.

Socrates and his student: Johann Friedrich Greuter, 1590: (San Francisco, Achenbach Foundation for Graphic Arts 

Socrates and his students: Painter: Johann Friedrich Greuter, 1590: (San Francisco, Achenbach Foundation for Graphic Arts)

Seminars can range from six or more students, up to 30 students in the same group. Because the general perception is that seminars work best when numbers are relatively small, they tend to be found more at graduate level or the last year of undergraduate programs, or anywhere where class size is around 25-30.

Seminars and tutorials again have a very long history, going back at least to the time of Socrates. Plato, the philosopher, was a student or follower of Socrates, although Socrates denied he was a teacher, rebelling against the idea common at that time in ancient Greece that ‘a teacher was a vessel that poured its contents into the cup of the student’. Instead, according to Plato, Socrates used dialogue and questioning ‘to help others recognize on their own what is real, true, and good.’ (Stanford Encyclopedia of Philosophy.) 

The format of seminars can vary a great deal. One common format, especially at graduate level, although similar practices can be found at the school/k-12 level, is for the teacher to set advance work for a selected number of students, and then have the selected students present their work to the whole group, for discussion, criticism and suggestions for improvement. Although there may be time for only two or three student presentations in each seminar, over a whole semester every student gets their turn. Another format is to ask all the students in a group to do some specified advanced reading or study, then for the teacher to introduce questions for general discussion within the seminar that requires students to draw on their earlier work.

Tutorials are a particular kind of seminar that are identified with Ivy League universities, and in particular Oxford or Cambridge. There may be as few as two students and a professor in a tutorial and the meeting often follows closely the Socratic method of the student presenting his or her findings and the professor rigorously questioning every assumption made by the student – and also drawing in the other student to the discussion.

Both these forms of dialogical learning can be found not only in classroom contexts, but also online. Online discussion forums go back to the 1970s, but really took off after the introduction of the WorldWide Web and high band telecommunications enabled the development of learning management systems, most of which now include an area for online discussions. These online discussion forums have some differences though with classroom seminars:

  • first,they are text based, not oral
  • second, they are asynchronous: participants can log in at any time, and from anywhere with an Internet connection, but this can cause some difficulties in following or participating in a particular argument or discussion
  • thus, third, many discussion forums allow for ‘threaded’ connections, enabling a response to be attached to the particular comment which prompted the response, rather than just displayed in chronological order. This allows for dynamic sub-topics to be developed, with sometimes more than ten responses within a single thread of discussion. This enables participants to follow multiple discussion topics over a period of time.

However, in general, the pedagogical similarities between online and face-to-face discussions are much greater than the differences. For academic and conceptual development, discussions need to be well organized by the teacher, and the teacher needs to provide the necessary support to enable the development of ideas and the construction of new knowledge for the students. There are several ways this can be done:

  • set clear goals for the discussions that are understood by the students, such as: ‘to explore gender and class issues in selected novels’ or ‘to compare and evaluate alternative methods of coding.’
  • set clear guidelines about expectations of students, such as ‘you should log in at least once a week to each discussion topic and make at least one substantive contribution to each topic each week.’
  • set clear, written codes of conduct for participating in discussions, and ensure that they are enforced
  • set topics for discussion that complement and expand issues in the study materials, and are relevant to answering assessment questions
  • provide the appropriate scaffolding or support, such as comments that help students develop their thinking around the topics, refer them back to study materials if necessary, or explain issues when students seem to be confused or misinformed
  • monitor the discussions to prevent them getting off topic or too personal
  • provide encouragement for those that are making real contributions to the discussion,  head off those that are trying to hog or dominate the discussions, and track those not participating, and help them to participate.

MOOCs

Massive, open, online courses usually include opportunities for discussion among students. The importance of discussion and the methods for organizing it, vary considerably within MOOCs. In instructionist MOOCs, based on video-recorded lectures, the discussion is usually ancillary, and is added to enable mainly clarification of concepts covered in the lectures. Because of the number of participants in instructionist MOOCs, it is unusual for the instructor responsible for the content of the MOOC to become heavily engaged in the discussions, although frequently teaching assistants may be asked to monitor the overall discussion and direct significant issues back to the main instructor for a general response.

In connectivist MOOCs, the interaction between participants is the core of the MOOC, and various methods and technologies are used to connect participants together. Thus hash tags may be used to enable participants to share in tweets from other participants, individuals may create their own blogs for their comments and reflections on the topics under discussion, with the blog urls being collected together and shared with other participants, or there may, less frequently, be a common discussion forum or area where all comments are posted. Even connectivist MOOCs though tend to have some form of loose central structure, with perhaps a variety of ‘experts’ being invited to start off conversations with some form of transmissive communication, such as a webcast or a reading, then the experts continue to participate in the following discussions.

These of course are two extremes, and as MOOCs develop, we are seeing some convergence, but for nearly all MOOCs, discussion between participants is seen as crucial for facilitating and developing learning. Nevertheless, there are some strong criticisms of the effectiveness of the discussion element of MOOCs for developing the high-level conceptual development required for academic learning. I have suggested that to develop deep, conceptual learning, there is a need in most cases for intervention by a subject expert, to clarify misunderstandings or misconceptions, to provide accurate feedback,  to ensure that the criteria for academic learning, such as use of evidence, clarity of argument, etc., are being met, and to ensure the necessary input and guidance to seek deeper understanding. Indeed, there has been a great deal of research into credit-based online courses that show instructor presence is a key factor in ensuring high completion rates for online courses. Firmin et al. (2014) have shown that when there is some form of instructor ‘encouragement and support of student effort and engagement’, results improve for all participants in MOOCs.

Furthermore, the more massive the course, the more likely participants are to feel ‘overload, anxiety and a sense of loss’, if there is not some instructor intervention or structure imposed (Knox, 2014). Without a structured role for subject experts, participants are faced with a wide variety of quality in terms of comments and feedback from other participants. There is again a great deal of research on the conditions necessary for the successful conduct of collaborative and co-operative group learning (see for instance, Dillenbourg, 1999, Lave and Wenger, 1991), and these findings certainly have not been applied to the management of MOOC discussions to date. (We will return to this topic in a later chapter.)

The counter argument is that MOOCs develop a new form of learning based on networking and collaboration that is essentially different from academic learning, and MOOCs are thus more appropriate to the needs of learners in a digital age. Adult participants in particular, it is claimed, have the ability to self-manage the development of high level conceptual learning.  MOOCs are ‘demand’ driven, meeting the interests of individual students who seek out others with similar interests and the necessary expertise to support them in their learning, and for many this interest may well not include the need for deep, conceptual learning but more likely the appropriate applications of such learning in specific contexts.

MOOCs do appear to work best for those who already have a high level of education and therefore bring many of the conceptual skills developed in formal education with them when they join a MOOC, and therefore contribute to helping those who come without such skills. Over time, as more experience is gained, MOOCs are likely to incorporate and adapt  for large numbers some of the findings from research on smaller group work. Indeed, MOOCs are likely to develop new ways to manage discussion effectively in very large groups. In the meantime, though, there is much work still to be done if MOOCs are to provide the support and structure needed to ensure deep, conceptual learning where this does not already exist in students.

Summary

For many faculty, the ideal teaching environment is Socrates sitting under the linden tree, with a small group of dedicated and interested students. Unfortunately, the reality of mass higher education makes this impossible for all but the most elite and expensive institutions. However, seminars for 25-30 students are not unrealistic, even in public undergraduate education. More importantly, seminar models enable the kind of teaching and learning that are most likely to facilitate the types of skills needed from our students in a digital age. Seminars are flexible enough to be offered in class or online, depending on the needs of the students. They are probably best used when students have done individual work before the seminar. Of upmost importance, though, is the ability of teachers to teach successfully in this manner, which requires different skills from transmissive lecturing.

We saw in Chapter 1 that although expansion of student numbers in higher education is part of the problem, it’s not the whole problem. Other factors, such as senior professors teaching less, and focusing mainly on graduate students, results in larger classes at undergraduate level, using transmissive lecturing. These classes are often taught by teaching assistants who have little more knowledge than the students they are teaching. And if more senior or experienced instructors switched from transmissive lectures, and instead required students to find and analyse content for themselves, this would free up more time for the instructors to spend on seminar-type teaching. So it as much an organizational issue, a matter of choice and priorities, as an economic issue. The more we can move towards a seminar approach to teaching and learning and away from large, transmissive lectures, the better, if we are to develop students with the skills needed in a digital age.

Over to you

Your feedback on this will be invaluable. In particular:

  • Do you agree that the kind of teaching conducted in seminar-type contexts is more appropriate for today’s learners than transmissive lectures? If so, why (or conversely, why not?)
  • is the description of the way dialogue and discussion operate to enhance learning accurate and if not, what should be changed?
  • are there important ways of teaching built around dialogue or discussion that have been missed and should be included?
  • do you agree with my comments about the current limitations of MOOCs for engendering the kind of discussions that lead to deep, conceptual learning? What could or do MOOCs do to help the development of such knowledge?
  • how realistic is it to move away from large, transmissive lecture classes to smaller, seminar-type teaching? What is preventing this from happening more often in our educational systems? Is it just a money issue, or are there other factors at work?

References

Asubel, D. et al. (1978) Educational Psychology: A Cognitive View New York: Holt, Reinhart and Winston

Dillenbourg, P. (ed.) (1999) Collaborative-learning: Cognitive and Computational Approaches. Oxford: Elsevier

Entwistle, N. and Peterson, E. (2004) Conceptions of Learning and Knowledge in Higher Education: Relationships with Study Behaviour and Influences of Learning Environments International Journal of Educational Research, Vol. 41. pp. 407-428

Firmin, R. et al. (2014) Case study: using MOOCs for conventional college coursework Distance Education, Vol. 35, No. 2

Harasim, L. (2012) Learning Theory and Online Technologies New York/London: Routledge

Knox, J. (2014) Digital culture clash: ‘massive’ education in the e-Learning and Digital Cultures Distance Education, Vol. 35, No. 2

Laurillard, D. (2001) Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies New York/London: Routledge

Lave, J. and Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press

Marton, F. and Saljö, R. (1997) Approaches to learning, in Marton, F., Hounsell, D. and Entwistle, N. (eds.) The experience of learning: Edinburgh: Scottish Academic Press (out of press, but available online)

Next up

Experiential learning (learning by doing): labs, field trips, apprenticeships and workplace/co-op training

Main lessons for developing skills for a digital age.

Guest blog: MOOCs: Disruptor or Indicator of Something Deeper?

Listen with webReader
Guest blogger: Nicole Christen

Guest blogger: Nicole Christen

Introduction

I don’t usually do guest blogs, and when I do it’s always because I know they will be of the highest quality – and I NEVER accept unrequested guest blogs from people I don’t know.

However, I was a participant in a study on MOOCs by Nicole Christen for a paper as part of her Master in Educational Technology program at the University of British Columbia. She kindly sent me a copy of her final paper. I was so struck by the quality of this paper and its significance that I immediately asked her if she would be willing to provide a summary in the form of a blog post. Here is the summary of her paper. I found no need to change it. I strongly recommend though that you read the paper in full, which is available here.

Nicole Christen

MOOCs: Disruptor or Indicator of Something Deeper?

Why have massive open online courses, known as MOOCs, established a stronghold in the educational marketplace? Are they responsible for disrupting the traditional system of higher education? And, how can post-secondary institutions survive the changes taking place?

In the summer of 2013, amidst the early hype surrounding MOOCs, I conducted a qualitative research project. My objective was to explore the motivations driving institutions to launch MOOCs and join MOOC consortiums. MOOCs have been labeled as a disruptive force to the traditional system of post-secondary education; however, my research argues otherwise. MOOCs, themselves, are not the source of disruption. Deeper forces are at work.

About My Research Project

In order to understand the reasons behind the rapid implementation of MOOCs by post-secondary institutions, I interviewed educational technology thought leaders from around the world whose areas of expertise included distance learning and open learning at the post-secondary level. During each 30 minute interview, I asked a series of questions designed to help me identify common underlying themes surrounding MOOCs and the overall concept of open learning. The themes extrapolated from my interview data provide a solid overview of fundamental shifts that have occurred as a result of the technological revolution and remain relevant regardless of any changes to MOOCs that have taken place since this research was conducted.

Forces Driving the MOOC Movement

Media hype that portrays MOOCs as an all-powerful disruptive force overlooks the underlying factors behind the adoption of MOOCs. In particular, the post-secondary marketplace is becoming increasingly driven by learner desires. Self-directed, distance education at the post-secondary level has existed for decades; however, the relative ease with which people around the world can now access the Internet, has created a tipping point. In many cases, learners are no longer as limited by geographical boundaries or technological limitations. Open learning initiatives, such as MOOCs, remove financial barriers as well. Instead, learners can (and do) go where their needs will best be met. The educational marketplace is becoming learner-driven.

Interpretations and Implications

Why, then, are MOOCs significant? Because MOOCs are a clear indicator that the realm of post-secondary education is changing as a result of advances in technology. The shift from a top-down, institution driven marketplace to one where a learner can use technology to create a  personalized, piecemeal learning experience from multiple institutions requires institutions to ask themselves what they offer learners that is unique. If one institution meets a unique need, and can fulfill this need on a mass scale for learners better than any other institution, then other institutions need to find a different competitive edge.

Furthermore, if MOOCs become a viable educational option (viable in the sense that employers begin to value emerging credentialing systems created by MOOC providers), then there is a real risk that MOOCs will encroach upon the territory of undergraduate education. Post-secondary institutions rely on heavy enrollment of first and second year students to fund their operations and programs. Losing first and second year students to MOOCs will be detrimental to any institutions.

With that said, according to many of the people I interviewed, there will always a be place for research universities and Ivy league schools. These research-based schools fulfill a market need for an element of prestige attached to credentials, networking opportunities with leaders in the field of study, and the opportunity to conduct innovative research. The institutions most at risk of losing students to online and open learning initiatives are those that simply disseminate information generated elsewhere (typically from prestigious research-based institutions).

Given the potential impact of MOOCs, they can certainly be classified as disruptive; however, they are not a disruptor. The shift toward a learner-directed marketplace, widespread access to high-speed Internet, and the ever-increasing global network of information are the true disruptive forces. If MOOCs had not emerged, then some other form of open learning would have emerged to meet the need for low-cost access to educational resources.

Additionally, MOOCs may not be a lasting phenomenon, especially because a sustainable model for operation has yet to be proven; however, if their popularity fades, another innovative open learning opportunity will arise. Things will not go back to the way they were. The demand for open learning will not disappear.

How can institutions survive the disruption taking place in post-secondary education?

My hope is that my research can provide a starting point for institutions to explore the ways in which they can withstand the changes taking place within post-secondary learning by exploring new niches to fill and discovering which specific learner needs they are best equipped to meet. For example, open learning programs (such as MOOCs) often provide information in a way that can be considered akin to a free, interactive textbook. Certain institutions can build on MOOCs by providing classes that help students understand the material being presented to them. In essence, the institutional programs would complement MOOCs.  The most important take-away from my research is that the conditions which have lead to the rise of MOOCs have also created new gaps in the educational marketplace, opening the door for many other innovative approaches to adult education.

My formal research report is titled Open Online Learning: This Changes Everything and can be found at http://nicolechristen.com/wp-content/uploads/2014/03/Open-Online-Learning.pdf

Bio: Nicole Christen is a digital media strategist and a recent graduate from the Master of Educational Technology program at UBC. Read more about Nicole’s professional background and areas of interest at www.nicolechristen.com/portfolio.

A balanced research report on the hopes and realities of MOOCs

Listen with webReader

Columbia MOOCs 2

Hollands, F. and Tirthali, D. (2014) MOOCs: Expectations and Reality New York: Columbia University Teachers’ College, Center for Benefit-Cost Studies of Education, 211 pp

We are now beginning to see a number of new research publications on MOOCs. The journal Distance Education will be publishing a series of research articles on MOOCs in June, but now Hollands and Tirthali have produced a comprehensive research analysis of MOOCs.

What the study is about

We have been watching for evidence that MOOCs are cost-effective in producing desirable educational outcomes compared to face-to-face experiences or other online interventions. While the MOOC phenomenon is not mature enough to afford conclusions on the question of long-term cost-effectiveness, this study serves as an exploration of the goals of institutions creating or adopting MOOCs and how these institutions define effectiveness of their MOOC initiatives. We assess the current evidence regarding whether and how these goals are being achieved and at what cost, and we review expectations regarding the role of MOOCs in education over the next five years. 

The authors used interviews with over 80 individuals covering 62 institutions ‘active in the MOOCspace’, cost analysis, and analysis of other research on MOOCs to support their findings. They identified six goals from the 29 institutions in the study that offered MOOCs, with following analysis of success or otherwise in accomplishing such goals:

1. Extending reach (65% 0f the 29 institutions)

Data from MOOC platforms indicate that MOOCs are providing educational opportunities to millions of individuals across the world. However, most MOOC participants are already well-educated and employed, and only a small fraction of them fully engages with the courses. Overall, the evidence suggests that MOOCs are currently falling far short of “democratizing” education and may, for now, be doing more to increase gaps in access to education than to diminish them. 

2. Building and maintaining brand (41%)

While many institutions have received significant media attention as a result of their MOOC activities, isolating and measuring impact of any new initiative on brand is a difficult exercise. Most institutions are only just beginning to think about how to capture and quantify branding-related benefits.

3. Reducing costs or increasing revenues (38%)

….revenue streams for MOOCs are slowly materializing but we do not expect the costs of MOOC production to fall significantly given the highly labor-intensive nature of the process. While these costs may be amortized across multiple uses and multiple years, they will still be additive costs to the institutions creating MOOCs. Free, non-credit bearing MOOCs are likely to remain available only from the wealthiest institutions that can subsidize the costs from other sources of funds. For most institutions, ongoing participation in the current MOOC experimentation will be unaffordable unless they can offer credentials of economic value to attract fee-paying participants, or can use MOOCs to replace traditional offerings more efficiently, most likely by reducing expensive personnel. 

4. Improving educational outcomes (38%)

for the most part, actual impact on educational outcomes has not been documented in any rigorous fashion. Consequently, in most cases, it is unclear whether the goal of improving educational outcomes has been achieved . However, there were two exceptions, providing evidence of improvement in student performance as a result of adopting MOOC strategies in on-campus courses

5. Innovation in teaching and learning (38%)

It is abundantly clear that MOOCs have prompted many institutions and faculty members to engage in new educational activities. The strategies employed online such as frequent assessments and short lectures interspersed with questions are being taken back on-campus. It is less clear what has been gained by these new initiatives because the value of innovation is hard to measure unless it can be tied to a further, more tangible objective. We …. conclude that most institutions are not yet making any rigorous attempt to assess whether MOOCs are more or less effective than other strategies to achieve these goals. 

6. Research on teaching and learning (28%)

A great deal of effort is being expended on trying to improve participant engagement and completion of MOOCs and less effort on determining whether participants actually gain skills or knowledge from the courses ….While the potential for MOOCs to contribute significantly to the development of personalized and adaptive learning is high, the reality is far from being achieved. 

Cost analysis

The report investigates the costs of developing MOOCs compared to those for credit-based online courses, but found wide variations and lack of reliable data.

Conclusions from the report

The authors came to the following conclusions:

1. there is no doubt that online and hybrid learning is here to stay and that MOOCs have catalyzed a shift in stance by some of the most strongly branded institutions in the United States and abroad.

2. MOOCs could potentially affect higher education in more revolutionary ways by:

  • offering participants credentials of economic value

  • catalyzing the development of true adaptive learning experiences

However, either of these developments face substantial barriers and will require major changes in the status quo.

My comments on the report

First this is an excellent, comprehensive and thoughtful analysis of the expectations and realities of MOOCs. It is balanced, but where necessary critical of the unjustified claims often made about MOOCs. This report should be required reading for anyone contemplating offering MOOCs.

Different people will take away different conclusions from this report, as one would expect from a balanced study. From my perspective, though, it has done little to change my views about MOOCs. MOOC providers to date have made little effort to identify the actual learning that takes place. It seems to be enough for many MOOC proponents to just offer a course, on the assumption that if people participate they will learn.

Nevertheless, MOOCs are evolving. Some of the best practices that have been used in credit-based online courses are now being gradually adopted as more MOOC players enter the market with experience of credit-based online learning. MOOCs will eventually occupy a small but important niche as an alternative form of non-formal, continuing and open education. They have proved valuable in making online learning more acceptable within traditional institutions that have resisted online learning previously. But no-one should fear them as a threat to credit-based education, either campus-based or online.

Research from the Michigan Virtual University on a connectivist MOOC

Listen with webReader

 MVU MOOC report

Ferdig, R. et al. (2014) Findings and reflections from the ‘K-12 Teaching in the 21st Century’ MOOC Lansing MI: Michigan Virtual Learning Research Institute

We are now beginning to get some in-depth research or evaluations of MOOCs. This one is from a team at Kent State University that developed a five week ‘connectivist’ MOOC aimed principally at three distinct audiences: high school students interested in becoming teachers, preservice teachers, and inservice teachers in the K-12 system.

I provide here a very brief summary of the report (as always, you should read the report for yourself if my summary gets you interested). Italics are direct quotes from the report.

Goal of the MOOC

How can we get teachers to think more deeply about reinventing education?

MOOC design

facilitators take on the role of connecting people around an idea for the purpose of bettering our understanding of the
idea. A connectivist-based MOOC draws on the extensive number of participants as well as the existing open repository of content to develop an experience. Participants are both teachers and learners in a process – not a product.

The course was designed around four principles often associated with teaching in the 21st century: connected learning, personalization, collaboration, and reflection.

Core technology

Coursesites by Blackboard provided the basic platform for content and discussion, supplemented by the use of participants’ social media networks and technologies. In addition participants were asked to create an ‘artifact’ to represent their learning.

Use of partners/co-facilitators

Kent State provided core facilitators for the MOOC, but they also invited other co-facilitators from schools, colleges and universities both in Michigan and from several other states.

Qualifications

Badges and continuing education units were given for successful participation.

Main results

Participants (data at time of enrollment, i.e. all participants)

Start of course: 673; end of course: 848; mainly from Michigan and surrounding states, although 12 were international

School teachers: 42%; k-12 students: 23%; post-secondary students: 16%; 19% other (inc. school administrators, university faculty); 80% female.

Participants’ response to the MOOC (168 participants who completed a post-course survey)

Most participants who responded enjoyed the MOOC, with in-service teachers enjoying it the most. Th main criticism (especially from the k-12 students) was the amount of work involved in following the MOOC.

Very active participation in the online discussion forums (within the Coursesites LMS)

There were over 6,000 actual posts (comments) and over 65,000 ‘hits’/looks over a five week period, from just over 300 of the participants – but almost to-thirds did not participate at all.

Types of participation

Lurkers (i.e. did not participate in LMS discussion forums – they may have participated through social media): 63%. There were accounts created in Facebook, Twitter, Delicious and blogs related to the course which indicated active social media connections both for registered participants and with those who had not registered for the course but were interested. However, these numbers were relatively small, and hard to measure.

Passive participation was defined as doing the minimum amount of work required to complete the course. Some of the passive participants were K-12 students forced to complete the MOOC for a class requirement.

There were also preservice teachers and inservice teachers who could be described as passive participants. These participants often completed the course; however, much like the high school students, their posts were limited to one or two sentences per posts. Their comments were also superficial, for example, “Nice job” or “I like what you did.”

Active participants participated in four ways:

  • informing personal practice
  • sharing the MOOC with their communities
  • leadership within the MOOC community
  • critical colleagues

The authors’ main conclusions

The seeking and sharing of digital media highlights that people want to form and engage in communities, and the growing interest in MOOCs shows this is true of educational communities as well….

Learning takes place in communities; depending on the implementation, technology has the capability to create and sustain the communities’ learning and practice….. Evidence in this report suggests that such activities can lead to positive outcomes, particularly as they relate to getting teachers to think more deeply about teaching and learning in the 21st century.

My comments

Even though (or perhaps because) this is a self-evaluation, this is a very useful report. I was fascinated for instance that this course ended with more participants than when it started, due to the ‘publicity’ of social media connections during the course itself.  It was interesting too that some of the participants in this MOOC were not necessarily willing participants – being forced to participate as part of a formal credit program. This seems to me to go against the whole purpose of a connectivist MOOC.

More importantly for me, the report highlights some of the ways research can be conducted on MOOCs and also some of the challenges. The study identifies the importance, from a research perspective, of having some kind of platform that can gather student data and track student behaviour, such as levels or types of participation. However, given the importance of social media for connectivist MOOCs, some way of accurately tracking related social media activity is critical. It seems to me that this is a problem that appropriate software could solve (further development of gRRShopper?), although privacy issues would need to be addressed as well. (Perhaps the spy agencies can help here – just joking!)

I agree completely with the authors when they write:

Researchers have already provided ample evidence that asking if a technology works is the wrong question. A more appropriate question is: under what conditions do certain types of MOOCs work?

Another even more pertinent question is: What prior research into credit-based online learning applies – and what does not apply – to different kinds of MOOCs? This might save a lot of time re-inventing the wheel, particularly for xMOOCs. I am getting sick of hearing from research on xMOOCs that immediate feedback helps retention – we have known that for nearly 100 years. We do need though for instance to assess the importance and most useful roles, if any, of instructors/facilitators/subject matter experts in MOOCs, and whether MOOCs can succeed with reduced ‘expert’ participation. This report suggests almost the opposite – connectivist MOOCs work best with a wide range of facilitators – but what are the hidden costs of this?

Finally, I also agree with the authors that completion rates are not the best measure of success for MOOCs. This MOOC does seem to have raised some interesting questions for participants. I’m just curious about their answers. Despite the very good work done by the instructors/researchers of this MOOC, I am still left with the question: what did the participants actually learn from this MOOC? For instance, what would an analysis of the student ‘artifacts’ have told us about their learning? Unless we try to answers questions about what actual learning took place then it will remain difficult if not impossible to measure the true value of different kinds of MOOC, and I think that would be a pity.

In the meantime, this report is definitely recommended reading for anyone interested in doing research on or evaluating MOOCs.