April 23, 2014

Research from the Michigan Virtual University on a connectivist MOOC

Listen with webReader

 MVU MOOC report

Ferdig, R. et al. (2014) Findings and reflections from the ‘K-12 Teaching in the 21st Century’ MOOC Lansing MI: Michigan Virtual Learning Research Institute

We are now beginning to get some in-depth research or evaluations of MOOCs. This one is from a team at Kent State University that developed a five week ‘connectivist’ MOOC aimed principally at three distinct audiences: high school students interested in becoming teachers, preservice teachers, and inservice teachers in the K-12 system.

I provide here a very brief summary of the report (as always, you should read the report for yourself if my summary gets you interested). Italics are direct quotes from the report.

Goal of the MOOC

How can we get teachers to think more deeply about reinventing education?

MOOC design

facilitators take on the role of connecting people around an idea for the purpose of bettering our understanding of the
idea. A connectivist-based MOOC draws on the extensive number of participants as well as the existing open repository of content to develop an experience. Participants are both teachers and learners in a process – not a product.

The course was designed around four principles often associated with teaching in the 21st century: connected learning, personalization, collaboration, and reflection.

Core technology

Coursesites by Blackboard provided the basic platform for content and discussion, supplemented by the use of participants’ social media networks and technologies. In addition participants were asked to create an ‘artifact’ to represent their learning.

Use of partners/co-facilitators

Kent State provided core facilitators for the MOOC, but they also invited other co-facilitators from schools, colleges and universities both in Michigan and from several other states.

Qualifications

Badges and continuing education units were given for successful participation.

Main results

Participants (data at time of enrollment, i.e. all participants)

Start of course: 673; end of course: 848; mainly from Michigan and surrounding states, although 12 were international

School teachers: 42%; k-12 students: 23%; post-secondary students: 16%; 19% other (inc. school administrators, university faculty); 80% female.

Participants’ response to the MOOC (168 participants who completed a post-course survey)

Most participants who responded enjoyed the MOOC, with in-service teachers enjoying it the most. Th main criticism (especially from the k-12 students) was the amount of work involved in following the MOOC.

Very active participation in the online discussion forums (within the Coursesites LMS)

There were over 6,000 actual posts (comments) and over 65,000 ‘hits’/looks over a five week period, from just over 300 of the participants – but almost to-thirds did not participate at all.

Types of participation

Lurkers (i.e. did not participate in LMS discussion forums – they may have participated through social media): 63%. There were accounts created in Facebook, Twitter, Delicious and blogs related to the course which indicated active social media connections both for registered participants and with those who had not registered for the course but were interested. However, these numbers were relatively small, and hard to measure.

Passive participation was defined as doing the minimum amount of work required to complete the course. Some of the passive participants were K-12 students forced to complete the MOOC for a class requirement.

There were also preservice teachers and inservice teachers who could be described as passive participants. These participants often completed the course; however, much like the high school students, their posts were limited to one or two sentences per posts. Their comments were also superficial, for example, “Nice job” or “I like what you did.”

Active participants participated in four ways:

  • informing personal practice
  • sharing the MOOC with their communities
  • leadership within the MOOC community
  • critical colleagues

The authors’ main conclusions

The seeking and sharing of digital media highlights that people want to form and engage in communities, and the growing interest in MOOCs shows this is true of educational communities as well….

Learning takes place in communities; depending on the implementation, technology has the capability to create and sustain the communities’ learning and practice….. Evidence in this report suggests that such activities can lead to positive outcomes, particularly as they relate to getting teachers to think more deeply about teaching and learning in the 21st century.

My comments

Even though (or perhaps because) this is a self-evaluation, this is a very useful report. I was fascinated for instance that this course ended with more participants than when it started, due to the ‘publicity’ of social media connections during the course itself.  It was interesting too that some of the participants in this MOOC were not necessarily willing participants – being forced to participate as part of a formal credit program. This seems to me to go against the whole purpose of a connectivist MOOC.

More importantly for me, the report highlights some of the ways research can be conducted on MOOCs and also some of the challenges. The study identifies the importance, from a research perspective, of having some kind of platform that can gather student data and track student behaviour, such as levels or types of participation. However, given the importance of social media for connectivist MOOCs, some way of accurately tracking related social media activity is critical. It seems to me that this is a problem that appropriate software could solve (further development of gRRShopper?), although privacy issues would need to be addressed as well. (Perhaps the spy agencies can help here – just joking!)

I agree completely with the authors when they write:

Researchers have already provided ample evidence that asking if a technology works is the wrong question. A more appropriate question is: under what conditions do certain types of MOOCs work?

Another even more pertinent question is: What prior research into credit-based online learning applies – and what does not apply – to different kinds of MOOCs? This might save a lot of time re-inventing the wheel, particularly for xMOOCs. I am getting sick of hearing from research on xMOOCs that immediate feedback helps retention – we have known that for nearly 100 years. We do need though for instance to assess the importance and most useful roles, if any, of instructors/facilitators/subject matter experts in MOOCs, and whether MOOCs can succeed with reduced ‘expert’ participation. This report suggests almost the opposite – connectivist MOOCs work best with a wide range of facilitators – but what are the hidden costs of this?

Finally, I also agree with the authors that completion rates are not the best measure of success for MOOCs. This MOOC does seem to have raised some interesting questions for participants. I’m just curious about their answers. Despite the very good work done by the instructors/researchers of this MOOC, I am still left with the question: what did the participants actually learn from this MOOC? For instance, what would an analysis of the student ‘artifacts’ have told us about their learning? Unless we try to answers questions about what actual learning took place then it will remain difficult if not impossible to measure the true value of different kinds of MOOC, and I think that would be a pity.

In the meantime, this report is definitely recommended reading for anyone interested in doing research on or evaluating MOOCs.

A comprehensive review of the literature on digital natives

Listen with webReader

Image retrieved from Hastac.org (Doug Beg's blog)

Image retrieved from Hastac.org (Stephen Berg’s blog)

Jones, C. and Shao, B. (2011) The Net Generation and Digital Natives: Implications for Higher Education Milton Keynes: Open University/Higher Education Academy

This paper is required reading for graduate students studying online learning or educational technology. The paper is little old (by Internet standards) but I just came across it looking for something else.

The discussion about ‘digital natives’ has gone quiet recently, and this paper might be one reason why. The authors have made a thorough review of the literature on this topic, with over 200 appropriate references, including surveys of relevant publications from countries in Europe, Asia, North America, Australia and South Africa. Here are some of their main conclusions, although the report is best read in full:

  • there is no evidence that there is a single new generation of young students entering Higher Education and the terms Net Generation and Digital Native do not capture the processes of change that are taking place;
  • demographic factors interact with age to pattern students’ responses to new technologies;
  • the gap between students and their teachers is not fixed, nor is the gulf so large that it cannot be bridged. In many ways the relationship is determined by the requirements teachers place upon their students to make use of new technologies and the way teachers integrate new technologies in their courses. There is little evidence that students enter university with demands for new technologies that teachers and universities cannot meet;
  • students do not naturally make extensive use of many of the most discussed new technologies such as Blogs, Wikis and 3D Virtual Worlds….Students who are required to use these technologies in their courses are unlikely to reject them and low use does not imply that they are inappropriate for educational use. The key point being made is that there is not a natural demand amongst students that teaching staff and universities should feel obliged to satisfy;
  • students will respond positively to changes in teaching and learning strategies that are well conceived, well explained and properly embedded in courses and degree programmes. However there is no evidence of a pent-up demand amongst students for changes in pedagogy or of a demand for greater collaboration;
  • the development of university infrastructures, such as new kinds of learning environments (for example Personal Learning Environments) should be choices about the kinds of provision that the university wishes to make and not a response to general statements about what a new generation of students are demanding; 
  • the evidence indicates that young students do not form a generational cohort and they do not express consistent or generationally organised demands. A key finding of this review is that political choices should be made explicit and not disguised by arguments about generational change.

Comment

This paper is a timely correction to the hype around digital natives, especially the claims made by Tapscott and Prensky. It is so easy to find a buzz-word or phrase and through constant repetition and media hype present a gross over-simplification of what are often subtle and complex changes.

It is also important to pay attention to what Jones and Shao are not saying. They are not saying that social media, personal learning environments, or collaborative learning are inappropriate, nor that the needs of students and the workforce are unchanging or unimportant, but the use of these tools or approaches should be driven by a holistic look at the needs of all students, the subject area and society, and not by an erroneous view of what a particular generation of students are demanding.

 

A review of a Harvard/MIT research paper on edX MOOCs

Listen with webReader

 edX graphic

Ho, A. et al. (2014) HarvardX and MITx: The First Year of Open Online Courses Fall 2012-Summer 2013 (HarvardX and MITx Working Paper No. 1), January 21

This 32 page report provides a range of data and statistics about the first 17 MOOCs offered through edX by MIT and Harvard.

Methodology

MOOCs raise a number of interesting challenges when doing research, such as measuring participation, and defining success. In any interpretation of the results, these methodological challenges need to be considered. The researchers identified the following challenges:

1. Post-hoc research. The research design was established after the courses were designed and delivered, so data on some key critical research variables (e.g., socio-economic status) were not available or collected.

2. Variation in the object of the research. Although limited to MOOCs offered on the edX platform, the 17 MOOCs varied considerably in educational objectives, style, length, types of learner and other factors.

3. Measuring levels of participation. Participants varied from those who logged in only once to those that completed a certificate (and then some who went on to take more MOOCs). As a result, the researchers came up with four mutually exclusive categories of participation:

  • Only Registered: Registrants who never access the courseware.
  • Only Viewed: Non-certified registrants who access the courseware, accessing less than half of the available chapters.
  • Only Explored: Non-certified Registrants who access more than half of the available chapters in the courseware.
  • Certified: Registrants who earn a certificate in the course.

4. Percentages are misleading when numbers are large. This was a new one for me. I know one should never use percentages when n <20, specially when generalizing beyond the sample, but in this instance, the researchers argue that small percentages (e.g. <5%) are also misleading when the number the percentage refers to can be very large, e.g. when 3% = 1,400 students who completed a certificate. In such cases, the absolute numbers matter more than the percentage, so the researchers claim.

5. Measures of success The researchers argue that traditional measures of academic success, such as the percentage of those who successfully complete a course, are not valid (the word used is ‘counter-productive’) for open online courses.

Main results

Participation

  • 17 MOOCs
  • 841,687 course registrations: average per MOOC: 51,263
  • 597,692 ‘persons’: average of 1.4 MOOCs per person
  • 292,852 (35%) never engaged with the content (“Only registered”)
  • 469,702 (56%) viewed (i.e. clicked on a module) less than half of the content (“Only viewed”)
  • 35,937 (4%) explored more than half the content, but did not get a certificate (average per MOOC: 2,114)
  • 43,196  (5%) earned certificates (average per MOOC: 2,540)

Participants

  • 234,463 (33%) report a high school education or lower
  • 66% of all participants, and 74% of all who obtained a certificate, have a bachelor’s degree or above
  • 213,672 (29%) of all participants, and 33% of all who obtained a certificate, are female
  • 26 was the median age, with 45,844 (6%) over 50 years of age
  • 20,745 (3%) of all participants were from the UN listed least developed countries
  • there are ‘considerable differences in …. demographics such as gender, age… across courses.”

Comments

First, congratulations to Harvard and MIT for not only doing this research on MOOCs, but also for making it openly available and releasing it early.

Second, I agree that percentages can be misleading, a focus on certification is not the best way to assess the value of a MOOC, and that absolute figures matter for assessing the value of MOOCs. However, this is NOT the way most commentators and the media have focused on MOOCs. Percentages and certification DO matter if MOOCs are being seen as a substitute or a replacement for formal education. MOOCs need to be judged for what they are, a somewhat unique – and valuable – form of non-formal education.

Third, if we do look at absolute numbers, they are in my view not that impressive – an average of 2,540 per course earning a certificate, and less than 5,000 per course following more than half the content. The Open University, with completely open access, was getting higher numbers of students completing credit-based foundation courses when it started. The History Channel (a cable TV channel in North America) does a lot better, in terms of numbers. We have already seen overall average numbers for MOOCs dropping considerably as they have become more common. So when we account for the Hawthorne effect, the results are certainly not startling.

Fourth, these results so much reminded me of the research on educational broadcasting 30 years ago (for more details, see footnote). If you substituted ‘MOOC’ for ‘educational television’, the results would be almost identical (except there was a higher proportion of women than men participating). Perhaps they should read my very old book, “Broadcasting in Education: An Evaluation.” (I still have a few copies in a cupboard somewhere).

Lastly, this brings me to my final point. Where is the reference to relevant previous research or theory (see, for instance the footnote to this post)? There are certainly unique aspects to MOOCs that deserve to be researched. However, while MOOCs may be new, non-formal learning is not, nor is credit-based online learning, nor is open education, nor is educational broadcasting, of which MOOCs are a new format. Much of what we already know about these areas also applies to some aspects of MOOCs. Once again, though, Harvard and MIT seem to live in an environment that pays no attention to what happens outside their cocoon. If it’s not theirs, it doesn’t count. This is simply not good enough. In no other field would you get away with ignoring all previous research or work in related areas such as credit-based online learning, open education or educational broadcasting.

Having got that off my chest, I did find the paper well written and interesting and certainly worth a careful read. I look forward to reading – and reviewing – future papers.

Footnote: MOOCs and the onion theory of educational broadcasting

I eventually found a copy of my book. I blew the dust off it and guess what I found.

Here’s what I wrote about ‘levels of commitment’ in non-formal educational broadcasting in 1984 (p.99):

At the centre of the onion is a small core of fully committed students who work through the whole course, and, where available, take an end-of-course assessment or examination. Around the small core will be a rather larger layer of students who do not take any examination but do enrol with a local class or correspondence school. There may be an even larger layer of students who, as well as watching and listening, also buy the accompanying textbook, but who do not enrol in any courses. Then, by far the largest group, are those that just watch or listen to the programmes. Even within this last group, there will be considerable variations, from those who watch or listen fairly regularly, to those, again a much larger number, who watch or listen to just one programme.

Now compare this to Figure 2 (p.13) of the Harvard/MIT report:

MOOC onionI also wrote (p.100):

A sceptic may say that the only ones who can be said to have learned effectively are the tiny minority that worked right through the course and successfully took the final assessment…A counter argument would be that broadcasting can be considered successful if it merely attracts viewers or listeners who might otherwise have shown no interest in the topic; it is the numbers exposed to the material that matter…the key issue then is whether broadcasting does attract to education those who would not otherwise have been interested, or merely provides yet another opportunity for those who are already well educated…There is a good deal of evidence that it is still the better educated in Britain and Europe that make the most use of non-formal educational broadcasting.

Thanks for the validation of my 1984 theory, Harvard/MIT.

Reference

Bates, A. (1984) Broadcasting in Education: An Evaluation. London: Constable

 

MOOCs, Norway, and the ecology of digital learning

Listen with webReader

© Ron Niebrugge MOOCs are just one species in the online digital forest - and a big and clumsy one

© Ron Niebrugge, 2013 MOOCs are just one species in the online digital forest – and a big and clumsy one

Earlier this week I was in Washington DC, at a conference called Transatlantic Science Week, aimed at promoting collaboration between research, innovation and educational institutions and organizations in the U.S.A, Canada and Norway. The main themes for the conference were International Security, Safety and Emergency Preparedness, Cyber Security and Education Policy/Education Research. (No prizes for guessing which theme I was invited to contribute to, although I have to say the others looked more interesting.) This year’s conference was organized by the Norwegian Embassy in Washington, with guidance from the Research Council of Norway.

The focus in the education theme is the digitalization of higher education, although in practice the focus is almost entirely on MOOCs. One reason for this is that Norway has set up a public commission ‘to inquire into the possibilities and challenges that accompany the development of MOOCs and similar offers.‘ The conference provides an interesting way to examine the current thinking on MOOCs of policy makers both in the U.S.A. and in Norway.

Because this is a rather long post, I’m posting the conclusions first, with more details about the conference which I hope will justify my conclusions.

Main take-aways

The value of this conference is that it brought together politicians, policy wonks, researchers and educational practitioners to share ideas and experiences. In particular, it gave me an insight into why MOOCs have resonated so much with policy-makers and others who are not embedded within the higher education system. It was clear from questions and discussions outside the sessions that policy makers remain convinced that MOOCs do offer the possibilities of lowering the cost of post-secondary education. At the same time, I find myself at conferences about MOOCs like the small boy running around shouting ‘The emperor has no clothes’ – and about as effectively.

So here are my main take-aways from the conference.

  • Houston, we have a problem – ‘we’ being universities and colleges. Publicly funded post-secondary institutions are perceived by important policy-makers as being unnecessarily expensive and perhaps even more importantly, not adapting fast enough to meet the demands of the 21st century
  • as a result, politicians and policy-makers are only too willing to grasp at anything that might disrupt the perceived complacency within the system. MOOCs fit this requirement to perfection
  • there is a growing tendency to conflate MOOCs with online learning in general. This suits of course the elite universities who have come 20 years late to the party – they are re-defining online learning according to their own interests
  • even re-designing a large class in a highly selective institution is now considered to be a MOOC, so as well as the conflation with online learning, MOOCs are now being equated with any large class delivered online. The concept of open-ness runs the risk of being lost, with the focus switching to free or cheap
  • if they can get past the hubris, Ivy League universities have a lot to offer online learning. There were several examples in the conference of innovative approaches to online learning from some of the top universities in the USA, but they weren’t MOOCs as most of us would understand the term. We need to bridge the gap between the Ivy League newcomers and those who have been working in online learning elsewhere. We will all benefit from this
  • as a profession we have failed miserably to disseminate best practices in online learning to busy practitioners/instructors. This is not entirely our fault. If there is no requirement for pre-service training to teach in a university, there is no opportunity to bring these best practices to the attention of all faculty. Training new faculty in modern teaching methods, including online learning, based on good pedagogy and cognitive science, is the best way to address the perception that universities and colleges are failing to adapt to the 21st century.

Above all, universities need to be more cost-effective, and if they aren’t, they are going to have methods forced on them that may not have the best outcomes, either for the institutions, or for the rest of us. How each country responds to MOOCs could well define which countries will end up more equal than others, and which will succeed or fail economically and socially in the latter part of this century.

The digital democratization of universities

This was the topic of the first parallel session in the education theme . The first speaker was Norway’s new Minister of Education and Research, Torbjørn Røe Isaksen, who has been in the job for less than one month, and is the youngest member of the cabinet, at 35. In his speech he demonstrated that he was well briefed on MOOCs and their potential, and is therefore looking forward to the Commission’s report (even though the commission was set up by the previous government). He raised some thoughtful questions about MOOCs, which makes me think he is keeping an open mind on the issue, in the best sense of the word.

The Rector of the University of Bergen gave a straightforward talk about the pros and cons of MOOCs, which would come as no surprise for anyone familiar with MOOCs, but was essential for providing a common understanding among all participants. There was the usual American hyperbole about MOOCs from no less than a representative from the American Science Foundation, e.g. ‘the important thing about MOOCs is they allow for the quantifiable measurement of learning on a massive scale‘. This from an electrical engineer, the world experts on educational measurement. I’m sorry, but qualitative assessment is not ‘bad’ but essential in many areas of higher education. There is more than one epistemology.

The most interesting presentation in this session came from Cathy Sandeen, the VP, Education Attainment and Innovation, at the American Council of Education. She reported that currently 18-24 year olds constitute less than 25% of all post-secondary students in the USA. Students aged 24-34 constitute 65% of all students now, most of whom are working at least part-time, and many of whom have children. Even more importantly, the U.S.A. participation rate in post-secondary education is now only 42%, putting it in the bottom quartile of OECD countries, whereas 20 years ago it was top. To catch up, it would need to add another one million places. She ended with a brief account of ACE’s efforts at accrediting MOOCs (for my take on this, see an earlier post.)

In short, I didn’t hear anything in this section that suggested that MOOCs or online learning were doing anything to ‘democratize’ higher education – they may be, but no evidence came out of this session.

MOOCs and the re-inventing of higher education

I was on a great panel, with Chris Dede, from the Harvard Graduate School of Education, and one of the best researchers in educational technology, taking the lead. The other panelists were Berit Kjelstadt, the chair of Norway’s public commission on MOOCs, and Wendy Newstetter, a cognitive scientist/engineer from Georgia Institute of Technology. Wendy got our panel organized. The whole day to date had been wall-to-wall presentations with no time for questions even, so we organized the session with plenty of time for questions and discussion.

Chris Dede was the lead speaker. He argued that high quality teaching required at least three conditions: cognitive knowledge, situated learning (learning embedded in context), and a community of learners (social learning). He pointed out that most MOOCs were able to scale only the cognitive knowledge element effectively (as he put it ‘old wine in new bottles’). He argued that virtual immersive environments or virtual worlds allowed for the other two elements to be scaled, and showed a video of EcoMUVE, a virtual reality eco pond,  a multi-user virtual environment in which students are immersed in a virtual environment and interact with avatar-based identities to investigate an ecosystem. More can be found on this approach here.

In my presentation I thought it important to place MOOCs within the broader framework of online learning, because it was clear that many participants seemed to be equating MOOCs with all online learning. I briefly summarized what was happening in credit-based online learning (high completion rates when best practice is used), hybrid learning, mobile learning, OERs, virtual worlds, remote labs, social media and learning, etc., and then argued that xMOOCs were re-inventing the wheel, and so far the wheel is square. Conclusion: MOOCs are just one species in the online learning forest, and a big and clumsy one at that.

More provocatively, I also argued that xMOOCs are more likely to increase inequality, by undermining publicly funded education, leaving an elite of campus-based universities for the very rich, resulting in high paid knowledge-worker employment for them, and massive information transmission delivered to the rest, who will be confined to low-wage service jobs because of their lack of high-level critical thinking skills. (For a copy of the slides, send me an e-mail (tony.bates@ubc.ca) and I will send an invitation to download them via Dropbox).

Berit Kjelstadt gave a brief summary of her commission’s mandate, then Wendy responded to the three presentations, with a particular emphasis on the need for problem-based approaches to education, particularly in science and engineering. The following questions and responses were lively, with a focus on the high costs of post-secondary education, and whether MOOCs will be a means by which to drive down costs.

The Gatsby curve - will MOOCs increase or reduce inequality? (© Globe and Mail, 2013)

The Gatsby curve – will MOOCs increase or reduce inequality? (© Globe and Mail, 2013)

Is blended learning the future in academia?

The program framed ‘blended learning’ as follows: ‘MOOCs provide an opportunity to …combine different learning practices, for instance, classroom instruction in Oslo, supplemented with streamed lectures from Stanford and online interaction with other students on and off campus.’ However, the speakers in this session didn’t quite see it this way.

Glynda Hull, of the University of California, Berkeley, described a really neat multimedia platform for collaborative learning designed and developed at UC Berkeley, linked to a Canvas LMS. This enables students to create and share multimedia objects and work collaboratively on projects. This looked a nice software development based on sound educational principles (although I suspect the same could be done, perhaps less elegantly, with a combination of WordPress, Mahara and Moodle), and the presentation was marred only by the usual hubris from faculty from elite universities and their re-writing of online learning history (‘Online learning to date has failed to enable effective collaborative learning…‘. Roll over Turoff, Hiltz, Scardamalia, Harasim, Pratt and Paloff, Salmon, etc. – and of course cMOOCs never existed. I was too transfixed with absorbing this to ask whether this tool was an open educational resource.

Bent Kure from the University of Oslo described how they have redesigned a first year philosophy class (mandatory for all students) into a MOOC-like course for the 2,000+ students a year who had previously studied this as self-directed learning ( ‘Here’s the textbook – turn up at the exam.’), because there was no way to fit them all into a lecture hall. The new version consisted of 8-10 minute videos+textbook+online discussion+mobile app+online tests. Well, ANYTHING would be better than the previous arrangement, wouldn’t it? And surprise, the other students who were privileged to attend the lectures also were using the stuff. But is this a MOOC? (and does it matter?).

The last session was about how George Washington University was designing its MOOC on the history of the U.S. Federal Reserve Bank, and especially how they were using professional-standard video rather than talking heads (probably wise since the participants include Bernanke, Greenspan and Volker). It seems to me that this will be one of the new generation of MOOCs in that it has involved instructional designers and a team approach that is often found in credit-based online courses.

Yes, blended learning is the future in academia, especially for very large first and second year classes. However, the external MOOC integrated into local teaching is probably not the model that will dominate. I suspect experience will show that better results can be obtained by careful re-design, including the flexible incorporation of a wide range of OERs, not only MOOCs. In particular, copyright issues need to be recognized, since Coursera and Udacity MOOC materials are not open for re-use without permission.

Technology-enhanced learning: what do we know and what is yet to be learned

The Research Council of Norway issued a contract for a complete review of the literature on technology-enhanced learning (the European term for information and communications technologies in education) over the last 20 years, covering the whole range from pre-school to post-secondary education to lifelong learning. This involved a trawl of over 1,000 journals (an example of how scattered the research is in this area), using the Thomson Reuters and Google Scholar indexing databases. The aim was to do an objective review of the research, based on a quantitative count of citations used.

Barbara Wasson and Konrad Morgan, the two contractors, gave a detailed presentation of the methodology and preliminary findings. This report when published will be extremely useful, but because of their desire to be totally objective, they were reluctant to ‘editorialize’. However, it is clear that a number of conclusions can be drawn already from this study:

  • there is a long history of research in this field, dating back over sixty years
  • some themes, such as computer aided instruction/CBL, collaborative learning, and robotic intelligent tutoring, have continued right through to today
  • just counting citations has its limits: for instance an ‘in-group’ can boost its count by cross-referencing each others’ work, without really impacting on practice or even the dissemination of knowledge to a wider group
  • the great majority of research is extremely short-term, with low samples: funding agencies should concentrate on more longitudinal studies and bigger samples
  • researchers are often isolated, working alone or in small groups, and therefore have little overall impact
  • a great majority of research is tool-based which goes quickly out of date as new tools arrive; researchers fail often to learn from earlier research on similar tools
  • there is a huge problem with aggregating, summarizing and disseminating the often very useful research to practitioners: it is largely inaccessible

As readers will know, I am not afraid to editorialize, summarize or disseminate, so I ended the session with my take on how prior online learning research could inform and improve the design of MOOCs (the same presentation I made to the MIT LINC conference.)

Where was Canada?

The conference attracted over 300 participants, two government ministers from Norway, including the Minister of Education and Research, one congressman from the USA, the CEO of the American Association for the Advancement of Science Education, and leading academics from some of the USA’s most prestigious universities and higher education organizations, with of course heavy representation from the Ministry of Education and Research, the Research Council, and universities in Norway.

By contrast, Canada was virtually unrepresented. From the participants list, it appears that only two Canadians attended, myself (invited by the Research Council of Norway), and Barbara Wasson, a Canadian researcher living and working in Norway. Where has the Canadian Embassy in Washington been on this event? There were as many representatives listed from the  Macedonian and Serbian embassies as from Canada, and I never found the Canadian Embassy person. No doubt Washington Canadian Embassy staff have been working to rule, as part of a widespread industrial dispute in Canada’s Department of Foreign Affairs, but since the conference next year is scheduled to be held in Toronto, the lack of Canadian representation at this conference was shocking, especially since Canada has some of the most knowledgeable people on MOOCs (and I’m not one of them), cyber security and emergency preparedness. It is Canada’s loss that we were not better represented at this most valuable conference, which is why I have spent so much effort on this post.

 

Examining the potential and reality of open educational resources: the 2013 COHERE conference

Listen with webReader
Panel: Robert Clougherty; George Veletsianos; Johan Fridell; Diane Salter; Rory McGreal

Panel: Robert Clougherty; George Veletsianos; Johan Fridell; Diane Salter; Rory McGreal

COHERE (Collaboration for Online Higher Education and Research) runs one of my favourite annual conferences. It is relatively small (around 75), the participants are mainly leading practitioners in online learning, and the sessions are excellent, usually encompassing leading developments in online learning, research reports, and extensive discussion from people in the front line of online learning. (Click here for a report on the 2012 conference.)

This year, the conference, jointly sponsored by the Canadian Society for the Study of Higher Education (CSSHE), and held at Kwantlen Polytechnic University’s Richmond campus in British Columbia, focused on ‘Open Resources, Open Courses: their Impact on Blended and Online Learning’.

The two main keynote speakers this year were Cable Green, the Director of the Creative Commons, and David Porter, the Director of BC Campus. Here is a summary of my closing remarks which aimed to review the conference (although I was able to attend only 25% of the parallel sessions – I can do any time, anywhere, but not yet two places at the same time).

To get copies of the slides of presenters, you will need to get a Dropbox invitation from Stacey Woods. I don’t have my own slides – just the notes below.

Creative Commons

This seemingly simple idea, of an ‘author’ creating a license enabling people to freely access and adapt copyright material, without charge or special permission, is one of the great ideas of the 21st century. This does not take away someone’s copyright but enables that copyright holder to give permission for different kinds of use of their material without charge or any bureaucracy. There is no real legal or technical barrier now to making educational material free. It does though require a particular mindset among both copyright holders – i.e. the creators of materials – and users – i.e. teachers and instructors who could use this material in their teaching. Thus the main challenge is one of cultural change.

The spectrum of Creative Commons licenses

The spectrum of Creative Commons licenses

Open textbooks

In some ways, open textbooks are a no brainer. There’s probably no greater racket than the school and college textbook industry (other than the research journal industry.) Often these textbooks are not ‘original’ work, in the sense of new knowledge, but carefully written and well illustrated summaries of current thinking in the different subject areas.

Students and governments, through grants and financial aid, pay billions of dollars each year on textbooks produced by people who are largely already employed in the public education sector (and who in any case are lucky to get 10% of any revenue generated). A student in Canada spends on average about $800 a year on textbooks, even more in the USA. In some subject areas, the cost is well over $1,000 a year per student.

Nothing seems more absurd to me than the sight of hundreds of students lining up for up to an hour every day for the first week of the semester at the UBC bookstore to buy their books. This is time lost studying. Cable Green pointed to research that shows that when first year math students have their textbooks from the first day, they do much better than students who often don’t get the key textbook until three weeks into the course. He also pointed to research from Florida Virtual Campus that indicates that many students (over 60%) simply do not buy all the required textbooks, for a variety of reasons, but the main one being cost. Indeed, students are often reluctant to take their books to campus in case they lose them (although he was referring to k-12 children here).

So why shouldn’t government pay the creators of textbooks directly, cut out the middleman (commercial publishers), save over 80% on the cost, and distribute the books to students (or anyone else) for free over the Internet, under a Creative Commons license? What’s not to like (unless you’re a publisher)? In his presentation, Cable Green came up with a great ‘vision’ for open textbooks: 100% of students have 100% free, digital access to all materials on day one.

So it was good to hear from David Porter that BC Campus is developing 40 open textbooks for first and second year university courses, and another 20 for two year college vocational and technical courses. BC is also collaborating with Alberta in Canada, and Washington State and Oregon, in the USA, to avoid duplication and to increase sharing of open textbooks. Explore the BC Campus web site: at the time of writing this post there are already seven open textbooks available, and perhaps more significantly, faculty from 20 of the 24 post-secondary institutions in B.C. are participating in the creation of open textbooks. Someone asked the question: what is Ontario (Canada’s largest province) doing about open textbooks? The answer to date is: nothing.

However, there were some cautionary concerns from some of the participants about open textbooks:

  • Elizabeth Murphy, from Memorial University, questioned the whole idea of textbooks, whether open or not. She saw textbooks as a relic of 19th century industrialism, a form of mass broadcasting. In the 21st century, students should be finding, accessing and collecting digital materials over the Internet. Textbooks are merely packaged learning, with the authors doing the work for students. Nevertheless, it has to be recognized that textbooks are still the basic currency for most forms of education, and while this remains the case, open textbooks are a much better alternative for students
  • others (including myself) questioned the likely impact of ‘open’ publishing on creating original works that are not likely to get subsidized by government because they are either too specialized, or are not yet part of a standard curriculum for the subject; in other words would it impact on the diversity of publishing? What is the incentive for someone now to publish a unique work, if there is no commercial reward for the effort (especially if you believe that open publishing will eventually wipe out commercial publishing, as I do)? Writing an original, single authored book remains hard work, however it is published.
  • although there is now a range of  ’open’ publishing services, there are still costs for an author to create original work. Who will pay, for instance, for specialized graphics, for editing or for review? How can these costs be recovered? In particular, in what format (pdf, html, Word, ePub, xml, a wiki) should original work be created so that it can become interactive, easily re-purposed, and multimedia? Much more work still needs to be done to support the open publishing of original work in book format, at least – or is the book itself a relic of another time? If so, what does that mean for how knowledge is created, disseminated and preserved? (Answers on a – digital – postcard, please).

Although these are all important concerns, they seem to me to be manageable. Just getting a proportion of the main textbooks available to students for free is a major step forward.

Openstax open textbooks

Openstax open textbooks

Getting faculty to use OERs

Increasingly faculty are creating open educational resources, or making resources freely available for others to use under a Creative Commons license. There are increasing numbers of depositories or portals where faculty can access open educational resources. Faculty have a number of choices:

  • create your own digital resources, and make them available to others (there are plenty of guides on how to do this.)
  • take OERs selectively from elsewhere, and incorporate or adapt them into your own teaching
  • take a whole course from elsewhere (e.g. a MOOC) and provide learner support and teaching around those materials (such as San Jose State University is doing, or as in the Carnegie-Mellon Open Learning Initiative.)

There is a lot of evidence to suggest that the take-up of OERs by instructors is still minimal. Diane Salter of Kwantlen Polytechnic University led a discussion on how best to get faculty to incorporate OERs into their teaching. Many of the suggestions made will be familiar to anyone concerned with change management in higher education: there was the discussion of the need for the move to be both top-down and bottom up; I argued the importance of these kinds of decision being made at a program level; faculty development and workshops are essential.

However, Keith Hampson raised a much more important barrier and that is how university faculty see themselves. They don’t see themselves as ‘just’ teachers, but creators and disseminators of new or original knowledge. Therefore their teaching needs to have their own stamp on it, which makes them reluctant to openly incorporate or ‘copy’ other people’s work. We can argue that this is absurd – we all stand on the shoulders of giants – but it’s the self-perception that’s important, and for research professors, there is a grain of truth in the argument. It makes sense for them to focus their teaching on their own research. But then how many Richard Feynmans are there out there? The problem is that OERs can easily be associated with ‘packaged’, reproductive knowledge, and not original work, changing faculty from ‘artists’ to ‘artisans’.

One practical step that could increase greater adoption of OERs would be some open course design templates into which OERs could be dropped, with examples, and places for instructors to add their own resources. These design templates could range from more didactic teaching somewhat similar to an open LMS (some of which exist already), to more open or more flexible designs where students find, analyse and apply open resources within an overall teaching framework. I would hope such templates would include spaces for student activity, and multimedia resources. This would take something like CoursePacker to the next level.

The problem remains though that even when faculty adopt OERs, they are often inside a closed teaching environment, such as an LMS. What the OERu is doing is opening up this whole process so that students can openly access whole programs and receive full qualifications from the participating institutions. In fact, the OERu was launched yesterday at Thompson Rivers University, in Kamloops, British Columbia (more on this in another post).

The panel responding to David Porter’s presentation also raised several interesting points. While open education activities were reported from both Memorial University and the University of Ottawa, they tended to be outside actual course design, focused on library initiatives or open publishing in journals. Incidentally it seems bizarre to me that institutions are paying up to $3,000 an article for publishers to make the article open access within a journal, when the research has been publicly funded (what David Porter described as ‘openwashing’).

Ron Owston reported on the dead hand Access Copyright has had on using secondary sources for teaching, having taken York University to court over its interpretation of fair dealing. This is a must win case for Canadian universities and if York is successful it should sue Access Copyright for wasting the court’s time and the university’s resources after an earlier clear decision by Canada’s Supreme Court.

Ron also raised the question of the poor quality of much of the OERs available a the moment – reams of text with no interaction, course design, or media other than text, often in PDFs that cannot easily be changed or adapted. If OERs are to be taken up, they will need to be better designed.

Research and development on MOOCs

Ah, MOOCs. Funny how these came up. Well, there were some interesting presentations on this topic.

WideWorld Ed

WideWorld Ed is a new Canadian, online open education platform, the main driving force behind which is Jenny Heyman, who gave an interesting presentation at the conference (declaration of interest: I’m on the Academic Advisory Board). Its first MOOC is a six week course: Online Instruction for Open Educators, which opened on October 14. The instructors include some famous names: Terry Anderson, Athabasca University; Dave Cormier, University of Prince Edward Island (UPEI); Bonnie Stewart, UPEI; Jenni Hayman, Wide World Ed; and Sean Gallagher, Wide World Ed. Interestingly, sponsorship for the course comes from Desire2Learn.

WideWorld Ed’s mission is to deliver well-designed and effective online courses and open education resources to diverse learners around the world. In particular it hopes to attract course providers from Canadian institutions and organizations, a Canadian edX.  In particular, I can see a real market for NGOs, charitable institutions wishing to get better outreach for their services, and public institutions seeking wider audiences.This is a brave start-up effort, and I hope it gets the support it deserves from Canadian universities and colleges, venture capitalists and other sponsors. Now if we just had a Federal Department of Education to put some money behind it…..

WideWorldEd

Athabasca University’s research on MOOCs

Marti Cleveland-Innes gave a presentation on Athabasca University’s research program examining MOOCs, funded by the Bill and Melinda Gates Foundation. This is just starting and the aim is to address the research gap by evaluating MOOCs, and how they impact teaching, learning, and education in general. There has been an rfp for proposals administered by Athabasca University, and the successful applicants will be presenting their proposals at a forthcoming conference at the University of Texas Arlington December 5-6

Research into students’ experience of MOOCs

George Veletsianos, who recently moved from the University of Texas at Austin to Royal Roads University, BC, and who is a leading researcher on emerging educational technologies, has also been conducting research on MOOCs, from the learners’ perspectives. George’s take on MOOCs is interesting. He sees the MOOC phenomenon as a result of chronic failures in the post-secondary education system (I’m not sure if he was talking generally, or just the USA). Among a range of failures he cites the lack of impact of educational technology research on course design, and the failure of educational technologists to make any impact on practice (ah, well, there’s a lifetime’s work down the toilet). It’s hard to disagree with him, though, given his results from interviewing MOOC students: that MOOCs suffer from a lack of course design, that instructor’s presence during the course (beyond a recorded lecture) is important, etc., results that to be honest are highly predictable from the research that was done before MOOCs were launched.

But is that the fault of educational technologists, or computer science professors who blindly failed to do any literature research into online learning or how students learn, or even talk to educators before launching their products? Talk about arrogance and ignorance combined, especially given Keith Hampson’s comment that the Carnegie Mellon Open Learning Initiative, based on cognitive science research and best instructional design practices, generally leads to a minimum of 25% improvement in student performance when colleges follow their approach (results also found by Carol Twigg’s course redesign studies.) To re-quote G.K. Chesterton, it’s not that research into educational technologies has been applied and failed – it’s never been applied by most instructors. That’s because we don’t train them in teaching, not because we don’t disseminate the results.

It’s great that both Athabasca and George are doing research on MOOCs. This is sorely needed. However, they are applying evidence-based research and rational analysis to a phenomenon that is political, emotional and largely irrational. For a true understanding of the MOOC phenomenon, we probably need a socio-cultural analysis, so their research could easily end the way that George has portrayed previous research into educational technology: ignored, though it is highly relevant.

Is open education becoming a tool of the right?

So here’s my socio-cultural analysis, for what it’s worth.

The reason that MOOCs have received such media hype is because the USA in particular has been destroying its own public higher education system through budget cuts and an unwillingness to pay taxes. If elite universities can deliver MOOCs for free, why do we need crappy state universities? The risk is a sharply divided two tier system, with a relatively small number of elite universities catering to the rich, and the masses going to MOOC-delivered courses with state universities providing minimal and low cost learner support for such courses.

This would be both a social and economic disaster, because it would fail to produce learners with the high-level skills that are going to be needed for good jobs in the the coming years – unless you believe that automation will remove all paid jobs except for a tiny elite (is this the hidden agenda?).

It should be noted that even for credit-based online program, content accounts for less than 15% of the total cost over five years; the main costs required to ensure high quality outcomes and high rates of completion are spent on learner support, providing the learning that matters most. The kind of MOOCs being promoted by politicians and the media fail spectacularly to do this.

We do need to be careful that the open education movement is not used as a stick by those in the U.S. (and elsewhere) who are deliberately trying to undermine public education for ideological and commercial reasons. Open content, OERs and MOOCs do not automatically lead to open access to high quality credentials.

In the end, a well-funded public higher education system remains the best way to assure access to higher education for the majority of the population. Having said that, there is enormous scope for improvements within that system. Open education and its tools offer a most promising way to bring about some much needed improvements. That is my main take-away from this excellent conference.