July 25, 2017

Online learning in 2016: a personal review


global-peace-index-2016-aglobal-peace-initiative-b

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Image: © Institute for Economics and Peace. Canada is ranked seventh most peaceful. We don’t know where it ranks though in terms of online learning.

A personal review

I am not going to do a review of all the developments in online learning in 2016 (for this, see Audrey Watters’ excellent HackEducation Trends). What I am going to do instead is review what I actually wrote about in 2016 in this blog, indicating what to me was of particular interest in online learning during 2016. I have identified 38 posts I wrote in which I have explored in some detail issues that bubbled up (at least for me) in 2016.

1. Tracking online learning

Building a national survey of online learning in Canada (134 hits)

A national survey of university online and distance learning in Canada (1,529 hits)

In the USA, fully online enrollments continue to grow in 2014 (91 hits)

Are you ready for blended learning? (389 hits)

What the Conference Board of Canada thinks about online learning (200 hits)

I indulged my obsession with knowing the extent to which online learning is penetrating post-secondary education with five posts on this topic. In a field undergoing such rapid changes, it is increasingly important to be able to track exactly what is going on. Thus a large part of my professional activity in 2016 has been devoted to establishing, almost from scratch, a national survey of online learning in Canadian post-secondary institutions. I would have written more about this topic, but until the survey has been successfully conducted in 2017, I have preferred to keep a low profile on this issue.

However, during 2016 it did become clear to me, partly as a result of pilot testing of the questionnaire, and partly through visits to universities, that blended learning is not only gaining ground in Canadian post-secondary education at a much faster rate than I had anticipated, but is raising critical questions about what is best done online and what face-to-face, and how to prepare institutions and instructors for what is essentially a revolution in teaching.

This can be best summarized by what I wrote about the Conference Board of Canada’s report:

What is going on is a slowly boiling and considerably variable revolution in higher education that is not easily measured or even captured in individual anecdotes or interviews.

2. Faculty development and training

Getting faculty and instructors into online learning (183 hits)

Initiating instructors to online learning: 10 fundamentals (529 hits)

Online learning for beginners: 10. Ready to go (+ nine other posts on this topic = 4,238 hits)

5 IDEAS for a pedagogy of online learning (708 hits)

This was the area to which I devoted the most space, with ten posts on ‘Online Learning for Beginners’, aimed at instructors resisting or unready for online learning. These ten posts were then edited and published by Contact North as the 10 Fundamentals of Teaching Online.

Two fundamental conclusions: we need not only better organizational strategies to ensure that faculty have the knowledge and training they will need for effective teaching and learning in a digital age, but we also need to develop new teaching strategies and approaches that can exploit the benefits and even more importantly avoid the pitfalls of blended learning and learning technologies. I have been trying to make a contribution in this area, but much more needs to be done.

3. Learning environments

Building an effective learning environment (6,173 hits)

EDEN 2016: Re-imagining Learning Environments (597 hits)

Culture and effective online learning environments (1,260 hits)

Closely linked to developing appropriate pedagogies for a digital age is the concept of designing appropriate learning environments, based on learners’ construction of knowledge and the role of instructors in guiding and fostering knowledge management, independent learning and other 21st century skills.

This approach I argued is a better ‘fit’ for learners in a digital age than thinking in terms of blended, hybrid or fully online learning, and recognizes that not only can technology to be used to design very different kinds of learning environments from school or campus based learning environments, but also that technology is just one component of a much richer learning context.
Slide15

4. Experiential learning online

A full day of experiential learning in action (188 hits)

An example of online experiential learning: Ryerson University’s Law Practice Program (383 hits)

Is networked learning experiential learning? (163 hits)

These three posts explored a number of ways in which experiential learning is being done online, as this is a key methodology for developing skills in particular.

5. Open education

Acorns to oaks? British Columbia continues its progress with OERs (185 hits)

Talking numbers about open publishing and online learning (113 hits)

Towards an open pedagogy for online learning (385 hits)

These posts also tracked the development of open publishing and open educational resources, particularly in British Columbia, leading me to conclude that the OER ‘movement’ has far too narrow a concept of open-ness and that in its place we need an open pedagogy into which open educational resources are again just one component, and perhaps not the most significant.

6. Technology applications in online learning

An excellent guide to multimedia course design (659 hits)

Is video a threat to learning management systems? (603 hits)

Some comments on synchronous online learning technologies (231 hits)

Amongst all the hype about augmented reality, learning analytics and the application of artificial intelligence, I found it more useful to look at some of the technologies that are in everyday use in online learning, and how these could best be used.

7. Technology and alienation

Technology and alienation: online learning and labour market needs (319 hits)

Technology and alienation: symptoms, causes and a framework for discussion (512 hits)

Technology, alienation and the role of education: an introduction (375 hits)

Automation or empowerment: online learning at the crossroads (1,571 hits)

Why digital technology is not necessarily the answer to your problem (474 hits)

These were more philosophical pieces, prompted to some extent by the wider concerns of the impact of technology on jobs and how that has influenced Brexit and the Trump phenomena.

Nevertheless this issue is also very relevant to the teaching context. In particular I was challenging the ‘Silicon Valley’ assumption that computers will eventually replace the need for teachers, and in particular the danger of using algorithms in teaching without knowing who wrote the algorithms, what their philosophy of teaching is, and thus what assumptions have been built into the use of data.

Image: Applift

Image: Applift

8. Learning analytics

Learning analytics and learning design at the UK Open University (90 hits)

Examining ethical and privacy issues surrounding learning analytics (321 hits)

Continuing more or less the same theme of analysing the downside as well as the upside of technology in education, these two posts looked at how some institutions, and the UK Open University in particular, are being thoughtful about the implications of learning analytics, and building in policies for protecting privacy and gaining student ‘social license’ for the use of analytics.

9. Assessment

Developing a next generation online learning assessment system (532 hits)

This is an area where much more work needs to be done. If we are to develop new or better pedagogies for a digital age, we will also need better assessment methods. Unfortunately the focus once again appears to be more on the tools of assessment, such as online proctoring, where large gains have been made in 2016, but which still focus on proctoring traditional assessment procedures such as time-restricted exams, multiple choice tests and essay writing. What we need are new methods of assessment that focus on measuring the types of knowledge and skills that are needed in a digital age.

For instance, e-portfolios have held a lot of promise for a long time, but are still being used and evaluated at a painfully slow rate. They do offer though one method for assessment that reflects much better the needs of assessing 21st century knowledge and skills. However we need more imagination and creativity in developing new assessment methods for measuring the knowledge and skills needed for a digital age.

That was the year that was

Well, it was 2016 from the perspective of someone no longer teaching online or managing online learning:

  • How far off am I, from your perspective?
  • What were the most significant developments for you in online learning in 2016?
  • What did I miss that you think should have been included? Perhaps I can focus on this next year.

I have one more post looking at 2016 to come, but that will be more personal, looking at my whole range of online learning activities in 2016.

In the meantime have a great seasonal break and I will be back in touch some time in the new year.

Online learning for beginners: 7. Why not just record my lectures?

Why not just put your lecture capture lessons online? Image: MediaCore, 2014

Why not just put your lecture capture lessons online?
Image: MediaCore, 2014

This is the seventh in a series of a dozen blog posts aimed at those new to online learning or thinking of possibly doing it. The other six are:

I gave a short answer to ‘Why not just record my lectures?’ in the fourth post in this series, but it deserves a fuller answer. It is natural that faculty and instructors want to use an approach to teaching that is not only familiar and comfortable, but has been used for hundreds of years, so has passed the test of time. However, there are several reasons why recorded classroom lectures are not a good idea for online learning, at least not as the main form of delivering online courses.

Start with the students

When designing online courses, you need to start by thinking about the context of the online learner. An online learner is usually studying in an isolated situation, without other students or the instructor physically present. There are many ways to overcome the isolation of the online learner (dealt with in later posts), but giving them recordings of 50 minute classroom presentations is not one of them.

In a classroom context there are many interactive cues or contexts – such as the response of other students, the look on students faces – that result in slight but important adjustments on the instructor’s part, and which help maintain student concentration and interest. Even if a live class was present at the time of the recording, these cues are usually lacking when students are studying a recorded video at home, in the library, or on the bus.

There is also research evidence that suggests for every hour of presentation, online students need to spend between two to three hours of additional time going over the recording, stopping and starting, to ensure they have fully understood. This is a good benefit of recording compared to even a live lecture, but it also ups the student workload, especially if there is other work to be done, such as readings, assignments, and practical work. Managing student workload is a key factor in ensuring high completion rates for online students.

Lastly, even when recorded lectures are strongly integrated with other activities, such as subsequent classroom discussions or assignments, students often skimp on the video preparation, either skimming the video or not watching it at all. The more isolated the student, the more likely this is to happen.

The changing nature of learning in a digital age

One of the main reasons for moving to online learning is to help develop the knowledge and skills needed in a knowledge-based society and particularly in a digital age. These new forms of knowledge – such as Internet-based sources and rapidly changing content – and in particular the skills required to master these forms of knowledge, such as knowledge management, independent learning and use of digital media – are not handled well through lectures. In particular the lecturer is doing the knowledge management, the modelling and the organization of content, not the students.

In other words lectures require a more passive approach from the learner which is not suitable for isolated learners who need to be active and engaged in their learning, as much for motivational reasons as for developing the knowledge and skills needed. (The same could also be true for classroom based students, incidentally.) Indeed, one of the principle reasons for moving to online teaching is to move away from the limitations of lecture-based classes, and to exploit the benefits of online study.

Video as a teaching medium

Asking ‘Why can’t I move my lectures online?’ is really the wrong question. It assumes that what I’m doing in the classroom will work equally well on video for online students. The right question though should be: ‘What is the best use of video for students studying online?’

In media terms, a recorded lecture is mainly a talking head, with, if students are lucky, textual illustration (e.g. Powerpoint slides). There has been a great deal of research on the best mix of voice, images, and text in video for teaching (see, for instance Mayer, 2009). To incorporate the factors that make the use of video effective for learning, the type of lecture usually delivered in a classroom would need to be considerably redesigned to make it more effective for remote learners.

In addition, there are many other, more creative and relevant ways than lectures for using video for teaching, such as demonstrations of equipment, experiments or processes, animation, and examples drawn from the real world to illustrate abstract concepts.

Successful uses of video for lectures

It could be argued that MOOCs, the Khan Academy, and TED talks are all examples of the successful use of lectures on video. However, they are not the typical classroom lecture delivered three times a week over a 13 week semester.

I have heard instructors say that their MOOC lectures are much better than their classroom lectures, because they put more time into the presentation. MOOC developers have learned to adapt the 50 minute lecture to better fit the online format, with shorter, 10-15 minutes videos, and shorter courses. This is fine for non-credit programming but does not fit the Carnegie-based 13 week semester model for credit programs. Costs for producing successful MOOC lectures run over $100,000 a lecture, production costs that are not sustainable for moving large numbers of classroom lectures online.

Sal Khan is an inspired lecturer who uses voice over combined with on-screen digital notes. His technique is not the same as recording a classroom lecture with whiteboard notes. For a start, the audio and screen quality is much higher, but it is also the technique of constructing teaching in appropriate chunks of recorded time that requires considerable thought and preparation. This is not to say that classroom lecturers could not do this, but it would require once again redesigning the classroom lecture.

Lastly, TED talks require a great deal of preparation and rehearsal, and again are much shorter than the typical classroom lecture.

So, yes, recorded video can work online, but it needs to be designed specifically to suit the mode of delivery. There are also other ways to design online learning that do not necessarily require so much work, and other uses of video for teaching that are more appropriate.

What are the alternatives?

Too many to list them all here, but one is to use an online learning management system, such as Blackboard, Moodle or D2L. These provide a weekly structure for ‘lessons’, organize content in the form of text or online readings, provide a forum for discussion on course topics, provide regular online activities and assignments, and could include links to short videos. Indeed, a short introductory video to a topic by the instructor is often a good idea, providing a personal link between you and your students.

I will discuss other possible online learning environments in later posts.

Implications

  1. A talking head delivering 50 minute lectures is in general not a good way to teach online learners.
  2. It is better in the long run to sit down with an instructional designer and build a course from scratch that is appropriate for an online learning environment, rather than try to force your classroom teaching online.
  3. Video is a good medium to use for online learning, but only if it exploits its unique pedagogical benefits.
  4. Talking heads are therefore useful only in particular contexts, and not as a way to deliver a whole course or program online.
  5. Developing quality video for online learning requires a professional approach involving lecturer, instructional designer and a multimedia or video producer.

Follow up

For a critique of the limitations of classroom lectures based on research by Donald Bligh, see Chapter 3.3 Transmissive lectures: learning by listening in Teaching in a Digital Age.

For a good summary of best design principles for developing video/multimedia for learning, based on research on the learning effectiveness of video, see the University of British Columbia’s Design Principles for Multimedia.

For a discussion of the pedagogical potential of video, see Chapter 7.4.2, Presentational features in Teaching in a Digital Age

If you want to follow up on the research and theory on which this post is based see:

  • Bligh, D. (2000) What’s the Use of Lectures? San Francisco: Jossey-Bass
  • Mayer, R. E. (2009). Multimedia Learning (2nd ed). New York: Cambridge University Press.
  • McKeachie, W. and Svinicki, M. (2006) McKeachie’s Teaching Tips: Strategies, Research and Theory for College and University Teachers Boston/New York: Houghton Mifflin

Up next

Won’t online learning be more work?

Your turn

If you have comments, questions or plain disagree, please use the comment box below.

MIT and German research on the [appalling] use of video in xMOOCs

Demonstration is one of the 18 video production styles from a Coursera course “Mechanics: Motion, Forces, Energy and Gravity, from Particles to Planets” (UNSW Australia)

Demonstration is one of the 18 video production styles from a Coursera course “Mechanics: Motion,
Forces, Energy and Gravity, from Particles to Planets” (UNSW Australia)

Hansch, A. et al. (2015) Video and Online Learning: Critical Reflections and Findings From the Field Berlin DE: Alexander von Humbolt Institut für Internet und Gesellschaft

The study

This exploratory study examines video as an instructional medium and investigates the following research questions:

  • How is video designed, produced, and used in online learning contexts, specifically with regard to pedagogy and cost?
  • What are the benefits and limitations of standardizing the video production process?

Findings are based on a literature review, our observation of online courses, and the results of 12 semi-structured interviews with practitioners in the field of educational video production

We reviewed a variety of different course and video formats offered on six major platforms: Coursera, edX, Udacity, iversity, FutureLearn, and Khan Academy.

Results

(My summary, the authors’ words in italics)

1. We found documentation on the use of video as an instructional tool for online learning to be a notably underexplored field. To date, little consideration has been given to the pedagogical affordances of video, what constitutes an effective learning video, and what learning situations the medium of video is best suited for.

2. On the whole, we found that video is the main method of content delivery in nearly all MOOCs. MOOC videos tend to be structured as short pieces of content, often separated by assessment questions. This seems to be one of the few best practices that is widely accepted within the field.

3. We found two video production styles that are most commonly used: (1) the talking head style, where the instructor is recorded lecturing into the camera, and (2) the tablet capture with voiceover style (e.g. Khan Academy style).

4. It appears that the use of video in online learning is taken for granted, and there is often not enough consideration given to whether or not video is the right medium to accomplish a MOOC’s pedagogical goals.

5. Video tends to be the most expensive part of MOOC production. There is a tendency for institutions to opt for a professional, studio-style setup when producing video… but.. there is little to no research showing the relevance of production value for learning.

6. More research is needed on how people learn from video.

Recommendations

1. Think twice before using video….it seems problematic that online learning pedagogy is concentrated so heavily in this medium. Hence, we want to discourage the use of video in online learning simply because there is an expectation for it, and rather encourage online learning producers and providers to question video’s extensive use at the expense of other pedagogical alternatives

2. Make the best use of video as a medium…Based on our findings, we have compiled an overview of the medium of video’s affordances for online learning. [Nine ‘affordances’ of video are recommended]

Comment

First, this is not really about video in online learning, but video in xMOOCs, which is just one, fairly esoteric use of video in online learning. Nevertheless, since xMOOCs are in widespread use, it is still a valid and important area of research.

Unfortunately, though, the authors’ literature search was barely adequate. I will forgive the failure to discuss the 20 years of research on television and video at the UK Open University, or the research done on the educational effects of television from Sesame Street, but although the authors of this paper include a reference to his book in the bibliography, the failure in the main text to recognise properly Richard Mayer’s contribution to what we know about using video for teaching and learning is unforgivable, as is the authors’ conclusion that the use of video as an instructional tool for online learning is a notably underexplored field. Sorry, but its the authors who haven’t looked in the right places.

Secondly, it’s not that I disagree with their recommendations, it’s that what they are recommending has been known for a long time. More research is always useful, but first the existing research needs to be read, learned and applied.

Thirdly, this paper reinforces what many of us with experience in online learning and/or in the use of video in education have known all along: those designing xMOOCs have made the most egregious of errors in effective design through sheer ignorance of prior research in the area. Since those making these stupid mistakes in course design come from elite, research-based institutions, the sin of ignoring prior research is even more unforgivable. Once gain we have MIT, Stanford and Harvard and the other xMOOC providers having to use new research to rediscover the wheel through ignorance and arrogance.

Fourthly, the real value of this paper comes from the authors’ typology of video production styles. They offer a total of 18 possible production styles, with a short description and a set of questions to be asked about each. This alone makes the paper worth reading for anyone considering using video in online learning, although the authors fail to point out which of the production styles should be avoided, and which used, according to the research.

Lastly, what this paper really reinforces above all is that we should stop taking xMOOCs seriously. They are badly designed by amateurs who don’t know what they are doing. Let’s move on to more important issues in online learning.

 

Seeking the unique pedagogical characteristics of video

Figure 9. The Open University on iTunesU

Figure 9. The Open University on iTunesU

This is the third post on the unique characteristics of different media, for my open textbook, Teaching in a Digital Age.

Although it will be seen that there are good pedagogical reasons for using video, it presents much more of a challenge to faculty than the use of text or audio. Producing video that exploits the unique characteristics of video is not something that most faculty have the time or ability to do themselves, and adds substantial cost to a course.

The alternative of course is video available as an open educational resources, and good luck with that. I had great difficulty in finding suitable open educational resources to use as examples (although there are talking heads in abundance). If anything, the availability of good quality video OERs has declined recently, with much of the material previously available through Open Learn and other sources such as iTunesU and even YouTube now removed. Copyright of good quality educational video is still pretty restricted, probably because of the high cost of producing it.

Reliability of OERs is becoming a critically important issue. If an instructor cannot rely on an OER being available in a year or two after incorporation into their teaching, OERs won’t get used. Maybe this is after all a good reason for learning object repositories.

Ideally, I would like to be able to link each one of these unique features to an open source video example. After two days trawling, I’ve come up with one (thank you, University of Nottingham, and Clint Lalonde for suggesting it!) So any suggestions for ‘open’ videos that provide examples for each of the characteristics below would be really appreciated. (Yes, I know I should ask a librarian, but I’m working on my own these days).

More power, more complexity

Although there have been massive changes in video technology over the last 25 years, resulting in dramatic reductions in the costs of both creating and distributing video, the unique educational characteristics are largely unaffected. (More recent computer-generated media such as simulations, will be analysed under ‘Computing’, in Section 9.5.4).

Video is a much richer medium than either text or audio, as in addition to its ability to offer text and sound, it can also offer dynamic or moving pictures. Thus while it can offer all the affordances of audio, and some of text, it also has unique pedagogical characteristics of its own. Once again, there has been considerable research on the use of video in education, and again I will be drawing on research from the Open University (Bates, 1985 2005; Koumi, 2006) as well as from Mayer (2009).

Presentational features

Video can be used to:

  • demonstrate experiments or experimental situations, particularly:
  • illustrate principles involving dynamic change or movement
  • illustrate abstract principles through the use of specially constructed physical models
  • illustrate principles involving three-dimensional space
  • demonstrate changes over time through the use of animation, slow-motion, or speeded-up video
  • substitute for a field visit, by:
    • providing students with an accurate, comprehensive visual picture of a site, in order to place the topic under study in context
    • demonstrating the relationship between different elements of a system under study (e.g. production processes, ecological balance)
    • by identifying and distinguishing between different classes or categories of phenomena at the site (e.g. in forest ecology)
    • to observe differences in scale and process between laboratory and mass-production techniques
    • through the use of models, animations or simulations, to teach certain advanced scientific or technological concepts (such as theories of relativity or quantum physics) without students having to master highly advanced mathematical techniques,
  • bring students primary resource or case-study material, i.e. recording of naturally occurring events which, through editing and selection, demonstrate or illustrate principles covered elsewhere in a course
  • demonstrate ways in which abstract principles or concepts developed elsewhere in the course have been applied to real-world problems
  • synthesise a wide range of variables into a single recorded event, e.g. to suggest how real world problems can be resolved
  • demonstrate decision-making processes or decisions ‘in action’ (e.g. triage in an emergency situation) by:
    • recording the decision-making process as it occurs in real contexts
    • recording ‘staged’ simulations, dramatisation or role-playing
  • demonstrate correct procedures in using tools or equipment (including safety procedures)
  • demonstrate methods or techniques of performance (e.g. mechanical skills such as stripping and re-assembling a carburetor, sketching, drawing or painting techniques, or dance)
  • record and archive events that are crucial to topics in a course, but which may disappear or be destroyed in the near future, such as, for instance, street graffiti or condemned buildings
  • demonstrate practical activities to be carried out by students, on their own.

Skills development

This usually requires the video to be integrated with student activities. The ability to stop, rewind and replay video becomes crucial for skills development, as student activity usually takes place separately from the actual viewing of the video. This may mean thinking through carefully activities for students related to the use of video.

If video is not used directly for lecturing, research clearly indicates that students generally need to be guided as to what to look for in video, at least initially in their use of video for learning. There are various techniques for relating concrete events with abstract principles, such as through audio narration, using a still frame to highlight the observation, or repeating a small section of the program. Bates and Gallagher (1977) found that using video for developing higher order analysis or evaluation was a teachable skill that needs to be built into the development of a course or program, to get the best results.

Typical uses of video for skills development include:

  • enabling students to recognize naturally occurring phenomena or classifications (e.g. teaching strategies, symptoms of mental illness, classroom behaviour) in context
  • enabling students to analyse a situation, using principles either introduced in the video recording or covered elsewhere in the course, such as a textbook or lecture
  • interpreting artistic performance (e.g. drama, spoken poetry, movies, paintings, sculpture, or other works of art)
  • analysis of music composition, through the use of musical performance, narration and graphics
  • testing the applicability or relevance of abstract concepts or generalisations in real world contexts
  • looking for alternative explanations for real world phenomena.

Strengths and weaknesses of video as a teaching medium

One factor that makes video powerful for learning is its ability to show the relationship between concrete examples and abstract principles, with usually the sound track relating the abstract principles to concrete events shown in the video. Video is particularly useful for recording events or situations where it would be too difficult, dangerous, expensive or impractical to bring students to such events.

Thus its main strengths are as follows:

  • linking concrete events and phenomena to abstract principles and vice versa
  • the ability of students to stop and start, so they can integrate activities with video
  • provides alternative approaches that can help students having difficulties in learning abstract concepts
  • adds substantial interest to a course by linking it to real world issues
  • a growing amount of freely available, high quality academic videos
  • good for developing some of the higher level intellectual skills and some of the more practical skills needed in a digital age
  • the use of low cost cameras and free editing software enables some forms of video to be cheaply produced.

The main weaknesses of video are:

  • many faculty have no knowledge or experience in using video other than for recording lecturing
  • there is currently a very limited amount of high quality educational video free for downloading, because the cost of developing high quality educational video that exploits the unique characteristics of the medium is still relatively high. Links also often go dead after a while, affecting the reliability of outsourced video. The availability of free material for educational use will improve over time, but currently finding appropriate and free videos that meet the specific needs of a teacher or instructor can be time-consuming or such material may just not be available or reliable
  • creating original material that exploits the unique characteristics of video is time-consuming, and still relatively expensive, because it usually needs professional video production
  • to get the most out of educational video, students need specially designed activities that sit outside the video itself
  • students often reject videos that require them to do analysis or interpretation; they often prefer direct instruction that focuses primarily on comprehension. Such students need to be trained to use video differently, which requires time to be devoted to developing such skills.

For these reasons, video is not being used enough education. When used it is often an afterthought or an ‘extra’, rather than an integral part of the design, or is used merely to replicate a classroom lecture, rather than exploiting the unique characteristics of video.

 Assessment

If video is being used to develop the skills outlined in Section 9.5.3.3, then it is essential that these skills are assessed and count for grading. Indeed, one possible means of assessment might be to ask students to analyse or interpret a selected video, or even to develop their own media project, using video they themselves have collected or produced, using their own devices.

Activity 9.4

1. Take one of the courses you are teaching. What key presentational aspects of video could be important for this course?

2. Look at the skills listed in Section 1.3 of this book. Which of these skills would best be developed through the use of video rather than other media? How would you do this using video-based teaching?

3. Under what conditions would it be more appropriate for students to be assessed by asking them to analyse or make their own video recording? How could this be done under assessment conditions?

4. Type in the name of your topic + video into Google.

  • How many videos come up?
  • What’s their quality like?
  • Could you use any of them in your teaching?
  • If so, how would you integrate them into your course?
  • Could you make a better video on the topic?
  • What would enable you to do this?

Here are some criteria I would apply to what you find:

  • it is relevant to what you want to teach
  • it demonstrates clearly a particular topic or subject and links it to what the student is intended to learn
  • it is short and to the point
  • the example is well produced (clear camera work, good presenter, clear audio)
  • it provides something that I could not do easily myself
  • it is freely available for non-commercial use

I have to say that most of the examples I found on the Internet do NOT meet all of these criteria! The videos I have linked to in this section do, but then some are produced for the Open University. Can traditional university in-house media departments meet this standard?

Feedback

1. Are there other characteristics unique to video that I’ve missed?

2. Is this the best way to approach this topic? (I accept I need lots more examples in video format). Will this approach to choosing/ using video be helpful for faculty?

3. Any examples of using video for assessment?

4. What do you think of the principles I suggested for selecting video OERs in the activity?  Can traditional university in-house media departments meet this standard in producing OERs, or is it just too expensive to make these kinds of video?

5. Any other suggested references?

References

Bates, A. (1985) Broadcasting in Education: An Evaluation London: Constables (out of print – try a good library)

Bates, A. (2005) Technology, e-Learning and Distance Education London/New York: Routledge

Koumi, J. (2006). Designing video and multimedia for open and flexible learning. London: Routledge.

Mayer, R. E. (2009). Multimedia learning (2nd ed). New York: Cambridge University Press.

 

What UBC has learned about doing MOOCs

Coursera certificate 2

Engle, W. (2104) UBC MOOC Pilot: Design and Delivery Vancouver BC: University of British Columbia

The University of British Columbia, a premier public research university in Canada, successfully delivered five MOOCs in the spring and summer of 2013, using the Coursera platform. This report is an evaluation of the experience.

The report is particularly valuable because it provides details of course development and delivery, including media used and costs. Also UBC has been developing online courses for credit for almost 20 years, so it is interesting to see how this has impacted on the design of their MOOCs.

The MOOCs

1. Game Theory I: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

2. Game Theory II: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

3. Useful Genetics: R. Redfield, UBC

4. Climate Literacy: S. Harris and S. Burch, UBC

5. Introduction to Systematic Program Design: G. Kizcales, UBC

In terms of comparability I’m going to treat Game Theory I and II as one MOOC, as combined they were about the same length as the other MOOCs (between 8-12 weeks)

Basic statistics

330,150 signed up (82,500 on average per course)

164,935 logged in at least once (41,000 per course)

12,031 took final exam (3,000 per course)

8,174 earned course certificate (2,000 per course)

60-70% already had a post-secondary degree

30-40% were North American, with participants from nearly every country in the world.

Course development

None of the instructors had taught an online course before, but were supported by instructional designers, media development staff, and academic assistants (graduate and undergraduate students).

One major difference between UBC MOOCs and its online credit courses (which are primarily LMS-based) was the extensive use of video, the main component of the MOOC pilot courses.

Video production

305 videos constituting a total of 65 hours were produced. Each MOOC used a different method of production:

  • Intensive studio (Climate Literacy)
  • Hybrid studio plus instructor desktop (Systematic Program Design)
  • Light studio production (Game Theory I and II)
  • Instructor desktop (Useful Genetics)

Web pages

All the MOOCs except Games Theory also included weekly modules as HTML-based web pages, which is a variation of the Coursera design default model. Altogether 98 HTML module pages were developed. The weekly modules were used to provide guidance to students on learning goals, amount of work expected, an overview of activities, and additional quiz or assignment help. (All standard practice in UBC’s LMS-based credit courses.)

Assessment

1,049 quiz questions were developed, of which just over half were graded.

There were four peer assessments in total across all the MOOCs.

Course delivery

As well as the faculty member responsible for each MOOC, graduate and undergraduate academic assistants were a crucial component of all courses, with the following responsibilities:

  • directly assisting learners
  • troubleshooting technical problems
  • conducting quality assurance activities

There was very little one-on-one interaction between the main instructor and learners, but academic assistants monitored and moderated the online forum discussions.

Costs

As always, costing is a difficult exercise. Appendix B of the report gives a pilot total of $217,657, but this excludes academic assistance or, perhaps the most significant cost, instructor time.

Working from the video production costs ($95,350) and the proportion of costs (44%) devoted to video production in Figure 1 in the report, I estimate the direct cost at $216,700, or approximately $54,000 per MOOC, excluding faculty time and co-ordination support, but including academic assistance.

However, the range of cost is almost as important. The video production costs for Climate Literacy, which used intensive studio production, were more than six times the video production costs of Systematic Program Design (hybrid studio + desktop).

MOOCs as OERs

  • the UBC instructors are using their MOOC materials in their own on-campus, for-credit classes in a flipped classroom model
  • courses are left open and active on Coursera for self-paced learning
  • porting of video materials as open access YouTube videos
  • two courses (Climate Literacy and Useful Genetics) added Creative Commons licenses for re-use

Challenges

  • copyright clearance (Coursera owns the copyright so third party copyright needs to be cleared)
  • higher than expected time demands on all involved
  • iterative upgrades to the Coursera platform
  • partner relationship management (UBC + Coursera + Stanford University) was time-consuming.
  • training and managing academic assistants, especially time management
  • the Coursera platform limited instructors’ ability to develop desired course activities
  • Coursera’s peer assessment functionality in particular was limiting

Lessons

  • UBC’s prior experience in credit-based online learning led to better-designed, more interactive and more engaging MOOCs
  • learners always responded positively to instructor ‘presence’ in forums or course announcements
  • MOOC students were motivated by grades
  • MOOC students were willing to critically engage in critiquing instructors’ expertise and teaching
  • open publishing via MOOCs is a strong motivator for instructors
  • MOOCs require significant investment.

Conclusion

All the MOOCs received positive feedback and comments from students. UBC was able to gain direct experience in and knowledge of MOOCs and look at how this might inform both their for-credit on-campus and online teaching. UBC was also able to bring its experience in for-credit online learning to strengthening the design of MOOCs. Lastly it was able to make much more widely known the quality of UBC instructors and course materials.

Comment

First, congratulations to UBC for

  • experimenting with MOOCs
  • conducting the evaluation
  • making the report publicly available.

It is clear from the comments of participants in the appendices that at least some of the participants (we don’t know how many) were very pleased with the courses. As usual though with evaluation reports on MOOCs, there is no assessment of learning other than the end of course quiz-based tests. I don’t care too much about completion rates, but some measurement of student satisfaction would have been helpful.

It is also significant that UBC has now decided to move from Coursera to edX as its platform for MOOCs. edX, which is open source and allows partners to modify and adapt the platform, provides the flexibility that Coursera lacked, despite its many iterative ‘improvements’.

This also demonstrates the hubris of MOOC platform developers in ignoring best design principles in online learning when they designed their platforms. It is clear that UBC designers were able to improve the design of their MOOCs by drawing on prior for-credit online experience, but also that the MOOC platforms are still very limited in enabling the kind of learning activities that lead to student engagement and success.

The UBC report also highlighted the importance (and cost) of providing some form of learner support in course delivery. The use of academic assistants in particular clearly made the MOOCs more interactive and engaging, as well as limited but effective interventions from the instructors themselves, once again supported by (and confirming) prior research on the importance of instructor presence for successful for-credit online learning.

I very much appreciate the cost data provided by UBC, and the breakdown of production and delivery costs is extremely valuable, but I have to challenge the idea of providing any costs that exclude the time of the instructors. This is by far the largest and most important cost in MOOCs and the notion that MOOCs are free of instructor cost is to fly in the face of any respectable form of economics.

It is clear that MOOCs are more expensive to date per hour of study time than LMS-based for-credit online courses. We still do not have enough data to give a precise figure, and in any case, as the UBC study shows, cost is very much a factor of design. However, even without instructors costs, the UBC MOOCs at $54,000 each for between 8-12 weeks are already more than the average cost of a 13 week for-credit LMS-based online course, including instructor time.

This is partly due to the increased instructor time in preparation/production, but also to the higher cost of video production.  I am not against the use of video in principle, but it must add value. Using it for content transmission when this can be done so much more cheaply textually and/or by audio is a waste of the medium’s potential (although perhaps more motivating for the instructor).

More importantly, every institution contemplating MOOCs needs to do a cost-benefit exercise. Is it better to invest in MOOCs or credit-based online learning or both? If MOOCs are more expensive, what are the added benefits they provide and does this more than make up for not only the extra cost, but the lost opportunity of investing in (more) credit-based online learning or other forms of campus-based learning? I know what my answer would be.