May 30, 2015

Yale University to offer an online master of medical science

Listen with webReader

Yale University 2

Korn, M. (2015) Yale Will Offer Web-Based Master of Medical Science Degree Wall Street Journal, March 10

This for me is much more significant than the announcement of the first xMOOCs. It is a sign that even the elite Ivy League universities are recognising the validity of online learning for credit, even in the most demanding of subject areas, after ignoring or even denigrating online learning for many years.

However, before you all rush to sign up, note the sticker price: US$83,162 – the same cost as for the on-campus program, which has been limited to 40 students a year. One reason probably for such a high price for an online program is that Yale is contracting 2U Inc to help with the design and delivery of the program.

Another high cost factor is that the program requires hands-on clinical stints at field sites near students and at least three meetings on Yale’s New Haven, Conn., campus for activities such as cadaver dissection. Yale is aiming eventually for about 360 students across both the on-campus and online programs.

Yale’s move reinforces my view that there is still room for major expansion by top research universities into the online professional masters’ market. However, it will be important to price these at a level that makes them attractive to lifelong learners.

So good on Yale for leading the way for other Ivy League institutions. Now let’s hope someone else can do this at a more reasonable tuition fee (which in my view would be in the range of $15,000-$25,000 for the equivalent of a one year master’s program).

A new way to look at the costs of digital media in education

Listen with webReader
Image: © Ehrenberg, D., AlleyWatch.com, 2013

Image: © Ehrenberg, D., AlleyWatch.com, 2013

I said it was going to be fun looking at the costs of digital media in education, but it wasn’t. When I came to write this section, I thought it would be a breeze. I wrote about this topic as recently as 2005. All I needed to do is tweak it a little to bring it up to date, I thought.

However, there has truly been a revolution in the media available for teaching and learning in the last ten years, and this revolution has completely up-ended many of the assumptions about costs previously made in this field. Most of the research on costs of educational media had been done by people (like myself) working mainly in distance education, because that was where technology was being mainly used for teaching. That has all changed now: media have gone mainstream.

What is really interesting though is how little research there has been done on the costs of new digital media in education (MOOCs are a slight exception). Nevertheless, when I dug into the topic, I came to what struck me at first as an astonishing conclusion: the costs of media don’t matter any more in media selection. Use what suits best your educational purpose, because it’s all low cost now.

Of course, that is a gross over-simplification. Like many other topics in this area, straight comparisons between different media don’t work. What you have to look at are the conditions or factors that influence costs, across all media. That’s what I’ve tried to explore in this section. Remember I’m targeting teachers and instructors, not economists or instructional designers. So here goes, and please, let me have feedback on this (see the end of this rather long post):

9.4.1 A revolution in media

Until as recently as ten years ago, cost was a major discriminator affecting the choice of technology (Hülsmann, 2000, 2003; Rumble, 2001; Bates, 2005). For instance, for educational purposes, audio (lectures, radio, audio-cassettes) was far cheaper than print, which in turn was far cheaper than most forms of computer-based learning, which in turn was far cheaper than video (television, cassettes or video-conferencing). All these media were usually seen as either added costs to regular teaching, or too expensive to use to replace face-to-face teaching, except for purely distance education on a fairly large scale.

However, there have been dramatic reductions in the cost of developing and distributing all kinds of media (except face-to-face teaching) in the last ten years, due to several factors:

  • rapid developments in consumer technologies such as smart phones that enable text, audio and video to be both created and transmitted by end users at low cost
  • compression of digital media, enabling even high bandwidth video or television to be carried over wireless, landlines and the Internet at an economic cost (at least in economically advanced countries)
  • improvements in media software, making it relatively easy for non-professional users to create and distribute all kinds of media
  • increasing amounts of media-based open educational resources, which are already developed learning materials that are free for teachers and students alike to use.

The good news then is that in general, and in principle, cost should no longer be an automatic discriminator in the choice of media. If you are happy to accept this statement at face value, than you can skip the rest of this chapter. Choose the mix of media that best meets your teaching needs, and don’t worry about which medium is likely to cost more. Indeed, a good case could be made that it would now be cheaper to replace face-to-face teaching with purely online learning, if cost was the only consideration.

In practice however costs can vary enormously both between and within media, depending once again on context and design. Since the main cost from a teacher’s perspective is their time, it is important to know what are the ‘drivers’ of cost, that is, what factors are associated with increased costs, depending on the context and the medium being used. These factors are less influenced by new technological developments, and can therefore be seen as ‘foundational’ principles when considering the costs of educational media.

Unfortunately there are many different factors that can influence the actual cost of using media in education, which makes a detailed discussion of costs very complex. As a result, I will try to identify the main cost drivers, then provide a table that provides a simplified guide to how these factors influence the costs of different media, including face-to-face teaching. This guide again should be considered as a heuristic device. So see this chapter as Media Costs 101.

9.4.1 Cost categories

The main cost categories to be considered in using educational media and technologies, and especially blended or online learning, are as follows:

9.4.1.2 Development

These are the costs needed to pull together or create learning materials using particular media or technologies. There are several sub-categories of development costs:

  • production costs: making a video or building a course section in a learning management system. Included in these costs will be the time of specialist staff, such as web designers or audio-visual specialists, as well as any costs in web design or video production
  • your time as an instructor: the work you have to do as part of developing or producing materials.  This will include planning/course design as well as development. Your time is money, and probably the largest single cost in using educational technologies, but more importantly, if you are developing learning materials you are not doing other things, such as research or interacting with students, so there is a real cost, even if it is not expressed in dollar terms.
  • copyright clearance if you are using third party materials such as photos or video clips. Again, this is more likely to be thought of as time rather than money
  • probably the cost of an instructional designer in terms of their time

Development costs are usually fixed or ‘once only’ and are independent of the number of students. Once media are developed, they are usually scalable, in that once produced, they can be used by any number of learners without increased development costs. Using open educational resources can help reduce greatly media development costs.

9.4.1.3 Delivery

This includes the cost of the educational activities needed during offering the course and would include instructional time spent interacting with students, instructional time spent on marking assignments, and would include the time of other staff supporting delivery, such as teaching assistants, adjuncts for additional sections and instructional designers and technical support staff.

Because of the cost of human factors such as instructional time and technical support needed in media-based teaching, delivery costs tend to increase as student numbers increase, and also have to be repeated each time the course is on offer, i.e. they are recurrent. However, increasingly with Internet-based delivery, there is usually a zero direct technology cost in delivery.

9.4.1.4 Maintenance costs

Once materials for a course are created, they need to be maintained. Urls go dead, set readings may go out of print or expire, and more importantly new developments in the subject area may need to be accommodated. Thus once a course is offered, there are ongoing maintenance costs.

Instructional designers and/or media professionals can manage some of the maintenance, but nevertheless teachers or instructors will need to be involved with decisions about content replacement or updating. Maintenance is not usually a major time consumer for a single course, but if an instructor is involved in the design and production of several online courses, maintenance time can build to a significant amount.

Maintenance costs are usually independent of the number of students, but are dependent on the number of courses an instructor is responsible for, and are recurrent each year.

9.4.1.5 Overheads

These include infrastructure or overhead costs, such as the cost of licensing a learning management system, lecture capture technology and servers for video steaming. These are real costs but not ones that can be allocated to a single course but will be shared across a number of courses. Overheads are usually considered to be institutional costs and, although important, probably will not influence a teacher’s decision about which media to use, provided these services are already in place and the institution does not directly charge for such services.

9.4.2 Cost drivers

The primary factors that drive cost are

  • the development/production of materials,
  • the delivery of materials,
  • number of students/scalability
  • the experience of an instructor working with the medium
  • whether the instructor develops materials alone (self-development) or works with professionals

Production of technology-based materials such as a video program, or a Web site, is a fixed cost, in that it is not influenced by how many students take the course. However, production costs can vary depending on the design of the course. Engle (2014) showed that depending on the method of video production, the development costs for a MOOC could vary by a factor of six (the most expensive production method – full studio production – being six times that of an instructor self-recording on a laptop).

Nevertheless, once produced, the cost is independent of the number of students. Thus the more expensive the course to develop, the greater the need to increase student numbers to reduce the average cost per student. (Or put another way, the greater the number of students, the more reason to ensure that high quality production is used, whatever the medium). In the case of MOOCs (which tend to be almost twice as expensive to develop as an online course for credit using a learning management system – University of Ottawa, 2013) the number of learners is so great that the average cost per student is very small. Thus there are opportunities for economies of scale from the development of digital material, provided that student course enrolments can be increased (which may not always be the case). This can be described as the potential for the scalability of a medium.

Similarly, there are costs in teaching the course once the course is developed. These tend to be variable costs, in that they increase as class size increases. If student-teacher interaction, through online discussion forums and assignment marking, is to be kept to a manageable level, then the teacher-student ratio needs to be kept relatively low (for instance, between 1:25 to 1:40, depending on the subject area and the level of the course). The more students, the more time a teacher will need to spend on delivery, or additional contract instructors will need to be hired. Either way, increased student numbers generally will lead to increased costs. xMOOCs are an exception. Their main value proposition is that they do not provide direct learner support, so have zero delivery costs. However, this is probably the reason why such a small proportion of participants successfully complete MOOCs.

There may be benefits then for a teacher or for an institution in spending more money up front for interactive learning materials if this leads to less demand for teacher-student interaction. For instance, a mathematics course might be able to use automated testing and feedback and simulations and diagrams, and pre-designed answers to frequently asked questions, with less or even no time spent on individual assignment marking or communication with the teacher. In this case it may be possible to manage teacher-student ratios as high as 1:200 or more, without significant loss of quality.

Also, experience in using or working with a particular medium or delivery method is important. The first time an instructor uses a particular medium such as podcasting, it takes much longer than subsequent productions or offerings. Some media or technologies though need much more effort to learn to use than others. Thus a related cost driver is whether the instructor works alone (self-development) or works with media professionals. Self-developing materials will usually take longer for an instructor than working with professionals.

There are advantages in teachers and instructors working with media professionals when developing digital media. Media professionals will ensure the development of a quality product, and above all can save teachers or instructors considerable time, for instance through the choice of appropriate software, editing, and storage and streaming of digital materials. Instructional designers can help in suggesting appropriate applications of different media for different learning outcomes. Thus as with all educational design, a team approach is likely to be more effective, and working with other professionals will help control the time teachers and instructors spend on media development.

Lastly, design decisions are critical. Costs are driven by design decisions within a medium. For instance cost drivers are different between lectures and seminars (or lab classes) in face-to-face teaching. Similarly, video can be used just to record talking heads, as in lecture capture, or can be used to exploit the affordances of the medium (see Section 9. 5), such as demonstrating processes or location shooting. Computing has a wide and increasing range of possible designs, including online collaborative learning (OCL), computer-based learning, animations, simulations or virtual worlds. Social media are another group of media that also need to be considered.

Figure 9.4 attempts to capture the complexity of cost factors, focusing mainly on the perspective of a teacher or instructor making decisions. Again, this should be seen as a heuristic device, a way of thinking about the issue. Other factors could be added (such as social media, or maintenance of materials). I have given my own personal ratings for each cell, based on my experience. I have taken conventional teaching as a medium or ‘average’ cost, then ranked cells as to whether there is a higher or lower cost factor for the particular medium. Other readers may well rate the cells differently.

Figure 9. Drivers of cost for educational media

Figure 9.4 Drivers of cost for educational media

Thus although in particular the time it takes to develop and deliver learning using different technologies is likely to influence an instructor’s decision about what technology to use, it is not a simple equation. For instance, developing a good quality online course using a mix of video and text materials may take much more of the instructor’s time to prepare than if the course was offered through classroom teaching. However, the online course may take less time in delivery over several years, because students may be spending more time on task online, and less time in direct interaction with the instructor. Once again, we see that design is a critical factor in how costs are assessed.

In short, from an instructor perspective, time is the critical cost factor. Technologies that take a lot of time to use are less likely to be used than those that are easy to use and thus save time. But once again design decisions can greatly affect how much time teachers or instructors need to spend on any medium, and the ability of teachers and students to create their own educational media is becoming an increasingly important factor.

9.4.3 Issues for consideration

In recent years, university faculty have generally gravitated more to lecture capture for online course delivery, particularly in institutions where online or distance learning is relatively new, because it is ‘simpler’ to do than redesign and create mainly text based materials in learning management systems. Lecture capture also more closely resembles the traditional classroom method. Pedagogically though (depending on the subject area) it may be less effective than an online course using collaborative learning and online discussion forums, as we shall see in Section 9.5. Also, from an institutional perspective lecture capture has a much higher technology cost than a learning management system.

Also, students themselves can now use their own devices to create multimedia materials for project work or for assessment purposes in the form of e-portfolios. Media allow instructors, if they wish, to move a lot of the hard work in teaching and learning from themselves to the students. Media allow students to spend more time on task, and low cost, consumer media such as mobile phones or tablets enable students themselves to create media artefacts, enabling them to demonstrate their learning in concrete ways. This does not mean that instructor ‘presence’ is no longer needed when students are studying online, but it does enable a shift in where and how a teacher or instructor can spend their time in supporting learning.

9.4.4 Questions for consideration

You may be better answering these questions when you have read Section 9.5 on the affordances of media. However, I think you will find it interesting to answer these questions before reading Section 9.5, then compare your answers after you have read Section 9.5

  1. Are concerns about the possible cost/demands on your time influencing your decisions on what media to use? If so in what ways? Has this section on costs changed your mind?
  2. How much time do you spend preparing lectures? Could that time be better spent preparing learning materials, then using the time saved from delivering lectures on interaction with students (online and/or face-to-face)?
  3. What kind of help can you get in your institution from instructional designers and media professionals for media design and development? What media decisions will the answer to this question suggest to you? For instance, if you are in a k-12 school with little or no chance for professional support, what kind of media and design decisions are you likely to make?
  4. To what extent have you explored open educational resources in your subject area? Type in the name of your course or topic + OER into Google  and see what comes up. How would the availability of such free media influence the design of your teaching?
  5. If you were filling in the cells for Figure 9.4, what differences would there be with my entries? Why?
  6. In Figure 9.4, add the following media: e-portfolios (in computing) and add another section under computing: social media. Add blogs, wikis and cMOOCs. How would you fill in the cells for each of these for development, delivery, etc.? Are there other media you would also add?
  7. Do you agree with the statement: It would now be cheaper to replace face-to-face teaching with purely online learning, if cost was the only consideration? What are the implications for your teaching if this is really true? What considerations would still justify face-to-face teaching?

Feedback

Please! In particular:

  1. Are there more recent publications on the costs of different media (as distinct from online or blended learning in general) that I have missed and should include?
  2. How do you react to Figure 9.4? Is it a helpful way to think of the different conditions or factors that influence costs? If not, what approach would you take to this topic?
  3. How useful are the questions for consideration above (9.4.4) from an instructor’s perspective? Can you suggest better ones?
  4. Do you agree with the following statements:
    1. cost should no longer be an automatic discriminator in the choice of mediaChoose the mix of media that best meets your teaching needs, and don’t worry about which medium is likely to cost more.
    2. It would now be cheaper to replace face-to-face teaching with purely online learning, if cost was the only consideration. 
    3. university faculty have generally gravitated more to lecture capture for online course delivery, particularly in institutions where online or distance learning is relatively new, because it is ‘simpler’ to do than redesign and create mainly text based materials in learning management systems.
    4. Media allow students to spend more time on task, and low cost, consumer media such as mobile phones or tablets enable students themselves to create media artefacts, enabling them to demonstrate their learning in concrete ways. This does not mean that instructor ‘presence’ is no longer needed when students are studying online, but it does enable a shift in where and how a teacher or instructor can spend their time in supporting learning.
  5. Does this approach to the costs of digital media work for you? If not, what would you suggest?

Up next

The pedagogical affordances of different media:

  • text
  • audio
  • video
  • computing
  • social media
  • face-to-face teaching

References

Bates, A. (2005) Technology, e-Learning and Distance Education London/New York: Routledge

Engle, W. (2104)UBC MOOC Pilot: Design and Delivery Vancouver BC: University of British Columbia

Hülsmann, T. (2000) The Costs of Open Learning: A Handbook Oldenburg: Bibliotheks- und Informationssytem der Universität Oldenburg

Hülsmann, T. (2003) Costs without camouflage: a cost analysis of Oldenburg University’s  two graduate certificate programs offered  as part of the online Master of Distance Education (MDE): a case study, in Bernath, U. and Rubin, E., (eds.) Reflections on Teaching in an Online Program: A Case Study Oldenburg, Germany: Bibliothecks-und Informationssystem der Carl von Ossietsky Universität Oldenburg

Rumble, G. (2001) The Cost and Costing of Networked Learning Journal of Asynchronous Learning Networks, Volume 5, Issue 2

University of Ottawa (2013)Report of the e-Learning Working Group Ottawa ON: The University of Ottawa

Strengths and weaknesses of MOOCs: Part 3, branding and cost

Listen with webReader
The MOOC value proposition is that they can eliminate variable costs of course delivery. Image: © OpenTuition.com, 2014

The MOOC value proposition is that MOOCs can eliminate the variable costs of course delivery. Image: © OpenTuition.com, 2014

The story so far

This is the fifth in a series of posts from my open textbook, Teaching in a Digital Age. I have already published four extracts from the book on MOOCs:

In this post, I examine their value for branding, and their costs.

Branding

Hollands and Tirthali (2014) in their survey on institutional expectations for MOOCs, found that building and maintaining brand was the second most important reason for institutions launching MOOCs (the most important was extending reach, which can also be seen as partly a branding exercise). Institutional branding through the use of MOOCs has been helped by elite Ivy League universities such as Stanford, MIT and Harvard leading the charge, and by Coursera limiting access to its platform to only ‘top tier’ universities. This of course has led to a bandwagon effect, especially since many of the universities launching MOOCs had previously disdained to move into credit-based online learning. MOOCs provided a way for these elite institutions to jump to the head of the queue in terms of status as ‘innovators’ of online learning, even though they arrived late to the party.

It obviously makes sense for institutions to use MOOCs to bring to a much wider public their areas of specialist expertise, such as the University of Alberta offering a MOOC on dinosaurs, MIT on electronics, and Harvard on Ancient Greek Heroes. MOOCs certainly help to widen knowledge of the quality of an individual professor (who are usually delighted to reach more students in one MOOC than in a lifetime of on-campus teaching). MOOCs are also a good way to give a glimpse of the quality of courses and programs offered by an institution.

However, it is difficult to measure the real impact of MOOCs on branding. As Hollands and Tirthali put it:

While many institutions have received significant media attention as a result of their MOOC activities, isolating and measuring impact of any new initiative on brand is a difficult exercise. Most institutions are only just beginning to think about how to capture and quantify branding-related benefits.

In particular, these elite institutions do not need MOOCs to boost the number of applicants for their campus-based programs (none to date is willing to accept successful completion of a MOOC for admission to credit programs), since elite institutions have no difficulty in attracting already highly qualified students.

Furthermore, once every other institution starts offering MOOCs, the branding effect gets lost to some extent. Indeed, exposing poor quality teaching or course planning to many thousands can have a negative impact on an institution’s brand, as Georgia Institute of Technology found when one of its MOOCs crashed and burned (Jaschik, 2013). However, by and large, most MOOCs succeed in the sense of bringing an institution’s reputation in terms of knowledge and expertise to many more people than it would in any other form of teaching.

Costs and economies of scale

The main value proposition of MOOCs is that they are free to participants. Once again we shall see this is more true in principle than in practice, because MOOC providers may charge a range of fees, especially for assessment. Although MOOCs may be free for participants, they are not without substantial cost to the provider institutions. Also, there are large differences in the costs of xMOOCs and cMOOCs, the latter being generally much cheaper to develop, although there are still some opportunity or actual costs even for cMOOCs.

Once again, there is very little information to date on the actual costs of designing and delivering a MOOC. However, we do know what the main cost factors are in online and distance learning, from previous research by Rumble (2001) and Hülsmann (2003). Using similar costing methodology, I tracked and analysed the cost of an online masters program at the University of British Columbia over a seven year period (Bates and Sangrà, 2011). This program used mainly a learning management system as the core technology, with instructors both developing the course and providing online learner support and assessment, assisted where necessary by extra adjunct faculty for handling larger class enrolments.

Costs of online learning break down into several categories:

  • initial program planning
  • course development
  • course delivery
  • course maintenance
  • institutional overheads.

Within each of these categories, there are sub-categories, such as the cost of instructors, media production and delivery costs, instructional design, and the cost of producing and delivering support materials. Not all costs apply in all circumstances, of course.

I found in my analysis of the costs of the UBC program that in 2003, development costs were approximately $20,000 to $25,000 per course. However, over a seven year period course development constituted less than 15% of the total cost, and occurred mainly in the first year or so of the program. Delivery costs, which included providing online learner support and student assessment, constituted more than a third of the total cost, and of course continued each year the course was offered (see Figure 6.8 below). Thus in credit-based online learning, delivery costs tend to be more than double the development costs over the life of a program.

Figure 6.8: Costs of an online masters program over seven years

Figure 6.8: Costs of an online masters program over seven years (from Bates and Sangrà, 2011, p. 172)

The main difference between MOOCs, credit-based online teaching, and campus-based teaching is that in principle MOOCs eliminate all delivery costs, because MOOCs do not provide learner support or instructor-delivered assessment, although again in practice this is not always true.

We do not have enough cases at the moment to draw firm conclusions about the costs of MOOCs but we do have some data. The University of Ottawa estimated the cost of developing an  xMOOC, based on figures provided to the university by Coursera, and on their own knowledge of the cost of developing online courses for credit, at around $100,000 (University of Ottawa, 2013).

Engle (2014) has reported on the actual cost of five MOOCs from the University of British Columbia. (In essence, there were really four UBC MOOCs, as one was in two shorter parts.) There are two important features concerning the UBC MOOCs that do not necessarily apply to other MOOCs. First, the UBC MOOCs used a wide variety of video production methods, from full studio production to desktop recording, so development costs varied considerably, depending on the sophistication of the video production technique. Second, the UBC MOOCs made extensive use of paid academic assistants, who monitored discussions and adapted or changed course materials as a result of student feedback, so there were substantial delivery costs as well.

Appendix B of the UBC report gives a pilot total of $217,657, but this excludes academic assistance or, perhaps the most significant cost, instructor time. Academic assistance came to 25% of the overall cost in the first year (excluding the cost of faculty). Working from the video production costs ($95,350) and the proportion of costs (44%) devoted to video production in Figure 1 in the report, I estimate the direct cost at $216,700, or approximately $54,000 per MOOC, excluding faculty time and co-ordination support (i.e. excluding program administration and overheads), but including academic assistance. However, the range of cost is almost as important. The video production costs for the MOOC which used intensive studio production were more than six times the video production costs of one of the other MOOCs.

There is also clearly a large opportunity cost involved in offering xMOOCs. By definition, the most highly valued faculty are involved in offering MOOCs. In a large research university, such faculty are likely to have, at a maximum, a teaching load of four to six courses a year. Although most instructors volunteer to do MOOCs, their time is limited. Either it means dropping one credit course for at least one semester, equivalent to 25 or more of their teaching load, or xMOOC development and delivery replaces time spent doing research. Furthermore, unlike credit-based courses, which run from anywhere between five to seven years, MOOCs are often offered only once or twice.

However one looks at it, the cost of xMOOC development, without including the time of the MOOC instructor, tends to be almost double the cost of developing an online credit course using a learning management system, because of the use of video in MOOCs. If the cost of the instructor is included, xMOOC production costs come closer to three times that of a similar length online credit course, especially given the extra time faculty tend put in for such a public demonstration of their teaching in a MOOC. The full cost of an xMOOC then seems to be currently around $100,000, but we really need some good, reliable data to substantiate this estimate. There is no reason though why xMOOCs could not use cheaper production methods, such as an LMS instead of video, for content delivery, or using and re-editing video recordings of classroom lectures via lecture capture.

Without learner support or academic assistance, though, delivery costs for MOOCs are zero, and this is where the huge potential for savings exist. The issue then is whether MOOCs can succeed without the cost of learner support and human assessment, or more likely, whether MOOCs can substantially reduce delivery costs through automation without loss of quality in learner performance. There is no evidence to date though that they can do this, and prior research on the importance of instructor presence for successful online credit programs suggests that learner support and assessment remain a major challenge for MOOCs.

In terms of sustainable business models, the elite universities have been able to move into xMOOCs because of generous donations from private foundations and use of endowment funds, but these forms of funding are limited for most institutions. Coursera and Udacity have the opportunity to develop successful business models through various means, such as charging MOOC provider institutions for use of their platform, by collecting fees for badges or certificates, through the sale of participant data, through corporate sponsorship, or through direct advertising.

However, particularly for publicly funded universities or colleges, most of these sources of income are not available or permitted, so it is hard to see how they can begin to recover the cost of a substantial investment in MOOCs, even with ‘cannibalising’ MOOC material for on-campus use. Every time a MOOC is offered, this takes away resources that could be used for online credit programs. Thus institutions are faced with some hard decisions about where to invest their resources for online learning. The case for putting scarce resources into MOOCs is far from clear, unless some way can be found to give credit for successful MOOC completion.

Next

I hope to wrap up what has turned out to be a whole chapter on MOOCs in my next post in this series, which will analyse the political and economic reasons behind the rapid development of MOOCs, and which will also provide some brief conclusions about MOOCs as a design model.

Help!

1. I am very conscious of the limited amount of data on either the success of MOOCs for branding, or in particular the true costs of developing and delivering MOOCs. So any information on the cost of MOOCs that you can provide or direct me to will be very much appreciated.

2. Has anyone attempted to measure the value of MOOCs for ‘branding’ a university or college? How would you do this?

3. Is it reasonable to compare the costs of xMOOCs to the costs of online credit courses? Are they competing for the same funds, or are they categorically different in their funding source and goals? If so, how?

4. Could you make the case that cMOOCs are a better value proposition than xMOOCs – or are they again too different to compare? In particular has anyone got data on the true cost of cMOOCs? How would you cost them?

As always, feedback, criticisms and comments are welcome.

References

Bates, A. and Sangrà, A. (2011) Managing Technology in Higher Education San Francisco: Jossey-Bass/John Wiley and Co

Engle, W. (2104) UBC MOOC Pilot: Design and Delivery Vancouver BC: University of British Columbia

Hollands, F. and Tirthali, D. (2014) MOOCs: Expectations and Realities New York: Columbia University Teachers’ College

Hülsmann, T. (2003) Costs without camouflage: a cost analysis of Oldenburg University’s  two graduate certificate programs offered  as part of the online Master of Distance Education (MDE): a case study, in Bernath, U. and Rubin, E., (eds.) Reflections on Teaching in an Online Program: A Case Study Oldenburg, Germany: Bibliothecks-und Informationssystem der Carl von Ossietsky Universität Oldenburg

Jaschik, S. (2013) MOOC Mess, Inside Higher Education, February 4

Rumble, G. (2001) The costs and costing of networked learning, Journal of Asynchronous Learning Networks, Vol. 5, No. 2

University of Ottawa (2013) Report of the e-Learning Working Group Ottawa ON: The University of Ottawa

The strengths and weaknesses of MOOCs: Part I

Listen with webReader
© Carson Kahn, 2012

© Carson Kahn, 2012

How many times has an author cried: ‘Oh, God, I wish I’d never started on this!’? Well, I wanted to have a short section on MOOCs within a chapter on design models for teaching and learning in my online textbook, ‘Teaching in a Digital Age‘ and it is probably poetic justice that the section on MOOCs is now ballooning into a monster of its own.

Although I don’t want to inflate the importance of MOOCs, I fear I’m probably going to have to devote a whole chapter to the topic. (Well, I do have to agree that the topic is relevant to teaching in a digital age.) However, whether MOOCs get their own chapter may well depend on how you, my readers, react to what I’m writing, which I’m putting into this blog via a series of posts.

I’ve already had two posts, one on the key design features of MOOCs in general, and another on the differences between cMOOCs and xMOOCs that has already generated quite a lot of heated comments. Here I’m posting the first part of my discussion on the strengths and weaknesses of MOOCs. I’ll do another couple of posts to wrap it up (I desperately hope).

Strengths and weaknesses of MOOCs

Because at the time of writing most MOOCs are less than three years old, there are not many research publications on MOOCs, although research activities are now beginning to pick up. Much of the research so far on MOOCs comes from the institutions offering MOOCs, mainly in the form of reports on enrolments. The commercial platform providers such as Coursera and Udacity have provided limited research information overall, which is a pity, because they have access to really big data sets. However, MIT and Harvard, the founding partners in edX, are conducting some research, mainly on their own courses. There is very little research to date on cMOOCs, and what there is is mainly qualitative.

However, wherever possible, I have tried to use any research that has been done that provides insight into the strengths and weaknesses of MOOCs. At the same time, we should be clear that we are discussing a phenomenon that to date has been marked largely by political, emotional and often irrational discourse, and in terms of hard evidence, we will have to wait for some time. Thus any analysis must also address philosophical or value issues, which is a sure recipe for generating heated discussion.

Lastly, it should be remembered when evaluating MOOCs is that I am applying the criteria of whether MOOCs are likely to lead to the kinds of learning needed in a digital age: in other words, do they help develop the knowledge and skills defined in Chapter 1 of Teaching in a Digital Age?

1. Open and free education

MOOCs, particularly xMOOCs, deliver high quality content from some of the world’s best universities for free to anyone with a computer and an Internet connection. This in itself is an amazing value proposition. In this sense, MOOCs are an incredibly valuable addition to educational provision. Who could argue against this? Certainly not me, so long as the argument for MOOCs goes no further.

However, this is not the only form of open and free education. Libraries, open textbooks and educational broadcasting are also open and free to end users and have been for some time, even if they do not have the same power and reach as Internet-based delivery. There are also lessons we can learn from these earlier forms of open and free education that also apply to MOOCs.

The first is that these earlier forms of open and free did not replace the need for formal, credit-based education, but were used to supplement or strengthen it. In other words, MOOCs are a tool for continuing and informal education, which has high value in its own right.

The second lesson is that there have been many attempts in the past to use open and massive education through educational broadcasting and satellite broadcasting in Third World countries (see Bates, 1985), and they all failed miserably for a variety of reasons, the most important being:

  • the high cost of ground equipment (especially security),
  • the need for local support for learners without high levels of education, and its high cost
  • the need to adapt to the culture of the receiving countries
  • the difficulty of covering the operational costs of management and administration, especially for assessment, qualifications and local accreditation.

Also the priority in most Third World countries is not for courses from high-level Stanford University professors, but for programs for elementary and high schools. Finally, while mobile phones are widespread in Africa, they operate on very narrow bandwidths. For instance, it costs US$2 to download a typical YouTube video – equivalent to a day’s salary for many Africans. Streamed 50 minute video lectures then have limited applicability.

This is not to say that MOOCs could not be valuable in Third World countries. They have features, such as integrated interaction, testing and feedback, and much lower cost, that make them a more powerful medium than educational broadcasting but they will still face the same challenges of educational broadcasting:

  • being realistic as to what they can actually deliver to countries with no or limited technology infrastructure
  • working in partnership with Third World educational institutions and systems and other partners
  • ensuring that the necessary local support – which costs real money – is put in place
  • adapting the design, content and delivery of MOOCs to the cultural and economic requirements of those countries.

Also, MOOCs need to be compared to other possible ways of delivering mass education in developing countries, within these parameters. The problem comes when it is argued that because MOOCs are open and free to end-users, they will inevitably force down the cost of conventional education, or eliminate the need for it altogether, especially in Third World countries.

Lastly, and very importantly, in many countries, all public education is already in essence open to all and in many cases free to those participating, if grants, endowments and other forms of state support to students are taken into account. MOOCs then will have to deliver the same quality or better at a lower price than public education if they are to replace it. I will return to this point later when I discuss their costs and the political and social issues around MOOCs.

2. The audience that MOOCs mainly serve

In a research report from Ho et al. (2014), researchers at Harvard University and MIT found that on the first 17 MOOCs offered through edX, 66 per cent of all participants, and 74 per cent of all who obtained a certificate, had a bachelor’s degree or above, 71 per cent were male, and the average age was 26. This and other studies also found that a high proportion of participants came from outside the USA, ranging from 40-60 per cent of all participants, indicating strong interest internationally in open access to high quality university teaching.

In a study based on over 80 interviews in 62 institutions ‘active in the MOOC space’, Hollands and Tirthali (2014), researchers at Columbia University Teachers’ College, concluded that:

Data from MOOC platforms indicate that MOOCs are providing educational opportunities to millions of individuals across the world. However, most MOOC participants are already well-educated and employed, and only a small fraction of them fully engages with the courses. Overall, the evidence suggests that MOOCs are currently falling far short of “democratizing” education and may, for now, be doing more to increase gaps in access to education than to diminish them.

Thus MOOCs, as is common with most forms of university continuing education, cater to the better educated, older and employed sectors of society.

3. Persistence and commitment

Hill (2013) identified five types of participants in Coursera courses:

© Phil Hill, 2013

© Phil Hill, 2013

The edX researchers (Ho et al., 2014) provided empirical support for Hill’s analysis. They identified different levels of commitment as follows across 17 edX MOOCs:

  • Only Registered: Registrants who never access the courseware (35%).
  • Only Viewed: Non-certified registrants who access the courseware, accessing less than half of the available chapters (56%).
  • Only Explored: Non-certified Registrants who access more than half of the available chapters in the courseware, but did not get a certificate (4%).
  • Certified: Registrants who earn a certificate in the course (5%).

Engle (2014) found similar patterns for the UBC MOOCs on Coursera (also replicated in other studies):

  • of those that initially sign up, between one third and a half do not participate in any other active way
  • of those that participate in at least one activity, between 5-10% go on to successfully complete a certificate

Those going on to achieve certificates usually are within the 5-10 per cent range of those that sign up and in the 10-20 per cent range for those who actively engaged with the MOOC at least once. Nevertheless, the numbers obtaining certificates are still large in absolute terms: over 43,000 across 17 courses on edX and 8,000 across four courses at UBC (between 2,000-2,500 certificates per course).

Milligan et al. (2013) found a similar pattern of commitment in cMOOCs, from interviewing a relatively small sample of participants (29 out of 2,300 registrants) about halfway through a cMOOC:

  • passive participants: in Milligan’s study these were those that felt lost in the MOOC and rarely but occasionally logged in.
  • lurkers: they were actively following the course but did not engage in any of the activities (these were just under half those interviewed)
  • active participants (again, just under half those interviewed) who were fully engaged in the course activities.

MOOC participation and persistence rates need to be judged for what they are, a somewhat unique – and valuable – form of non-formal education. Once again, these results are very similar to research into non-formal educational broadcasts (e.g. the History Channel). One would not expect a viewer to watch every episode of a History Channel series then take an exam at the end. Ho et al. (p.13) produced the following diagram to show the different levels of commitment to xMOOCs:

Ho et al., 2014

Ho et al., 2014

Now compare that to what I wrote in 1985 about educational broadcasting in Britain:

(p.99): At the centre of the onion is a small core of fully committed students who work through the whole course, and, where available, take an end-of-course assessment or examination. Around the small core will be a rather larger layer of students who do not take any examination but do enrol with a local class or correspondence school. There may be an even larger layer of students who, as well as watching and listening, also buy the accompanying textbook, but who do not enrol in any courses. Then, by far the largest group, are those that just watch or listen to the programmes. Even within this last group, there will be considerable variations, from those who watch or listen fairly regularly, to those, again a much larger number, who watch or listen to just one programme. 

I also wrote (p.100):

A sceptic may say that the only ones who can be said to have learned effectively are the tiny minority that worked right through the course and successfully took the final assessment…A counter argument would be that broadcasting can be considered successful if it merely attracts viewers or listeners who might otherwise have shown no interest in the topic; it is the numbers exposed to the material that matter…the key issue then is whether broadcasting does attract to education those who would not otherwise have been interested, or merely provides yet another opportunity for those who are already well educated…There is a good deal of evidence that it is still the better educated in Britain and Europe that make the most use of non-formal educational broadcasting.

Exactly the same could be said about MOOCs. In a digital age where easy and open access to new knowledge is critical for those working in knowledge-based industries, MOOCs will be one valuable source or means of accessing that knowledge. The issue is though whether there are more effective ways to do this.

Furthermore, percentages, completion and certification DO matter if MOOCs are being seen as a substitute or a replacement for formal education. Thus MOOCs are a useful – but not really revolutionary – contribution to non-formal continuing education. We need though to look at whether they can meet the demands of more formal education, in terms of ensuring as many students succeed as possible.

To come

I think that’s more than enough for today. In my next post, I will try to cover the following strengths and weaknesses of MOOCs:

4. What do participants learn in MOOCs?

5. Costs and economies of scale

6. Branding

7. Ethical issues

8. Meeting the needs of learners in a digital age.

I will probably then do another short post on:

a. The politico-economic context that drives the MOOC phenomena

b. a short summary.

Over to you

Remembering that this is less than half the section on strengths and weaknesses, and that the criterion I am using for this is the ability of MOOCs to meet the learning needs of a digital age:

1. Are these the right topics for assessing MOOC’s strengths and weaknesses?

2. Would you have discussed these three topics differently? Do you agree or disagree with my conclusions?

3. Is ‘the ability of MOOCs to meet the learning needs of a digital age’ a fair criterion and if not how should they be judged?

4. Is the educational broadcasting comparison fair or relevant?

References

Bates, A. (1985) Broadcasting in Education: An Evaluation London: Constables

Engle, W. (2104) UBC MOOC Pilot: Design and Delivery Vancouver BC: University of British Columbia

Friedland, T. (2013) Revolution hits the universities, New York Times, January 26

Hill, P. (2013) Some validation of MOOC student patterns graphic, e-Literate, August 30

Ho, A. et al. (2014) HarvardX and MITx: The First Year of Open Online Courses Fall 2012-Summer 2013 (HarvardX and MITx Working Paper No. 1), January 21

Hollands, F. and Tirthali, D. (2014) MOOCs: Expectations and Reality New York: Columbia University Teachers’ College, Center for Benefit-Cost Studies of Education, 211 pp

Milligan, C., Littlejohn, A. and Margaryan, A. (2013) Patterns of engagement in connectivist MOOCs, Merlot Journal of Online Learning and Teaching, Vol. 9, No. 2

Yousef, A. et al. (2014) MOOCs: A Review of the State-of-the-Art Proceedings of 6th International Conference on Computer Supported Education – CSEDU 2014, Barcelona, Spain

What UBC has learned about doing MOOCs

Listen with webReader

Coursera certificate 2

Engle, W. (2104) UBC MOOC Pilot: Design and Delivery Vancouver BC: University of British Columbia

The University of British Columbia, a premier public research university in Canada, successfully delivered five MOOCs in the spring and summer of 2013, using the Coursera platform. This report is an evaluation of the experience.

The report is particularly valuable because it provides details of course development and delivery, including media used and costs. Also UBC has been developing online courses for credit for almost 20 years, so it is interesting to see how this has impacted on the design of their MOOCs.

The MOOCs

1. Game Theory I: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

2. Game Theory II: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

3. Useful Genetics: R. Redfield, UBC

4. Climate Literacy: S. Harris and S. Burch, UBC

5. Introduction to Systematic Program Design: G. Kizcales, UBC

In terms of comparability I’m going to treat Game Theory I and II as one MOOC, as combined they were about the same length as the other MOOCs (between 8-12 weeks)

Basic statistics

330,150 signed up (82,500 on average per course)

164,935 logged in at least once (41,000 per course)

12,031 took final exam (3,000 per course)

8,174 earned course certificate (2,000 per course)

60-70% already had a post-secondary degree

30-40% were North American, with participants from nearly every country in the world.

Course development

None of the instructors had taught an online course before, but were supported by instructional designers, media development staff, and academic assistants (graduate and undergraduate students).

One major difference between UBC MOOCs and its online credit courses (which are primarily LMS-based) was the extensive use of video, the main component of the MOOC pilot courses.

Video production

305 videos constituting a total of 65 hours were produced. Each MOOC used a different method of production:

  • Intensive studio (Climate Literacy)
  • Hybrid studio plus instructor desktop (Systematic Program Design)
  • Light studio production (Game Theory I and II)
  • Instructor desktop (Useful Genetics)

Web pages

All the MOOCs except Games Theory also included weekly modules as HTML-based web pages, which is a variation of the Coursera design default model. Altogether 98 HTML module pages were developed. The weekly modules were used to provide guidance to students on learning goals, amount of work expected, an overview of activities, and additional quiz or assignment help. (All standard practice in UBC’s LMS-based credit courses.)

Assessment

1,049 quiz questions were developed, of which just over half were graded.

There were four peer assessments in total across all the MOOCs.

Course delivery

As well as the faculty member responsible for each MOOC, graduate and undergraduate academic assistants were a crucial component of all courses, with the following responsibilities:

  • directly assisting learners
  • troubleshooting technical problems
  • conducting quality assurance activities

There was very little one-on-one interaction between the main instructor and learners, but academic assistants monitored and moderated the online forum discussions.

Costs

As always, costing is a difficult exercise. Appendix B of the report gives a pilot total of $217,657, but this excludes academic assistance or, perhaps the most significant cost, instructor time.

Working from the video production costs ($95,350) and the proportion of costs (44%) devoted to video production in Figure 1 in the report, I estimate the direct cost at $216,700, or approximately $54,000 per MOOC, excluding faculty time and co-ordination support, but including academic assistance.

However, the range of cost is almost as important. The video production costs for Climate Literacy, which used intensive studio production, were more than six times the video production costs of Systematic Program Design (hybrid studio + desktop).

MOOCs as OERs

  • the UBC instructors are using their MOOC materials in their own on-campus, for-credit classes in a flipped classroom model
  • courses are left open and active on Coursera for self-paced learning
  • porting of video materials as open access YouTube videos
  • two courses (Climate Literacy and Useful Genetics) added Creative Commons licenses for re-use

Challenges

  • copyright clearance (Coursera owns the copyright so third party copyright needs to be cleared)
  • higher than expected time demands on all involved
  • iterative upgrades to the Coursera platform
  • partner relationship management (UBC + Coursera + Stanford University) was time-consuming.
  • training and managing academic assistants, especially time management
  • the Coursera platform limited instructors’ ability to develop desired course activities
  • Coursera’s peer assessment functionality in particular was limiting

Lessons

  • UBC’s prior experience in credit-based online learning led to better-designed, more interactive and more engaging MOOCs
  • learners always responded positively to instructor ‘presence’ in forums or course announcements
  • MOOC students were motivated by grades
  • MOOC students were willing to critically engage in critiquing instructors’ expertise and teaching
  • open publishing via MOOCs is a strong motivator for instructors
  • MOOCs require significant investment.

Conclusion

All the MOOCs received positive feedback and comments from students. UBC was able to gain direct experience in and knowledge of MOOCs and look at how this might inform both their for-credit on-campus and online teaching. UBC was also able to bring its experience in for-credit online learning to strengthening the design of MOOCs. Lastly it was able to make much more widely known the quality of UBC instructors and course materials.

Comment

First, congratulations to UBC for

  • experimenting with MOOCs
  • conducting the evaluation
  • making the report publicly available.

It is clear from the comments of participants in the appendices that at least some of the participants (we don’t know how many) were very pleased with the courses. As usual though with evaluation reports on MOOCs, there is no assessment of learning other than the end of course quiz-based tests. I don’t care too much about completion rates, but some measurement of student satisfaction would have been helpful.

It is also significant that UBC has now decided to move from Coursera to edX as its platform for MOOCs. edX, which is open source and allows partners to modify and adapt the platform, provides the flexibility that Coursera lacked, despite its many iterative ‘improvements’.

This also demonstrates the hubris of MOOC platform developers in ignoring best design principles in online learning when they designed their platforms. It is clear that UBC designers were able to improve the design of their MOOCs by drawing on prior for-credit online experience, but also that the MOOC platforms are still very limited in enabling the kind of learning activities that lead to student engagement and success.

The UBC report also highlighted the importance (and cost) of providing some form of learner support in course delivery. The use of academic assistants in particular clearly made the MOOCs more interactive and engaging, as well as limited but effective interventions from the instructors themselves, once again supported by (and confirming) prior research on the importance of instructor presence for successful for-credit online learning.

I very much appreciate the cost data provided by UBC, and the breakdown of production and delivery costs is extremely valuable, but I have to challenge the idea of providing any costs that exclude the time of the instructors. This is by far the largest and most important cost in MOOCs and the notion that MOOCs are free of instructor cost is to fly in the face of any respectable form of economics.

It is clear that MOOCs are more expensive to date per hour of study time than LMS-based for-credit online courses. We still do not have enough data to give a precise figure, and in any case, as the UBC study shows, cost is very much a factor of design. However, even without instructors costs, the UBC MOOCs at $54,000 each for between 8-12 weeks are already more than the average cost of a 13 week for-credit LMS-based online course, including instructor time.

This is partly due to the increased instructor time in preparation/production, but also to the higher cost of video production.  I am not against the use of video in principle, but it must add value. Using it for content transmission when this can be done so much more cheaply textually and/or by audio is a waste of the medium’s potential (although perhaps more motivating for the instructor).

More importantly, every institution contemplating MOOCs needs to do a cost-benefit exercise. Is it better to invest in MOOCs or credit-based online learning or both? If MOOCs are more expensive, what are the added benefits they provide and does this more than make up for not only the extra cost, but the lost opportunity of investing in (more) credit-based online learning or other forms of campus-based learning? I know what my answer would be.