October 24, 2014

Review of ‘Online Distance Education: Towards a Research Agenda.’

Listen with webReader
Drop-out: the elephant in the DE room that no-one wants to talk about

Drop-out: the elephant in the DE room that no-one wants to talk about

Zawacki-Richter, O. and Anderson, T. (eds.) (2014) Online Distance Education: Towards a Research Agenda Athabasca AB: AU Press, pp. 508

It is somewhat daunting to review a book of over 500 pages of research on any topic. I doubt if few other than the editors are likely to read this book from cover to cover. It is more likely to be kept on one’s bookshelf (if these still exist in a digital age) for reference whenever needed. Nevertheless, this is an important work that anyone working in online learning needs to be aware of, so I will do my best to cover it as comprehensively as I can.

Structure of the book

The book is a collection of about 20 chapters by a variety of different authors (more on the choice of authors later). Based on a Delphi study and analysis of ‘key research journals’ in the field, the editors have organized the topic into three sections, with a set of chapters on each sub-section, as follows:

1. Macro-level research: distance education systems and theories

  • access, equity and ethics
  • globalization and cross-cultural issues
  • distance teaching systems and institutions
  • theories and models
  • research methods and knowledge transfer

2. Meso-level research: management, organization and technology

  • management and organization
  • costs and benefits
  • educational technology
  • innovation and change
  • professional development and faculty support
  • learner support services
  • quality assurance

3. Micro-level: teaching and learning in distance education

  • instructional/learning design
  • interaction and communication
  • learner characteristics.

In addition, there is a very useful preface from Otto Peters, an introductory chapter by the editors where they justify their structural organization of research, and a short conclusion that calls for a systematic research agenda in online distance education research.

More importantly, perhaps, Terry Anderson and Olaf Zawacki-Richter demonstrate empirically that research in this field has been skewed towards micro-level research (about half of all publications).  Interestingly, and somewhat surprisingly given its importance, costs and benefits of online distance education is the least researched area.

What I liked

It is somewhat invidious to pick out particular chapters, because different people will have different interests from such a wide-ranging list of topics. I have tended to choose those that I found were new and/or particularly enlightening for me, but other readers’ choices will be different. However, by selecting a few excellent chapters, I hope to give some idea of the quality of the book.

1. The structuring/organization of research

Anderson and Zawacki-Richter have done an excellent job in providing a structural framework for research in this field. This will be useful both for those teaching about online and distance education but in particular for potential Ph.D. students wondering what to study. This book will provide an essential starting point.

2. Summary of the issues in each area of research

Again, the editors have done an excellent job in their introductory chapter in summarizing the content of each of the chapters that follows, and in so doing pulling out the key themes and issues within each area of research. This alone makes the book worthwhile.

3. Globalization, Culture and Online Distance Education

Charlotte (Lani) Gunawardena of the University of New Mexico has written the most comprehensive and deep analysis of this issue that I have seen, and it is an area in which I have a great deal of interest, since most of the online teaching I have done has been with students from around the world and sometimes multi-lingual.

After a general discussion of the issue of globalization and education, she reviews research in the following areas:

  • diverse educational expectations
  • learners and preferred ways of learning
  • socio-cultural environment and online interaction
  • help-seeking behaviours
  • silence
  • language learning
  • researching culture and online distance learning

This chapter should be required reading for anyone contemplating teaching online.

4. Quality assurance in Online Distance Education

I picked this chapter by Colin Latchem because he is so deeply expert in this field that he is able to make what can be a numbingly boring but immensely important topic a fun read, while at the same time ending with some critical questions about quality assurance. In particular Latchem looks at QA from the following perspectives:

  • definitions of quality
  • accreditation
  • online distance education vs campus-based teaching
  • quality standards
  • transnational online distance education
  • open educational resources
  • costs of QA
  • is online distance education yet good enough?
  • an outcomes approach to QA.

This chapter definitely showcases a master at the top of his game.

5. The elephant in the room: student drop-out

This is a wonderfully funny but ultimately serious argument between Ormond Simpson and Alan Woodley about the elephant in the distance education room that no-one wants to mention. Here they start poking the elephant with some sticks (which they note is not likely to be a career-enhancing move.) The basic argument is that institutions should and could do more to reduce drop-out/increase course completion. This chapter also stunned me with providing hard data about really low completion rates for most open university students. I couldn’t help comparing these with the high completion rates for online credit courses at dual-mode (campus-based) institutions, at least in Canada (which of course are not ‘open’ institutions in that students must have good high school qualifications.)

Woodley’s solution to reducing drop-out is quite interesting (and later well argued):

  • make it harder to get in
  • make it harder to get out

In both cases, really practical and not too costly solutions are offered that nevertheless are consistent with open access and high quality teaching.

In summary

The book contains a number of really good chapters that lay out the issues in researching online distance education.

What I disliked

I have to say that I groaned when I first saw the list of contributors. The same old, same old list of distance education experts with a heavy bias towards open universities. Sure, they are nearly all well-seasoned experts, and there’s nothing wrong with that per se (after all, I see myself as one of them.)

But where are the young researchers here, and especially the researchers in open educational resources, MOOCs, social media applications in online learning, and above all researchers from the many campus-based universities now mainstreaming online learning? There is almost nothing in the book about research into blended learning, and flipped classrooms are not even mentioned. OK, the book is about online distance learning but the barriers or distinctions are coming down with a vengeance. This book will never reach those who most need it, the many campus-based instructors now venturing for the first time into online learning in one way or another. They don’t see themselves as primarily distance educators.

And a few of the articles were more like lessons in history than an up-to-date review of research in the field. Readers of this blog will know that I strongly value the history of educational technology and distance learning. But these lessons need to be embedded in the here and now. In particular, the lessons need to be spelled out. It is not enough to know that Stanford University researchers as long ago as 1974 were researching the costs and benefits of educational broadcasting in developing countries, but what lessons does this have for some of the outrageous claims being made about MOOCs? A great deal in fact, but this needs explaining in the context of MOOCs today.

Also the book is solely focused on post-secondary university education. Where is the research on online distance education in the k-12/school sector or the two-year college/vocational sector? Maybe they are topics for other books, but this is where the real gap exists in research publications in online learning.

Lastly, although the book is reasonably priced for its size (C$40), and is available as an e-text as well as the fully printed version, what a pity it is not an open textbook that could then be up-dated and crowd-sourced over time.

Conclusion

This is essential reading for anyone who wants to take a professional, evidence-based approach to online learning (distance or otherwise). It will be particularly valuable for students wanting to do research in this area. The editors have done an incredibly good job of presenting a hugely diverse and scattered area in a clear and structured manner. Many of the chapters are gems of insight and knowledge in the field.

However, we have a huge challenge of knowledge transfer in this field. Repeatedly authors in the book lamented that many of the new entrants to online learning are woefully ignorant of the research previously done in this field. We need a better way to disseminate this research than a 500 page printed text that only those already expert in the field are likely to access. On the other hand, the book does provide a strong foundation from which to find better ways to disseminate this knowledge. Knowledge dissemination in a digital world then is where the research agenda for online learning needs to focus.

 

Developing a coherent approach to mobile learning

Listen with webReader
The JIBC Emergency Social Services app

The JIBC Emergency Social Services app

The Justice Institute of British Columbia (JIBC) is an unusual organization, focusing on the training of police, fire, corrections and paramedical personnel, as well as providing training for social service departments of the BC provincial government.

Tannis Morgan, Associate Dean, JIBC Centre for Teaching, Learning and Innovation, has just posted a very interesting post, Mobile Learning at an Applied Institution, on why and how JIBC has developed its mobile learning strategy, which includes issuing pre-loaded iPads to participants in some programs. If you have any interest in mobile learning, the post is well worth reading. You should click on the graphic at the end of the post under ‘Presentation for ETUG 2014′ for more information from slides prepared for an ETUG workshop.

Particularly worth noting is that most of JIBC’s mobile applications are open and free, because the materials are of direct value to all personnel working in this area, whether they are taking a JIBC course or not. Making the material developed for clients such as the government’s emergency social services department open and free has resulted in more enrolments, as employees and managers see the value of the service that JIBC is providing.

The role of communities of practice in a digital age

Listen with webReader
Bank of America's Vital Voices progam links women executives of small and medium sized enterprises  Image: © Belfast Telegraph, 2014

Bank of America’s Vital Voices progam links women executives of small and medium sized enterprises from around the world
Image: © Belfast Telegraph, 2014

The story so far

I have published the first five chapters of my open textbook, ‘Teaching in a Digital Age‘.  I am now working on Chapter 6, ‘Models for Designing Teaching and Learning.’

In my last three posts I discussed respectively the appropriateness for a digital age of the classroom model, the ADDIE model, and the competency-based learning model. In this post, I explore the learning model based on communities of practice.

The theories behind communities of practice

The design of teaching often integrates different theories of learning. Communities of practice are one of the ways in which experiential learning, social constructivism, and connectivism can be combined, illustrating the limitations of trying to rigidly classify learning theories. Practice tends to be more complex.

What are communities of practice?

Definition: 

Communities of practice are groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly.

Wenger, 2014

The basic premise behind communities of practice is simple: we all learn in everyday life from the communities in which we find ourselves. Communities of practice are everywhere. Nearly everyone belongs to some community of practice, whether it is through our working colleagues or associates, our profession or trade, or our leisure interests, such as a book club. Wenger (2000) argues that a community of practice is different from a community of interest or a geographical community in that it involves a shared practice: ways of doing things that are shared to some significant extent among members.

Wenger argues that there are three crucial characteristics of a community of practice:

  • domain: a common interest that connects and holds together the community
  • community: a community is bound by the shared activities they pursue (for example, meetings, discussions) around their common domain
  • practice: members of a community of practice are practitioners; what they do informs their participation in the community; and what they learn from the community affects what they do.

Wenger (2000) has argued that although individuals learn through participation in a community of practice, more important is the generation of newer or deeper levels of knowledge through the sum of the group activity. If the community of practice is centered around business processes, for instance, this can be of considerable benefit to an organization. Smith (2003) notes that:

…communities of practice affect performance..[This] is important in part because of their potential to overcome the inherent problems of a slow-moving traditional hierarchy in a fast-moving virtual economy. Communities also appear to be an effective way for organizations to handle unstructured problems and to share knowledge outside of the traditional structural boundaries. In addition, the community concept is acknowledged to be a means of developing and maintaining long-term organizational memory.

Brown and Duguid (2000) describe a community of practice developed around the Xerox customer service representatives who repaired the machines in the field. The Xerox reps began exchanging tips and tricks over informal meetings at breakfast or lunch and eventually Xerox saw the value of these interactions and created the Eureka project to allow these interactions to be shared across the global network of representatives. The Eureka database has been estimated to have saved the corporation $100 million. Companies such as Google and Apple are encouraging communities of practice through the sharing of knowledge across their many specialist staff.

Technology provides a wide range of tools that can support communities of practice, as indicated by Wenger (2010) in the diagram below:

Image: © Etienne Wenger, 2010

Designing effective communities of practice

Most communities of practice have no formal design and tend to be self-organising systems. They have a natural life cycle, and come to an end when they no longer serve the needs of the community. However, there is now a body of theory and research that has identified actions that can help sustain and improve the effectiveness of communities of practice.

 Wenger, McDermott and Snyder (2002) have identified seven key design principles for creating effective and self-sustaining communities of practice, related specifically to the management of the community, although the ultimate success of a community of practice will be determined by the activities of the members of the community themselves. Designers of a community of practice need to:

  1. Design for evolution: ensuring that the community can evolve and shift in focus to meet the interests of the participants without moving too far from the common domain of interest
  2. Open a dialogue between inside and outside perspectives: encourage the introduction and discussion of new perspectives that come or are brought in from outside the community of practice
  3. Encourage and accept different levels of participation, The strength of participation varies from participant to participant. The ‘core’ (most active members) are those who participate regularly. There are others who follow the discussions or activities but do not take a leading role in making active contributions. Then there are those (likely the majority) who are on the periphery of the community but may become more active participants if the activities or discussions start to engage them more fully. All these levels of participation need to be accepted and encouraged within the community.
  4. Develop both public and private community spaces: communities of practice are strengthened if they encourage individual or group activities that are more personal or private as well as the more public general discussions; for instance, individuals may decide to blog about their activities, or in a larger online community of practice a small group that live or work close together may also decide to meet informally on a face-to-face basis
  5. Focus on value. Attempts should be made explicitly to identify, through feedback and discussion, the contributions that the community most values, then focus the discussion and activities around these issues.
  6. Combine familiarity and excitement, by focusing both on shared, common concerns and perspectives, but also by introducing radical or challenging perspectives for discussion or action
  7. Create a rhythm for the community: there needs to be a regular schedule of activities or focal points that bring participants together on a regular basis, within the constraints of participants’ time and interests.

Subsequent research has identified a number of critical factors that influence the effectiveness of participants in communities of practice, These include being:

  • aware of social presence: individuals need to feel comfortable in engaging socially with other professionals or ‘experts’ in the domain, and those with greater knowledge must be willing to share in a collegial manner that respects the views and knowledge of other participants (social presence is defined as the awareness of others in an interaction combined with an appreciation of the interpersonal aspects of that interaction.)
  • motivated to share information for the common good of the community
  • able and willing to collaborate.

EDUCAUSE has developed a step-by-step guide for designing and cultivating communities of practice in higher education (Cambridge, Kaplan and Suter, 2005).

Lastly, research on other related sectors, such as collaborative learning or MOOCs, can inform the design and development of communities of practice. For instance, communities of practice need to balance between structure and chaos: too much structure and many participants are likely to feel constrained in what they need to discuss; too little structure and participants can quickly lose interest or become overwhelmed.

Many of the other findings about group and online behaviour, such as the need to respect others, observing online etiquette, and preventing certain individuals from dominating the discussion, are all likely to apply. However, because many communities of practice are by definition self-regulating, establishing rules of conduct and even more so enforcing them is really a responsibility of the participants themselves.

Learning through communities of practice in a digital age

Communities of practice are a powerful manifestation of informal learning. They generally evolve naturally to address commonly shared interests and problems. By their nature, they tend to exist outside formal educational organisations. Participants are not usually looking for formal qualifications, but to address issues in their life and to be better at what they do. Furthermore, communities of practice are not dependent on any particular medium; participants may meet face-to-face socially or at work, or they can participate in online or virtual communities of practice.

It should be noted that communities of practice can be very effective in a digital world, where the working context is volatile, complex, uncertain and ambiguous.  A large part of the lifelong learning market will become occupied by communities of practice and self-learning, through collaborative learning, sharing of knowledge and experience, and crowd-sourcing new ideas and development. Such informal learning provision will be particularly valuable for non-governmental or charitable organizations, such as the Red Cross, Greenpeace or UNICEF, or local government, looking for ways to engage communities in their areas of operation.

These communities of learners will be open and free, and hence will provide a competitive alternative to the high priced lifelong learning programs being offered by research universities. This will put pressure on universities and colleges to provide more flexible arrangements for recognition of informal learning, in order to hold on to their current monopoly of post-secondary accreditation.

One of the significant developments in recent years has been the use of massive open online courses (MOOCs) for developing online communities of practice. To date the focus of the majority of MOOCs from providers such as Coursera, Udacity and edX, has been on academic ‘courses’, on topics such as artificial intelligence or dinosaurs, which do have a widespread interest. However, these more instructionist MOOCs are not really developed as communities of practice, because  they use mainly a transmissive pedagogy, from experts to those considered less expert. Even though there may be massive numbers participating in online forums, they are not constructed to maximise the contributions from the participants (despite the fact that most MOOC participants already have high levels of education.). Indeed there is evidence that in really large instructionist MOOCs, participants feel overwhelmed by the magnitude and lack of structuring of the participant contributions (see for instance Knox, 2014).

In comparison, connectivist MOOCs are an ideal way to bring together specialists scattered around the world to focus on a common interest or domain. Connectivist MOOCs are much closer to being virtual communities of practice, in that they put much more emphasis on sharing knowledge between more or less equal participants. However, current connectivist MOOCs do not always incorporate what research indicates are best practices for developing communities of practice, and those wanting to establish a virtual community of practice at the moment need some kind of MOOC provider to get them started and give them access to the necessary MOOC software.

In the long run, MOOCs need to evolve to the point where it is possible for those with a common interest to easily create their own open, online communities of practice. As open source MOOC platforms evolve, it should become easier for people without computer science degrees to create and more importantly manage their own MOOCs, without having to go through a MOOC provider such as Coursera or edX. Also, there are other simpler tools, such as wikis, or more complex ones, such as virtual worlds, that may in the long run have more potential for virtual communities of practice created and organised by the participants themselves.

Although communities of practice are likely to become more rather than less important in a digital age, it is probably a mistake to think of them as a replacement for traditional forms of education. There is no single, ‘right’ approach to the design of teaching. Different groups have different needs. Communities of practice are more of an alternative for certain kinds of learners, such as lifelong learners, and are likely to work best when participants already have some domain knowledge and can contribute personally and in a constructive manner – which suggests the need for at least some form of prior general education or training for those participating in effective communities of practice.

In conclusion, it is clear is that in an increasingly volatile, uncertain, complex, and ambiguous world, and given the openness of the Internet, the social media tools now available, and the need for sharing of knowledge on a global scale, virtual communities of practice will become even more common and important. Smart educators and trainers will look to see how they can harness the strength of this design model, particularly for lifelong learning. However, merely lumping together large numbers of people with a common interest is unlikely to lead to effective learning.  Attention needs to be paid to those design principles that lead to effective communities of practice.

Over to you

Once again, I’m not an expert on communities of practice, so feedback on what I have written on this model of learning will be much appreciated. In particular:

  • Have a got it wrong? Are their important elements missing?
  • Is there good research on the design of communities of practice that I have missed?
  • Do you agree that they are NOT a replacement for other forms of education?
  • Can you really design a community of practice or do they just evolve naturally? If so, what conditions are needed for success other than those already discussed in this post?
  • How would you evaluate the success of a community of practice? What would you look for? How could this be identified or described?

I would love to hear from anyone who has attempted to create communities of practice and what design elements they would recommend.

References

Brown, J. and Duguid, Paul (2000). “Balancing act: How to capture knowledge without killing it”Harvard Business Review.

Cambridge, D., Kaplan, S. and Suter, V. (2005) Community of Practice Design Guide Louisville  CO: EDUCAUSE

Knox, J. (2004) Digital culture clash: “massive” education in the E-learning and Digital Cultures MOOC Distance Education, Vol. 35, No. 2

Wenger, E. (2000) Communities of Practice: Learning, Meaning and Identity Cambridge UK: Cambridge University Press

Wenger, E. (2014) Communities of practice: a brief introduction, accessed 26 September, 2014

Wenger, E, McDermott, R., and Snyder, W. (2002). Cultivating Communities of Practice (Hardcover). Harvard Business Press; 1 edition.

 

Writing an open textbook: a mid-term report on the technology

Listen with webReader

Open textbooks free 2

I’m about half-way through writing my open textbook, ‘Teaching in a Digital Age.’ I’ve done about five and a half chapters, and I would like to share my views on the underlying technology that I am using, because, while it does the job reasonably well, we are clearly in the Version 1.0 stage of software development, from an author’s perspective. I believe there is a major opportunity to develop a software authoring framework that fully exploits the open characteristics of a textbook, but we are not there yet.

Background

I’m writing this book more or less on my own, although I do have some support from an instructional designer and I’m anticipating getting some help with marketing once the book is complete. I’m also getting a lot of useful feedback, because I am publishing as the book is being developed (the first five chapters are already available here) and also publishing excerpts in this blog.

My main technical support is coming from BCcampus, which is managing a large open textbook project on behalf of the British Columbia provincial government. My book is not directly related to the provincial government-funded project, which at this stage is focused primarily on converting existing print textbooks to open, online versions. However, as the project advances, more open textbooks will need to be written from scratch. (For more on the BCcampus open textbook project project, see here.)

BCcampus has taken an ‘off-the-shelf’ open source authoring software ‘shell’ called Pressbooks, which in itself is based on WordPress. BCcampus has made some further adaptations to Pressbooks for the open textbooks that BCcampus is helping to develop. I have used the BCcampus version of Pressbooks to create my own textbook. However, anyone can use Pressbooks for free, if they wish to write an openly published book.

What I am trying to do

My goals are two-fold:

  • to openly publish a textbook on teaching in a digital age, aimed at teachers, instructors and faculty.
  • to explore ways to incorporate best teaching practice and an open education philosophy within the design of the book.

This is a report on where I’ve got to so far in authoring the book, using the Pressbooks/BCcampus template, and in particular on what I’m finding regarding the potential and limitations of the software for authoring an open textbook.

What works

It is extremely easy to start authoring with Pressbooks. After you log in to the Pressbooks main page, you can easily set up an account which is password protected. Once you have an account, you will be assigned a url which will take you to your admin page, from where you can author your book.

Anyone who has used WordPress for blogging will have no difficulty whatsoever in getting started in Pressbooks. If you already have a structure for the book in your mind, and know what you want to write, you can be writing within less than ten minutes of signing up with Pressbooks. You can also open accounts for others, such as co-authors, an editor, or an instructional designer, with password-protected access to the editing part.

Pressbooks allows you to work in private or to publish each chapter or section when ready. You can ‘export’ , in several versions, such as ePub, pdf or html, for free downloading. BCcampus is also making available, at cost, printed versions of their textbooks. The ‘exported’ version looks clean and replicates almost exactly the edited version, with embedded urls, diagrams, headings and indentation. The variety of exported formats enables use of the textbooks on various mobile devices and tablets. If the recommended technological structure is followed when writing and editing, the reader can easily navigate through the book in a variety of ways.

Thus, for basic book writing and publishing, Pressbooks is easy to use, comprehensive in the devices it can be used on, and pleasant to read.

Challenges

From the perspective however of an open textbook, I found the following challenges:

Lack of interactivity

Those of you used to using a learning management system are likely to be frustrated by the lack in Pressbooks of common features found within an LMS, such as ways to provide feedback on exercises, places where readers/students can add their own contributions, or places where monitored and edited discussions can take place. Thus some of the key opportunities to make a book more interactive and open are currently not available, without going outside the Pressbooks environment. There are two reasons for this.

1. Pressbooks was originally designed for supporting fiction writers, and as such works perfectly for them (providing they can manage to write easily in WordPress). If you want a straight read through a book, it is perfect, but this is not what you necessarily want with an educational textbook.

2. BCcampus has added some useful features, such as widgets that allow you to insert text boxes for learning objectives, student exercises, and key take-aways, but has had to disable the comment feature because the textbooks are likely to be used by many instructors with different classes. BCcampus is rightly worried that it would be confusing and overwhelming for multiple instructors if students across all the classes shared the same comment boxes. However, as an author, I want to integrate both the activities and the student responses to the activities, and above all I want comments and feedback on what I’ve written.

There are in fact really several distinct stages or uses of an open textbook:

  • book creation (which I am going through now), where feedback is needed by the author. At this stage, the comment feature is really essential. Ideally, it should be at the end of each chapter and part.
  • response from individual readers once the book is completed. I’m already getting these, as I’m publishing as I go. At least in the early days, feedback is again essential, and it would be quite manageable for the author to monitor the comments at this stage. However, over time, adoption by instructors, accumulated spam, and repetitious comments may lead the author to want to disable this feature.
  • adoption as part of a course. At this stage the comment feature needs to be disabled (or cleared), and replaced probably by a course web site, wiki or discussion forum linked specifically to a particular instructor and their course.

What I’d really like is a widget where I can just drop in a comment box in the right place, and the ability as an author to open, clear or disable it, as well as monitoring and where necessary editing it. It could be switched to open or private.

I have also explored some possible open source discussion forums or wikis, and computer-based test services, but these would have to sit outside the textbook, and I haven’t found a satisfactory service yet (although I haven’t looked very hard – suggestions welcome.)

The technological structure of the book

Unlike many online books that you will find on Kindle or iPads, Pressbooks does not output in discrete pages. The way it manages the structure of the book to enable fluent navigation by the reader is not immediately transparent to an author writing a book.

The two key features are Parts and Chapters. I assumed (incorrectly) that Parts were sub-units or sections of Chapters. This suited me, as I’m expecting a diverse audience with a wide range of prior knowledge. I assumed that many would not want to read a whole chapter on say design models, but may have a particular interest in some of the models and not in others. However, I made the basic mistake of not reading the BCcampus Authors’ Manual carefully before starting (and when I did read it, I did not understand it.) What I hadn’t realised was that Chapters link to Parts and the Parts are not intended to have much, if any, content.

Parts are really an introduction to the substance, a kind of organiser for the actual following content, which take place in the Chapters. Think of a novel: Part 1: 1969, Chapter 1: Boy meets girl. However, I rushed off and wrote Parts like sections of a chapter then cut and pasted each Part into a Chapter. I got half-way through writing the book before realising this was a mistake, thanks to a very helpful recent meeting with staff from BCcampus.

So I have ended up using a Part like an advance organiser for a chapter, and the Chapter feature for each section of a ‘Part’. This works well now, the navigation is much better, and it avoids the reader having to scroll down through an 8,000 word chapter. Some ‘Chapters’ in Pressbooks terminology are only a couple of paragraphs long and I have renamed them sections, with the Part containing the Chapter name. I also use the Part to state the purpose of the Chapter, what is covered in the chapter, and the key takeaways.

However, as you can see, the Pressbooks terminology of Parts and Chapters is really misleading. Worse, I spent two whole days cutting and repasting content I had already written in order to get the content into the right technological structure required by the software.

No mark-up facility

Unlike Word, an editor or a co-author cannot mark up drafts in Pressbooks (or WordPress for that matter – if there is a plug-in for this, please let me know.) This makes co-production of a book and getting feedback much more frustrating, especially as there is no comment feature.

If you are writing a co-edited or co-authored book, this is a major limitation, and a better strategy might be to initially edit in Google Docs or Word, then transfer everything when finished into Pressbooks or another publishing software shell. Even then, this is not a good solution because of the high risk of losing material during the transfer – and in any case, when is an open textbook ever finished? It should be a work in continuous updating.

Even for a single author, though, the inability to mark up drafts in Pressbooks is a considerable nuisance, especially if the comment feature is disabled. Not only my instructional designer, but also several readers who are following the development of the book, are copying sections from the Pressbook version into Word, marking up suggested corrections in Word, sending me the Word document, which I then go through then make any necessary changes in the Pressbooks version.

What is needed of course is a mark-up plug-in for WordPress, which would have much wider value than just open textbook authoring.

Limitations of WordPress

Some of these limitations are also limitations of writing and editing in WordPress. The feature for creating tables is so difficult to use that it is essentially useless. Some of the formatting doesn’t transfer when cutting and pasting to another screen page (which I have to do often), such as text alignment. I spend an enormous amount of time scrolling up to the top of the page, looking for the toolbox menu, to add urls, italics, lists, or indents, sometimes accidentally transferring out of the editing page and thus losing some of the more recent writing. (Apparently, in the new version of WordPress 4, the scrolling issue to get to the toolbar will be resolved – the toolbar will stay at the top of the screen, however far down you scroll).

However, I am spending far too much time on editing and not enough on creative writing. Editing is always a time-consuming but necessary activity when writing, but I really could do without technology frustrations when editing.

Conclusions

Pressbooks is a workable solution for writing an open textbook, but it works best if you want just a simple read through by the reader, in the manner of a traditional textbook. If though you want to make it more interactive, and open to comment, criticisms and substantive contributions from other people, then the current Pressbooks software is very limiting.

Pressbooks is a classic case of taking a new medium and merely transferring the format and structure of a previously existing medium. Although this is probably an essential and useful first step, what is really required is a complete re-design that fully exploits the characteristics or affordances of the new medium. For this to happen, though, a partnership between software engineers, potential authors and instructional designers is needed. However, there is a great opportunity here for creating truly innovative open source software for supporting open textbooks, if anyone has the time and resources to do this.

Authors such as myself also need to work out the difference (if any) between an open textbook and a learning management system. There are real difficulties in making everything in a course open, mainly because of hacking, spam and other external nuisances that can seriously disrupt a serious, engaged educational experience. The same applies to blogs and open textbooks. If the comment feature is too open it becomes overwhelmed with hacking and spam (I’m clearing about 50 bot-generated messages a day from my blog comment box – I don’t want to also have to spend this time keeping the comments on an open textbook under control.)

However, even accepting that an open textbook is not a substitute for an LMS, authors need to think carefully how the textbook can best be integrated or adopted within a course. Sample activities, suggestions for model answers, etc., can all be included. Above all, though, authors need to be clear when writing as to what will be done within the technological limitations of the textbook, what is best done outside the textbook, and how best to integrate these two elements.

I have to say I haven’t worked this out yet. It’s still a work in progress.

Over to you

As you can see, I am somewhat bumbling my way through the technology side of the writing, learning mainly through experience, although BCcampus has been more than helpful. I’d really like to hear though from other open textbook authors: is your experience similar or very different and if so why? Have you used different authoring software and how did that go?

Also, on the technology side, I’m still very open to other technology solutions, so long as they can be seamlessly integrated with Pressbooks. I have gone too far now to move to another software solution. But any suggestions welcome.

What UBC has learned about doing MOOCs

Listen with webReader

Coursera certificate 2

Engle, W. (2104) UBC MOOC Pilot: Design and Delivery Vancouver BC: University of British Columbia

The University of British Columbia, a premier public research university in Canada, successfully delivered five MOOCs in the spring and summer of 2013, using the Coursera platform. This report is an evaluation of the experience.

The report is particularly valuable because it provides details of course development and delivery, including media used and costs. Also UBC has been developing online courses for credit for almost 20 years, so it is interesting to see how this has impacted on the design of their MOOCs.

The MOOCs

1. Game Theory I: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

2. Game Theory II: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

3. Useful Genetics: R. Redfield, UBC

4. Climate Literacy: S. Harris and S. Burch, UBC

5. Introduction to Systematic Program Design: G. Kizcales, UBC

In terms of comparability I’m going to treat Game Theory I and II as one MOOC, as combined they were about the same length as the other MOOCs (between 8-12 weeks)

Basic statistics

330,150 signed up (82,500 on average per course)

164,935 logged in at least once (41,000 per course)

12,031 took final exam (3,000 per course)

8,174 earned course certificate (2,000 per course)

60-70% already had a post-secondary degree

30-40% were North American, with participants from nearly every country in the world.

Course development

None of the instructors had taught an online course before, but were supported by instructional designers, media development staff, and academic assistants (graduate and undergraduate students).

One major difference between UBC MOOCs and its online credit courses (which are primarily LMS-based) was the extensive use of video, the main component of the MOOC pilot courses.

Video production

305 videos constituting a total of 65 hours were produced. Each MOOC used a different method of production:

  • Intensive studio (Climate Literacy)
  • Hybrid studio plus instructor desktop (Systematic Program Design)
  • Light studio production (Game Theory I and II)
  • Instructor desktop (Useful Genetics)

Web pages

All the MOOCs except Games Theory also included weekly modules as HTML-based web pages, which is a variation of the Coursera design default model. Altogether 98 HTML module pages were developed. The weekly modules were used to provide guidance to students on learning goals, amount of work expected, an overview of activities, and additional quiz or assignment help. (All standard practice in UBC’s LMS-based credit courses.)

Assessment

1,049 quiz questions were developed, of which just over half were graded.

There were four peer assessments in total across all the MOOCs.

Course delivery

As well as the faculty member responsible for each MOOC, graduate and undergraduate academic assistants were a crucial component of all courses, with the following responsibilities:

  • directly assisting learners
  • troubleshooting technical problems
  • conducting quality assurance activities

There was very little one-on-one interaction between the main instructor and learners, but academic assistants monitored and moderated the online forum discussions.

Costs

As always, costing is a difficult exercise. Appendix B of the report gives a pilot total of $217,657, but this excludes academic assistance or, perhaps the most significant cost, instructor time.

Working from the video production costs ($95,350) and the proportion of costs (44%) devoted to video production in Figure 1 in the report, I estimate the direct cost at $216,700, or approximately $54,000 per MOOC, excluding faculty time and co-ordination support, but including academic assistance.

However, the range of cost is almost as important. The video production costs for Climate Literacy, which used intensive studio production, were more than six times the video production costs of Systematic Program Design (hybrid studio + desktop).

MOOCs as OERs

  • the UBC instructors are using their MOOC materials in their own on-campus, for-credit classes in a flipped classroom model
  • courses are left open and active on Coursera for self-paced learning
  • porting of video materials as open access YouTube videos
  • two courses (Climate Literacy and Useful Genetics) added Creative Commons licenses for re-use

Challenges

  • copyright clearance (Coursera owns the copyright so third party copyright needs to be cleared)
  • higher than expected time demands on all involved
  • iterative upgrades to the Coursera platform
  • partner relationship management (UBC + Coursera + Stanford University) was time-consuming.
  • training and managing academic assistants, especially time management
  • the Coursera platform limited instructors’ ability to develop desired course activities
  • Coursera’s peer assessment functionality in particular was limiting

Lessons

  • UBC’s prior experience in credit-based online learning led to better-designed, more interactive and more engaging MOOCs
  • learners always responded positively to instructor ‘presence’ in forums or course announcements
  • MOOC students were motivated by grades
  • MOOC students were willing to critically engage in critiquing instructors’ expertise and teaching
  • open publishing via MOOCs is a strong motivator for instructors
  • MOOCs require significant investment.

Conclusion

All the MOOCs received positive feedback and comments from students. UBC was able to gain direct experience in and knowledge of MOOCs and look at how this might inform both their for-credit on-campus and online teaching. UBC was also able to bring its experience in for-credit online learning to strengthening the design of MOOCs. Lastly it was able to make much more widely known the quality of UBC instructors and course materials.

Comment

First, congratulations to UBC for

  • experimenting with MOOCs
  • conducting the evaluation
  • making the report publicly available.

It is clear from the comments of participants in the appendices that at least some of the participants (we don’t know how many) were very pleased with the courses. As usual though with evaluation reports on MOOCs, there is no assessment of learning other than the end of course quiz-based tests. I don’t care too much about completion rates, but some measurement of student satisfaction would have been helpful.

It is also significant that UBC has now decided to move from Coursera to edX as its platform for MOOCs. edX, which is open source and allows partners to modify and adapt the platform, provides the flexibility that Coursera lacked, despite its many iterative ‘improvements’.

This also demonstrates the hubris of MOOC platform developers in ignoring best design principles in online learning when they designed their platforms. It is clear that UBC designers were able to improve the design of their MOOCs by drawing on prior for-credit online experience, but also that the MOOC platforms are still very limited in enabling the kind of learning activities that lead to student engagement and success.

The UBC report also highlighted the importance (and cost) of providing some form of learner support in course delivery. The use of academic assistants in particular clearly made the MOOCs more interactive and engaging, as well as limited but effective interventions from the instructors themselves, once again supported by (and confirming) prior research on the importance of instructor presence for successful for-credit online learning.

I very much appreciate the cost data provided by UBC, and the breakdown of production and delivery costs is extremely valuable, but I have to challenge the idea of providing any costs that exclude the time of the instructors. This is by far the largest and most important cost in MOOCs and the notion that MOOCs are free of instructor cost is to fly in the face of any respectable form of economics.

It is clear that MOOCs are more expensive to date per hour of study time than LMS-based for-credit online courses. We still do not have enough data to give a precise figure, and in any case, as the UBC study shows, cost is very much a factor of design. However, even without instructors costs, the UBC MOOCs at $54,000 each for between 8-12 weeks are already more than the average cost of a 13 week for-credit LMS-based online course, including instructor time.

This is partly due to the increased instructor time in preparation/production, but also to the higher cost of video production.  I am not against the use of video in principle, but it must add value. Using it for content transmission when this can be done so much more cheaply textually and/or by audio is a waste of the medium’s potential (although perhaps more motivating for the instructor).

More importantly, every institution contemplating MOOCs needs to do a cost-benefit exercise. Is it better to invest in MOOCs or credit-based online learning or both? If MOOCs are more expensive, what are the added benefits they provide and does this more than make up for not only the extra cost, but the lost opportunity of investing in (more) credit-based online learning or other forms of campus-based learning? I know what my answer would be.