October 2, 2014

The role of communities of practice in a digital age

Listen with webReader
Bank of America's Vital Voices progam links women executives of small and medium sized enterprises  Image: © Belfast Telegraph, 2014

Bank of America’s Vital Voices progam links women executives of small and medium sized enterprises from around the world
Image: © Belfast Telegraph, 2014

The story so far

I have published the first five chapters of my open textbook, ‘Teaching in a Digital Age‘.  I am now working on Chapter 6, ‘Models for Designing Teaching and Learning.’

In my last three posts I discussed respectively the appropriateness for a digital age of the classroom model, the ADDIE model, and the competency-based learning model. In this post, I explore the learning model based on communities of practice.

The theories behind communities of practice

The design of teaching often integrates different theories of learning. Communities of practice are one of the ways in which experiential learning, social constructivism, and connectivism can be combined, illustrating the limitations of trying to rigidly classify learning theories. Practice tends to be more complex.

What are communities of practice?

Definition: 

Communities of practice are groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly.

Wenger, 2014

The basic premise behind communities of practice is simple: we all learn in everyday life from the communities in which we find ourselves. Communities of practice are everywhere. Nearly everyone belongs to some community of practice, whether it is through our working colleagues or associates, our profession or trade, or our leisure interests, such as a book club. Wenger (2000) argues that a community of practice is different from a community of interest or a geographical community in that it involves a shared practice: ways of doing things that are shared to some significant extent among members.

Wenger argues that there are three crucial characteristics of a community of practice:

  • domain: a common interest that connects and holds together the community
  • community: a community is bound by the shared activities they pursue (for example, meetings, discussions) around their common domain
  • practice: members of a community of practice are practitioners; what they do informs their participation in the community; and what they learn from the community affects what they do.

Wenger (2000) has argued that although individuals learn through participation in a community of practice, more important is the generation of newer or deeper levels of knowledge through the sum of the group activity. If the community of practice is centered around business processes, for instance, this can be of considerable benefit to an organization. Smith (2003) notes that:

…communities of practice affect performance..[This] is important in part because of their potential to overcome the inherent problems of a slow-moving traditional hierarchy in a fast-moving virtual economy. Communities also appear to be an effective way for organizations to handle unstructured problems and to share knowledge outside of the traditional structural boundaries. In addition, the community concept is acknowledged to be a means of developing and maintaining long-term organizational memory.

Brown and Duguid (2000) describe a community of practice developed around the Xerox customer service representatives who repaired the machines in the field. The Xerox reps began exchanging tips and tricks over informal meetings at breakfast or lunch and eventually Xerox saw the value of these interactions and created the Eureka project to allow these interactions to be shared across the global network of representatives. The Eureka database has been estimated to have saved the corporation $100 million. Companies such as Google and Apple are encouraging communities of practice through the sharing of knowledge across their many specialist staff.

Technology provides a wide range of tools that can support communities of practice, as indicated by Wenger (2010) in the diagram below:

Image: © Etienne Wenger, 2010

Designing effective communities of practice

Most communities of practice have no formal design and tend to be self-organising systems. They have a natural life cycle, and come to an end when they no longer serve the needs of the community. However, there is now a body of theory and research that has identified actions that can help sustain and improve the effectiveness of communities of practice.

 Wenger, McDermott and Snyder (2002) have identified seven key design principles for creating effective and self-sustaining communities of practice, related specifically to the management of the community, although the ultimate success of a community of practice will be determined by the activities of the members of the community themselves. Designers of a community of practice need to:

  1. Design for evolution: ensuring that the community can evolve and shift in focus to meet the interests of the participants without moving too far from the common domain of interest
  2. Open a dialogue between inside and outside perspectives: encourage the introduction and discussion of new perspectives that come or are brought in from outside the community of practice
  3. Encourage and accept different levels of participation, The strength of participation varies from participant to participant. The ‘core’ (most active members) are those who participate regularly. There are others who follow the discussions or activities but do not take a leading role in making active contributions. Then there are those (likely the majority) who are on the periphery of the community but may become more active participants if the activities or discussions start to engage them more fully. All these levels of participation need to be accepted and encouraged within the community.
  4. Develop both public and private community spaces: communities of practice are strengthened if they encourage individual or group activities that are more personal or private as well as the more public general discussions; for instance, individuals may decide to blog about their activities, or in a larger online community of practice a small group that live or work close together may also decide to meet informally on a face-to-face basis
  5. Focus on value. Attempts should be made explicitly to identify, through feedback and discussion, the contributions that the community most values, then focus the discussion and activities around these issues.
  6. Combine familiarity and excitement, by focusing both on shared, common concerns and perspectives, but also by introducing radical or challenging perspectives for discussion or action
  7. Create a rhythm for the community: there needs to be a regular schedule of activities or focal points that bring participants together on a regular basis, within the constraints of participants’ time and interests.

Subsequent research has identified a number of critical factors that influence the effectiveness of participants in communities of practice, These include being:

  • aware of social presence: individuals need to feel comfortable in engaging socially with other professionals or ‘experts’ in the domain, and those with greater knowledge must be willing to share in a collegial manner that respects the views and knowledge of other participants (social presence is defined as the awareness of others in an interaction combined with an appreciation of the interpersonal aspects of that interaction.)
  • motivated to share information for the common good of the community
  • able and willing to collaborate.

EDUCAUSE has developed a step-by-step guide for designing and cultivating communities of practice in higher education (Cambridge, Kaplan and Suter, 2005).

Lastly, research on other related sectors, such as collaborative learning or MOOCs, can inform the design and development of communities of practice. For instance, communities of practice need to balance between structure and chaos: too much structure and many participants are likely to feel constrained in what they need to discuss; too little structure and participants can quickly lose interest or become overwhelmed.

Many of the other findings about group and online behaviour, such as the need to respect others, observing online etiquette, and preventing certain individuals from dominating the discussion, are all likely to apply. However, because many communities of practice are by definition self-regulating, establishing rules of conduct and even more so enforcing them is really a responsibility of the participants themselves.

Learning through communities of practice in a digital age

Communities of practice are a powerful manifestation of informal learning. They generally evolve naturally to address commonly shared interests and problems. By their nature, they tend to exist outside formal educational organisations. Participants are not usually looking for formal qualifications, but to address issues in their life and to be better at what they do. Furthermore, communities of practice are not dependent on any particular medium; participants may meet face-to-face socially or at work, or they can participate in online or virtual communities of practice.

It should be noted that communities of practice can be very effective in a digital world, where the working context is volatile, complex, uncertain and ambiguous.  A large part of the lifelong learning market will become occupied by communities of practice and self-learning, through collaborative learning, sharing of knowledge and experience, and crowd-sourcing new ideas and development. Such informal learning provision will be particularly valuable for non-governmental or charitable organizations, such as the Red Cross, Greenpeace or UNICEF, or local government, looking for ways to engage communities in their areas of operation.

These communities of learners will be open and free, and hence will provide a competitive alternative to the high priced lifelong learning programs being offered by research universities. This will put pressure on universities and colleges to provide more flexible arrangements for recognition of informal learning, in order to hold on to their current monopoly of post-secondary accreditation.

One of the significant developments in recent years has been the use of massive open online courses (MOOCs) for developing online communities of practice. To date the focus of the majority of MOOCs from providers such as Coursera, Udacity and edX, has been on academic ‘courses’, on topics such as artificial intelligence or dinosaurs, which do have a widespread interest. However, these more instructionist MOOCs are not really developed as communities of practice, because  they use mainly a transmissive pedagogy, from experts to those considered less expert. Even though there may be massive numbers participating in online forums, they are not constructed to maximise the contributions from the participants (despite the fact that most MOOC participants already have high levels of education.). Indeed there is evidence that in really large instructionist MOOCs, participants feel overwhelmed by the magnitude and lack of structuring of the participant contributions (see for instance Knox, 2014).

In comparison, connectivist MOOCs are an ideal way to bring together specialists scattered around the world to focus on a common interest or domain. Connectivist MOOCs are much closer to being virtual communities of practice, in that they put much more emphasis on sharing knowledge between more or less equal participants. However, current connectivist MOOCs do not always incorporate what research indicates are best practices for developing communities of practice, and those wanting to establish a virtual community of practice at the moment need some kind of MOOC provider to get them started and give them access to the necessary MOOC software.

In the long run, MOOCs need to evolve to the point where it is possible for those with a common interest to easily create their own open, online communities of practice. As open source MOOC platforms evolve, it should become easier for people without computer science degrees to create and more importantly manage their own MOOCs, without having to go through a MOOC provider such as Coursera or edX. Also, there are other simpler tools, such as wikis, or more complex ones, such as virtual worlds, that may in the long run have more potential for virtual communities of practice created and organised by the participants themselves.

Although communities of practice are likely to become more rather than less important in a digital age, it is probably a mistake to think of them as a replacement for traditional forms of education. There is no single, ‘right’ approach to the design of teaching. Different groups have different needs. Communities of practice are more of an alternative for certain kinds of learners, such as lifelong learners, and are likely to work best when participants already have some domain knowledge and can contribute personally and in a constructive manner – which suggests the need for at least some form of prior general education or training for those participating in effective communities of practice.

In conclusion, it is clear is that in an increasingly volatile, uncertain, complex, and ambiguous world, and given the openness of the Internet, the social media tools now available, and the need for sharing of knowledge on a global scale, virtual communities of practice will become even more common and important. Smart educators and trainers will look to see how they can harness the strength of this design model, particularly for lifelong learning. However, merely lumping together large numbers of people with a common interest is unlikely to lead to effective learning.  Attention needs to be paid to those design principles that lead to effective communities of practice.

Over to you

Once again, I’m not an expert on communities of practice, so feedback on what I have written on this model of learning will be much appreciated. In particular:

  • Have a got it wrong? Are their important elements missing?
  • Is there good research on the design of communities of practice that I have missed?
  • Do you agree that they are NOT a replacement for other forms of education?
  • Can you really design a community of practice or do they just evolve naturally? If so, what conditions are needed for success other than those already discussed in this post?
  • How would you evaluate the success of a community of practice? What would you look for? How could this be identified or described?

I would love to hear from anyone who has attempted to create communities of practice and what design elements they would recommend.

References

Brown, J. and Duguid, Paul (2000). “Balancing act: How to capture knowledge without killing it”Harvard Business Review.

Cambridge, D., Kaplan, S. and Suter, V. (2005) Community of Practice Design Guide Louisville  CO: EDUCAUSE

Knox, J. (2004) Digital culture clash: “massive” education in the E-learning and Digital Cultures MOOC Distance Education, Vol. 35, No. 2

Wenger, E. (2000) Communities of Practice: Learning, Meaning and Identity Cambridge UK: Cambridge University Press

Wenger, E. (2014) Communities of practice: a brief introduction, accessed 26 September, 2014

Wenger, E, McDermott, R., and Snyder, W. (2002). Cultivating Communities of Practice (Hardcover). Harvard Business Press; 1 edition.

 

‘The Economist’ on online learning

Listen with webReader
© The Economist, 2014

© The Economist, 2014

Insight (2014) The democratisation of learning The Economist, September 26

This is a well-balanced and well-researched article on online learning in post-secondary education, even if it is very U.S.-focused. Doesn’t say anything new, but it covers the right ground.

What I liked is that it places MOOCs within the broader context of what is happening in online learning. A nice article to give to faculty skeptical about online learning (especially if they are economists).

Writing an open textbook: a mid-term report on the technology

Listen with webReader

Open textbooks free 2

I’m about half-way through writing my open textbook, ‘Teaching in a Digital Age.’ I’ve done about five and a half chapters, and I would like to share my views on the underlying technology that I am using, because, while it does the job reasonably well, we are clearly in the Version 1.0 stage of software development, from an author’s perspective. I believe there is a major opportunity to develop a software authoring framework that fully exploits the open characteristics of a textbook, but we are not there yet.

Background

I’m writing this book more or less on my own, although I do have some support from an instructional designer and I’m anticipating getting some help with marketing once the book is complete. I’m also getting a lot of useful feedback, because I am publishing as the book is being developed (the first five chapters are already available here) and also publishing excerpts in this blog.

My main technical support is coming from BCcampus, which is managing a large open textbook project on behalf of the British Columbia provincial government. My book is not directly related to the provincial government-funded project, which at this stage is focused primarily on converting existing print textbooks to open, online versions. However, as the project advances, more open textbooks will need to be written from scratch. (For more on the BCcampus open textbook project project, see here.)

BCcampus has taken an ‘off-the-shelf’ open source authoring software ‘shell’ called Pressbooks, which in itself is based on WordPress. BCcampus has made some further adaptations to Pressbooks for the open textbooks that BCcampus is helping to develop. I have used the BCcampus version of Pressbooks to create my own textbook. However, anyone can use Pressbooks for free, if they wish to write an openly published book.

What I am trying to do

My goals are two-fold:

  • to openly publish a textbook on teaching in a digital age, aimed at teachers, instructors and faculty.
  • to explore ways to incorporate best teaching practice and an open education philosophy within the design of the book.

This is a report on where I’ve got to so far in authoring the book, using the Pressbooks/BCcampus template, and in particular on what I’m finding regarding the potential and limitations of the software for authoring an open textbook.

What works

It is extremely easy to start authoring with Pressbooks. After you log in to the Pressbooks main page, you can easily set up an account which is password protected. Once you have an account, you will be assigned a url which will take you to your admin page, from where you can author your book.

Anyone who has used WordPress for blogging will have no difficulty whatsoever in getting started in Pressbooks. If you already have a structure for the book in your mind, and know what you want to write, you can be writing within less than ten minutes of signing up with Pressbooks. You can also open accounts for others, such as co-authors, an editor, or an instructional designer, with password-protected access to the editing part.

Pressbooks allows you to work in private or to publish each chapter or section when ready. You can ‘export’ , in several versions, such as ePub, pdf or html, for free downloading. BCcampus is also making available, at cost, printed versions of their textbooks. The ‘exported’ version looks clean and replicates almost exactly the edited version, with embedded urls, diagrams, headings and indentation. The variety of exported formats enables use of the textbooks on various mobile devices and tablets. If the recommended technological structure is followed when writing and editing, the reader can easily navigate through the book in a variety of ways.

Thus, for basic book writing and publishing, Pressbooks is easy to use, comprehensive in the devices it can be used on, and pleasant to read.

Challenges

From the perspective however of an open textbook, I found the following challenges:

Lack of interactivity

Those of you used to using a learning management system are likely to be frustrated by the lack in Pressbooks of common features found within an LMS, such as ways to provide feedback on exercises, places where readers/students can add their own contributions, or places where monitored and edited discussions can take place. Thus some of the key opportunities to make a book more interactive and open are currently not available, without going outside the Pressbooks environment. There are two reasons for this.

1. Pressbooks was originally designed for supporting fiction writers, and as such works perfectly for them (providing they can manage to write easily in WordPress). If you want a straight read through a book, it is perfect, but this is not what you necessarily want with an educational textbook.

2. BCcampus has added some useful features, such as widgets that allow you to insert text boxes for learning objectives, student exercises, and key take-aways, but has had to disable the comment feature because the textbooks are likely to be used by many instructors with different classes. BCcampus is rightly worried that it would be confusing and overwhelming for multiple instructors if students across all the classes shared the same comment boxes. However, as an author, I want to integrate both the activities and the student responses to the activities, and above all I want comments and feedback on what I’ve written.

There are in fact really several distinct stages or uses of an open textbook:

  • book creation (which I am going through now), where feedback is needed by the author. At this stage, the comment feature is really essential. Ideally, it should be at the end of each chapter and part.
  • response from individual readers once the book is completed. I’m already getting these, as I’m publishing as I go. At least in the early days, feedback is again essential, and it would be quite manageable for the author to monitor the comments at this stage. However, over time, adoption by instructors, accumulated spam, and repetitious comments may lead the author to want to disable this feature.
  • adoption as part of a course. At this stage the comment feature needs to be disabled (or cleared), and replaced probably by a course web site, wiki or discussion forum linked specifically to a particular instructor and their course.

What I’d really like is a widget where I can just drop in a comment box in the right place, and the ability as an author to open, clear or disable it, as well as monitoring and where necessary editing it. It could be switched to open or private.

I have also explored some possible open source discussion forums or wikis, and computer-based test services, but these would have to sit outside the textbook, and I haven’t found a satisfactory service yet (although I haven’t looked very hard – suggestions welcome.)

The technological structure of the book

Unlike many online books that you will find on Kindle or iPads, Pressbooks does not output in discrete pages. The way it manages the structure of the book to enable fluent navigation by the reader is not immediately transparent to an author writing a book.

The two key features are Parts and Chapters. I assumed (incorrectly) that Parts were sub-units or sections of Chapters. This suited me, as I’m expecting a diverse audience with a wide range of prior knowledge. I assumed that many would not want to read a whole chapter on say design models, but may have a particular interest in some of the models and not in others. However, I made the basic mistake of not reading the BCcampus Authors’ Manual carefully before starting (and when I did read it, I did not understand it.) What I hadn’t realised was that Chapters link to Parts and the Parts are not intended to have much, if any, content.

Parts are really an introduction to the substance, a kind of organiser for the actual following content, which take place in the Chapters. Think of a novel: Part 1: 1969, Chapter 1: Boy meets girl. However, I rushed off and wrote Parts like sections of a chapter then cut and pasted each Part into a Chapter. I got half-way through writing the book before realising this was a mistake, thanks to a very helpful recent meeting with staff from BCcampus.

So I have ended up using a Part like an advance organiser for a chapter, and the Chapter feature for each section of a ‘Part’. This works well now, the navigation is much better, and it avoids the reader having to scroll down through an 8,000 word chapter. Some ‘Chapters’ in Pressbooks terminology are only a couple of paragraphs long and I have renamed them sections, with the Part containing the Chapter name. I also use the Part to state the purpose of the Chapter, what is covered in the chapter, and the key takeaways.

However, as you can see, the Pressbooks terminology of Parts and Chapters is really misleading. Worse, I spent two whole days cutting and repasting content I had already written in order to get the content into the right technological structure required by the software.

No mark-up facility

Unlike Word, an editor or a co-author cannot mark up drafts in Pressbooks (or WordPress for that matter – if there is a plug-in for this, please let me know.) This makes co-production of a book and getting feedback much more frustrating, especially as there is no comment feature.

If you are writing a co-edited or co-authored book, this is a major limitation, and a better strategy might be to initially edit in Google Docs or Word, then transfer everything when finished into Pressbooks or another publishing software shell. Even then, this is not a good solution because of the high risk of losing material during the transfer – and in any case, when is an open textbook ever finished? It should be a work in continuous updating.

Even for a single author, though, the inability to mark up drafts in Pressbooks is a considerable nuisance, especially if the comment feature is disabled. Not only my instructional designer, but also several readers who are following the development of the book, are copying sections from the Pressbook version into Word, marking up suggested corrections in Word, sending me the Word document, which I then go through then make any necessary changes in the Pressbooks version.

What is needed of course is a mark-up plug-in for WordPress, which would have much wider value than just open textbook authoring.

Limitations of WordPress

Some of these limitations are also limitations of writing and editing in WordPress. The feature for creating tables is so difficult to use that it is essentially useless. Some of the formatting doesn’t transfer when cutting and pasting to another screen page (which I have to do often), such as text alignment. I spend an enormous amount of time scrolling up to the top of the page, looking for the toolbox menu, to add urls, italics, lists, or indents, sometimes accidentally transferring out of the editing page and thus losing some of the more recent writing. (Apparently, in the new version of WordPress 4, the scrolling issue to get to the toolbar will be resolved – the toolbar will stay at the top of the screen, however far down you scroll).

However, I am spending far too much time on editing and not enough on creative writing. Editing is always a time-consuming but necessary activity when writing, but I really could do without technology frustrations when editing.

Conclusions

Pressbooks is a workable solution for writing an open textbook, but it works best if you want just a simple read through by the reader, in the manner of a traditional textbook. If though you want to make it more interactive, and open to comment, criticisms and substantive contributions from other people, then the current Pressbooks software is very limiting.

Pressbooks is a classic case of taking a new medium and merely transferring the format and structure of a previously existing medium. Although this is probably an essential and useful first step, what is really required is a complete re-design that fully exploits the characteristics or affordances of the new medium. For this to happen, though, a partnership between software engineers, potential authors and instructional designers is needed. However, there is a great opportunity here for creating truly innovative open source software for supporting open textbooks, if anyone has the time and resources to do this.

Authors such as myself also need to work out the difference (if any) between an open textbook and a learning management system. There are real difficulties in making everything in a course open, mainly because of hacking, spam and other external nuisances that can seriously disrupt a serious, engaged educational experience. The same applies to blogs and open textbooks. If the comment feature is too open it becomes overwhelmed with hacking and spam (I’m clearing about 50 bot-generated messages a day from my blog comment box – I don’t want to also have to spend this time keeping the comments on an open textbook under control.)

However, even accepting that an open textbook is not a substitute for an LMS, authors need to think carefully how the textbook can best be integrated or adopted within a course. Sample activities, suggestions for model answers, etc., can all be included. Above all, though, authors need to be clear when writing as to what will be done within the technological limitations of the textbook, what is best done outside the textbook, and how best to integrate these two elements.

I have to say I haven’t worked this out yet. It’s still a work in progress.

Over to you

As you can see, I am somewhat bumbling my way through the technology side of the writing, learning mainly through experience, although BCcampus has been more than helpful. I’d really like to hear though from other open textbook authors: is your experience similar or very different and if so why? Have you used different authoring software and how did that go?

Also, on the technology side, I’m still very open to other technology solutions, so long as they can be seamlessly integrated with Pressbooks. I have gone too far now to move to another software solution. But any suggestions welcome.

The strengths and weaknesses of competency-based learning in a digital age

Listen with webReader

Online student 2

Chapter 5 of my open textbook, ‘Teaching in a Digital Age‘, is now published. In Chapter 5, I developed the concept of a learning environment.

I am now working on Chapter 6, ‘Models for Designing Teaching and Learning.’ In my last two posts I discussed respectively the appropriateness of the classroom model and the ADDIE model for a digital age. In this post, I explore the same issue for competency-based learning model. (Some of this material has been published earlier by Contact North in its Gamechangers series.)

Competency-based learning

Competency-based learning attempts to break away from the regularly scheduled classroom model, where students study the same subject matter at the same speed in a cohort of fellow students.

What is competency-based learning?

Competency-based learning begins by identifying specific competencies or skills, and enables learners to develop mastery of each competency or skill at their own pace, usually working with a mentor. Learners can develop just the competencies or skills they feel they need (for which increasingly they may receive a ‘badge’ or some form of validated recognition), or can combine a whole set of competencies into a full qualification, such as a certificate, diploma or increasingly a full degree. Learners work individually, rather than in cohorts. If learners can demonstrate that they are already have mastery of a particular competency or skill, through a test or some form of prior learning assessment, they may be allowed to move to the next level of competency without having to repeat a prescribed course of study for the prior competency.

Its value for developing practical or vocational skills or competencies is more obvious, but increasingly competency-based learning is being used for education requiring more abstract or academic skills development, sometimes combined with other cohort-based courses or programs. The Western Governors University, with nearly 40,000 students, has pioneered competency-based learning, but with the more recent support of the Federal Department of Education it is expanding rapidly in the USA.

Competency-based learning is particularly appropriate for adult learners with life experience who may have developed competencies or skills without formal education or training, for those who started school or college and dropped out and wish to return to formal study, but want their earlier learning to be recognized, or for those learners wanting to develop specific skills but not wanting a full program of studies. Competency-based learning can be delivered through a campus program, but it is increasingly delivered fully online, because many students taking such programs are already working or seeking work.

Designing competency-based learning

There are various approaches, but the Western Governors model illustrates many of the key steps.

Defining competencies

A feature of most competency-based programs is a partnership between employers and educators in identifying the competencies required, at least at a high level. Some of the skills outlined in Chapter 1, such as problem-solving or critical thinking, may be considered high-level, but competency-based learning tries to break down abstract or vague goals into specific, measurable competencies.

For instance, at Western Governors University (WGU), for each degree, a high-level set of competencies is defined by the University Council, and then a working team of contracted subject matter experts takes the ten or so high level competencies for a particular qualification and breaks them down into about 30 more specific competencies, around which are built online courses to develop mastery of each competency. Competencies are based upon what graduates are supposed to know in the workplace and as professionals in a chosen career. Assessments are designed specifically to assess the mastery of each competency; thus students receive either a pass/no pass following assessment. A degree is awarded when all 30 specified competencies are successfully achieved.

Defining competencies that meet the needs of students and employers in ways that are progressive (i.e. one competency builds on earlier competencies and leads to more advanced competencies) and coherent (in that the sum of all the competencies produces a graduate with all the knowledge and skills required within a business or profession) is perhaps the most important and most difficult part of competency-based learning.

Course and program design

At WGU, courses are created by in-house subject matter experts selecting existing online curriculum from third parties and/or resources such as e-textbooks through contracts with publishers. Increasingly open educational resources are used. WGU does not use an LMS but a specially designed portal for each course. E-textbooks are offered to students without extra cost to the student, through contracts between WGU and the publishers. Courses are pre-determined for the student with no electives. Students are admitted on a monthly basis and work their way through each competency at their own pace.

Students who already possess competencies may accelerate through their program in two ways: transferring in credits from a previous associate degree in appropriate areas (e.g. general education, writing); or by taking exams when they feel they are ready.

Learner support

Again this varies from institution to institution. WGU currently employs approximately 750 faculty who act as mentors. There are two kinds of mentors: ‘student’ mentors and ‘course’ mentors. Student mentors, who have qualifications within the subject domain, usually at a masters level, are in at least bi-weekly telephone contact with their students, depending on the needs of the student in working through their courses, and are the main contact for students. A student mentor is responsible for roughly 85 students. Students start with a mentor from their first day and stay with their mentor until graduation. Student mentors assist students in determining and maintaining an appropriate pace of study and step in with help when students are struggling.

Course mentors are more highly qualified, usually with a doctorate, and provide extra support for students when needed. Course mentors will be available to between 200-400 students at a time, depending on the subject requirement.

Students may contact either student or course mentors at any time (unlimited access) and mentors are expected to deal with student calls within one business day. Student mentors are pro-active, calling students regularly (at least once every two weeks, more if necessary) to maintain contact. Mentors are full-time but work flexible hours, usually from home. Mentors are reasonably well paid, and receive extensive training in mentoring.

Remote proctoring of exams

Remote proctoring of exams

Assessment

WGU uses written papers, portfolios, projects, observed student performance and computer-marked assignments as appropriate, with detailed rubrics. Assessments are submitted online and if they require human evaluation, qualified graders (subject matter experts trained by WGU in assessment) are randomly assigned to mark work on a pass/fail basis. If students fail, the graders provide feedback on the areas where competency was not demonstrated. Students may resubmit if necessary.

Students will take both formative (pre-assessment) and summative (proctored) exams. WGU is increasingly using online proctoring, enabling students to take an exam at home under video supervision, using facial recognition technology to ensure that the registered student is taking the exam. In areas such as teaching and health, student performance or practice is  assessed in situ by professionals (teachers, nurses).

Strengths of a competency-based approach to design

Proponents have identified a number of strengths in the competency-based learning approach:

  • it meets the immediate needs of businesses and professions; students are either already working, and receive advancement within the company, or if unemployed, are more likely to be employed once qualified
  • it enables learners with work or family commitments to study at their own pace
  • for some students, it speeds up time to completion of a qualification by enabling prior learning to be recognized
  • students get individual support and help from their mentors
  • tuition fees are affordable ($6,000 per annum at WGU) and programs can be self-funding from tuition fees alone, since WGU uses already existing study materials and increasingly open educational resources
  • increasingly, competency-based education is being recognized as eligible for Federal loans and student aid in the USA.

Consequently, institutions such as WGU, the University of Southern New Hampshire, and Northern Arizona University, using a competency-based approach, at least as part of their operations, have seen annual enrolment growth in the range of 30-40 per cent per annum.

Weaknesses of a competency-based approach to design

Its main weakness is that it works well with some learning environments and less well with others. In particular:

  • it focuses on immediate employer needs and is less focused on preparing learners with the flexibility needed for a more uncertain future
  • it does not suit subject areas where it is difficult to prescribe specific competencies or where new skills and new knowledge need to be rapidly accommodated
  • it takes an objectivist approach to learning
  • it ignores the importance of social learning
  • it will not fit the preferred learning styles of many students.

In conclusion

Competency-based learning is a relatively new approach to learning design which is proving increasingly popular with employers and suits certain kinds of learners such as adult learners seeking to re-skill or searching for mid-level jobs requiring relatively easily identifiable skills. It does not suit though all kinds of learners and may be limited in developing the higher level, more abstract knowledge and skills requiring creativity, high-level problem-solving and decision-making and critical thinking.

Over to you

I have been very gratified by the feedback and open-ness of many of the readers of this blog on my earlier drafts for this book. I do not feel I am an expert on competency-based learning, never having designed a course this way, so feedback and advice from more experienced practitioners will be particularly welcome. In particular:

1. I have focused mainly on the Western Governors design model for competency-based learning, which is entirely online. Do you know other models of designing competency-based learning that I should have included?

2. What are your views on the competency-based model? Is it a useful model for designing teaching in a digital age? Do you agree with my criticisms of the model?

3. What is the difference between a competency and a skill? Or is there no difference?

4. What key articles, videos or books on competency-based learning would you recommend?

What’s next

So far I have done drafts of the following (as blogs)

  • What is a design model?
  • The classroom model
  • Classroom models in online learning
    • LMSs
    • lecture capture
  • ADDIE
  • competency-based learning

Still to come:

  • Connectivist models, including communities of practice and cMOOCs
  • PLEs
  • AI approaches
  • Flexible design models
  • Conclusion

My next post in this series then will be on the appropriateness of connectivist design models for teaching in a digital age.

What UBC has learned about doing MOOCs

Listen with webReader

Coursera certificate 2

Engle, W. (2104) UBC MOOC Pilot: Design and Delivery Vancouver BC: University of British Columbia

The University of British Columbia, a premier public research university in Canada, successfully delivered five MOOCs in the spring and summer of 2013, using the Coursera platform. This report is an evaluation of the experience.

The report is particularly valuable because it provides details of course development and delivery, including media used and costs. Also UBC has been developing online courses for credit for almost 20 years, so it is interesting to see how this has impacted on the design of their MOOCs.

The MOOCs

1. Game Theory I: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

2. Game Theory II: K. Leyton Brown (UBC); M. Jackson and Y.Shoham (Stanford University)

3. Useful Genetics: R. Redfield, UBC

4. Climate Literacy: S. Harris and S. Burch, UBC

5. Introduction to Systematic Program Design: G. Kizcales, UBC

In terms of comparability I’m going to treat Game Theory I and II as one MOOC, as combined they were about the same length as the other MOOCs (between 8-12 weeks)

Basic statistics

330,150 signed up (82,500 on average per course)

164,935 logged in at least once (41,000 per course)

12,031 took final exam (3,000 per course)

8,174 earned course certificate (2,000 per course)

60-70% already had a post-secondary degree

30-40% were North American, with participants from nearly every country in the world.

Course development

None of the instructors had taught an online course before, but were supported by instructional designers, media development staff, and academic assistants (graduate and undergraduate students).

One major difference between UBC MOOCs and its online credit courses (which are primarily LMS-based) was the extensive use of video, the main component of the MOOC pilot courses.

Video production

305 videos constituting a total of 65 hours were produced. Each MOOC used a different method of production:

  • Intensive studio (Climate Literacy)
  • Hybrid studio plus instructor desktop (Systematic Program Design)
  • Light studio production (Game Theory I and II)
  • Instructor desktop (Useful Genetics)

Web pages

All the MOOCs except Games Theory also included weekly modules as HTML-based web pages, which is a variation of the Coursera design default model. Altogether 98 HTML module pages were developed. The weekly modules were used to provide guidance to students on learning goals, amount of work expected, an overview of activities, and additional quiz or assignment help. (All standard practice in UBC’s LMS-based credit courses.)

Assessment

1,049 quiz questions were developed, of which just over half were graded.

There were four peer assessments in total across all the MOOCs.

Course delivery

As well as the faculty member responsible for each MOOC, graduate and undergraduate academic assistants were a crucial component of all courses, with the following responsibilities:

  • directly assisting learners
  • troubleshooting technical problems
  • conducting quality assurance activities

There was very little one-on-one interaction between the main instructor and learners, but academic assistants monitored and moderated the online forum discussions.

Costs

As always, costing is a difficult exercise. Appendix B of the report gives a pilot total of $217,657, but this excludes academic assistance or, perhaps the most significant cost, instructor time.

Working from the video production costs ($95,350) and the proportion of costs (44%) devoted to video production in Figure 1 in the report, I estimate the direct cost at $216,700, or approximately $54,000 per MOOC, excluding faculty time and co-ordination support, but including academic assistance.

However, the range of cost is almost as important. The video production costs for Climate Literacy, which used intensive studio production, were more than six times the video production costs of Systematic Program Design (hybrid studio + desktop).

MOOCs as OERs

  • the UBC instructors are using their MOOC materials in their own on-campus, for-credit classes in a flipped classroom model
  • courses are left open and active on Coursera for self-paced learning
  • porting of video materials as open access YouTube videos
  • two courses (Climate Literacy and Useful Genetics) added Creative Commons licenses for re-use

Challenges

  • copyright clearance (Coursera owns the copyright so third party copyright needs to be cleared)
  • higher than expected time demands on all involved
  • iterative upgrades to the Coursera platform
  • partner relationship management (UBC + Coursera + Stanford University) was time-consuming.
  • training and managing academic assistants, especially time management
  • the Coursera platform limited instructors’ ability to develop desired course activities
  • Coursera’s peer assessment functionality in particular was limiting

Lessons

  • UBC’s prior experience in credit-based online learning led to better-designed, more interactive and more engaging MOOCs
  • learners always responded positively to instructor ‘presence’ in forums or course announcements
  • MOOC students were motivated by grades
  • MOOC students were willing to critically engage in critiquing instructors’ expertise and teaching
  • open publishing via MOOCs is a strong motivator for instructors
  • MOOCs require significant investment.

Conclusion

All the MOOCs received positive feedback and comments from students. UBC was able to gain direct experience in and knowledge of MOOCs and look at how this might inform both their for-credit on-campus and online teaching. UBC was also able to bring its experience in for-credit online learning to strengthening the design of MOOCs. Lastly it was able to make much more widely known the quality of UBC instructors and course materials.

Comment

First, congratulations to UBC for

  • experimenting with MOOCs
  • conducting the evaluation
  • making the report publicly available.

It is clear from the comments of participants in the appendices that at least some of the participants (we don’t know how many) were very pleased with the courses. As usual though with evaluation reports on MOOCs, there is no assessment of learning other than the end of course quiz-based tests. I don’t care too much about completion rates, but some measurement of student satisfaction would have been helpful.

It is also significant that UBC has now decided to move from Coursera to edX as its platform for MOOCs. edX, which is open source and allows partners to modify and adapt the platform, provides the flexibility that Coursera lacked, despite its many iterative ‘improvements’.

This also demonstrates the hubris of MOOC platform developers in ignoring best design principles in online learning when they designed their platforms. It is clear that UBC designers were able to improve the design of their MOOCs by drawing on prior for-credit online experience, but also that the MOOC platforms are still very limited in enabling the kind of learning activities that lead to student engagement and success.

The UBC report also highlighted the importance (and cost) of providing some form of learner support in course delivery. The use of academic assistants in particular clearly made the MOOCs more interactive and engaging, as well as limited but effective interventions from the instructors themselves, once again supported by (and confirming) prior research on the importance of instructor presence for successful for-credit online learning.

I very much appreciate the cost data provided by UBC, and the breakdown of production and delivery costs is extremely valuable, but I have to challenge the idea of providing any costs that exclude the time of the instructors. This is by far the largest and most important cost in MOOCs and the notion that MOOCs are free of instructor cost is to fly in the face of any respectable form of economics.

It is clear that MOOCs are more expensive to date per hour of study time than LMS-based for-credit online courses. We still do not have enough data to give a precise figure, and in any case, as the UBC study shows, cost is very much a factor of design. However, even without instructors costs, the UBC MOOCs at $54,000 each for between 8-12 weeks are already more than the average cost of a 13 week for-credit LMS-based online course, including instructor time.

This is partly due to the increased instructor time in preparation/production, but also to the higher cost of video production.  I am not against the use of video in principle, but it must add value. Using it for content transmission when this can be done so much more cheaply textually and/or by audio is a waste of the medium’s potential (although perhaps more motivating for the instructor).

More importantly, every institution contemplating MOOCs needs to do a cost-benefit exercise. Is it better to invest in MOOCs or credit-based online learning or both? If MOOCs are more expensive, what are the added benefits they provide and does this more than make up for not only the extra cost, but the lost opportunity of investing in (more) credit-based online learning or other forms of campus-based learning? I know what my answer would be.

 

Transforming university teaching and learning: UBC’s strategy for flexible learning

Listen with webReader
UBC campus, Vancouver BC

UBC campus, Vancouver BC

Flexible Learning Implementation Team (2014) Flexible Learning – Charting a Strategic Vision for UBC (Vancouver Campus. Vancouver BC: Office of the Provost, University of British Columbia

The University of British Columbia is one of Canada’s premier research universities with almost 60,000 students. It usually features within the top 30 universities worldwide in university rankings.

For the last 18 months, UBC has been developing a comprehensive strategy for teaching and learning for the future, and last week issued a report on its vision and how it plans to implement that vision. Although Flexible Learning is the term UBC has chosen to describe this strategy, it is in fact far more comprehensive and wide ranging than just blended or fully online learning. It is really about the transformation of teaching and learning in response to local, regional and global changes and challenges, based on a substantial amount of prior research, internal discussion, and input from external consultants (declaration of interest: I played a very small part in some of the early discussions of strategy).

First, the breaking news, then a summary of the main points from the strategy document.

Breaking news

This really represents the first concrete actions resulting from this strategic initiative.

  1. Research report published on UBC’s first four MOOCs: These MOOCs were delivered through the Coursera platform. I will cover this report in a separate blog post.
  2. Moving from Coursera to edX: UBC has now joined edX as a Charter Member, giving it a seat on edX’s Academic Advisory Board. UBC will develop four new MOOCs for delivery on edX in 2014-2015.
  3. Revamping Continuing and Professional Education: UBC has established, within the Provost’s Office, a new unit to work in close partnership with Faculties in developing both applied and access programs. More on this and how it affects the current Division of Continuing Studies later in this post.
  4. Improving the learning technology ecosystem: basically a response to widespread faculty disenchantment with the implementation of the latest version of UBC’s LMS, Blackboard Connect.

However, these four developments are literally the tip of an iceberg, which is much larger and more significant.

The strategic vision

As always, I recommend a careful reading of the whole 22 page document, even though it is not the easiest of reads. Any summary diminishes the complexity of the discussion, because there are so many inter-related themes and developments to which the university is attempting to respond. I provide this summary though in the hope that it will spike your interest enough to make the effort, as I see this document as one of the most significant for the future of public higher education in Canada – and elsewhere.

What does the university mean by flexible learning?

From the document (p.2)

We define Flexible Learning as UBC’s response to the opportunities and challenges presented by rapid advances in information and communication technologies, informed by the results of learning research and motivated by the objectives of improving student learning, extending access to UBC and strengthening university operating effectiveness.

See below for more detail on what that actually means.

What’s driving the change?

  • learner and employer expectations: need for a flexible workforce, greater flexibility in delivery and offerings, and more emphasis on measurable outcomes
  • demographics: increased global demand, with the local population of students older and often working
  • policy of governments (generally): growing reliance on tuition revenue; a belief that online learning is cheaper
  • disruptive technologies: MOOCs, cloud, mobile, adaptive learning, automated assessment, learning analytics…..

Market segmentation

Different categories of learners:

  • traditional university students (65% of the market), younger, mainly ‘commuting’: want rich campus-based learning experiences
  • convenience-driven degree-seekers: older, working, want blended/online learning
  • practitioners: seeking credentials for professional development; able to pay; under-represented to date at UBC
  • growth learners: seeking non-credentialed learning; a large and growing market segment.

All segments want more flexibility, both in delivery and range of content offerings.

Main objectives (for flexible learning)

  1. improved student learning
  2. expanded access to UBC content
  3. greater operating effectiveness

Main strategies

1. Strengthening UBC’s traditional role: through:

  • blended learning (including integration of MOOC content)
  • improving the campus experience and more personalization of learning through more modular programming
  • strategic academic program transformation

2. Revenue growth: through:

  • strategic expansion of continuing/professional education, especially applied master’s programs, certificates, badges
  • expanding access through ‘bridging’, e.g. PLA, MOOCs, summer programs

3. Academic partnerships (joining edX is one example)

Governance and management

The UBC Board and Executive approved the outline plan in 2013. Two teams were established within the Provost’s Office:

  • a leadership team, responsible for developing vision, strategy and policies, chaired by the Provost, with eight members
  • an implementation team, with another eight members, chaired by a Vice Provost.

Support is also provided by staff from the Centre for Teaching, Learning and Technology and from the IT Division, as well as designated contact people within each Faculty.

UBC has committed a total of $5 million ($1 million already spent) to support this initiative. (The total UBC annual operating budget is over $1 billion).

Comment

I’m watching this as someone completely outside the university. UBC is a very large and complex organization, once described by one former Provost as being managed by 12 barons all plotting to become king (although the climate is very different today). I cannot judge how far the reality of what’s happening on the ground differs from the vision, and in any case it is still very early days.

However, it is important to stress that this is a university-wide initiative (at least for the main Vancouver campus – UBC also has a semi-autonomous and much smaller campus in the interior of the province.) The strategy seems to have widespread support at the senior executive level, and a lot of momentum resulting from an infusion of significant money but more importantly as a result of widespread discussion and consultation within the university. Certainly the blended learning component is already getting a lot of traction, with some major re-designs of large undergraduate classes already in progress. How all this affects though the main body of the faculty and students at the hard edge of teaching and learning is impossible for me to judge.

The establishment of a new ‘hub’ within the Provost’s office for continuing and professional education (CPE) is particularly interesting since UBC has long had a strong and extensive Division of Continuing Studies, which offers a wide range of non-credit programming. However,

  • the ability to re-purpose existing content from credit courses into certificates, badges and non-credentialed offerings such as MOOCs,
  • the growing market for professional masters programs, especially online,
  • the increasing reconfiguration of higher education as a continuous lifelong learning escalator rather than a series of different, discrete floors (bachelors, masters, doctorates, non-credit),
  • the opportunities for revenue generation flowing directly back to the faculties,

all make essential a rethinking of the whole CPE activities of a university.

At the same time, the Division of Continuing Studies at UBC, as elsewhere, has many staff with a range of special skills and knowledge, such as

  • marketing,
  • direct access to employers and industry (often through the hiring of working professionals as part-time instructors),
  • the ability to identify and take risks with emerging content areas,
  • experience in operating in a highly market-driven, competitive cost-recovery/profit environment.

These are not attributes currently within the capacity or even interest of most academic departments. It will be an interesting challenge to see how the knowledge and experience of the Division of Continuing Studies can best be integrated with the new initiative, and how the new development in the Provost’s Office affects the operation of the Division of Continuing Studies.

Another critical factor is the appointment of a new President, who has pledged support for the strategy. However, he also said on his inauguration that the university will increase its base funding for research by at least $100-million. He did not specify though where the money would come from. I leave you to compare that to the $5 million allocated to this initiative and to judge how much impact finding another $100 million base funding for research might have on teaching and learning at UBC. I know, it’s not a zero sum game, but….

Overall, though, I find it heartening that UBC is showing such leadership and initiative in grappling with the major forces now impacting on public universities. It has a vision and a plan for teaching and learning in the future, that looks at teaching, technology, students and the changing external environment in an integrated and thoughtful manner, which in itself is a major accomplishment. It will be fascinating to see how all this actually plays out over time.

Is the ADDIE model appropriate for teaching in a digital age?

Listen with webReader
© Flexible Learning Australia, 2014

Click on the graphic for the interactive version © Flexible Learning Australia, 2014

Chapter 5 of my open textbook, ‘Teaching in a Digital Age‘, is now published. In Chapter 5, I developed the concept of a learning environment.

I am now working on Chapter 6, ‘Models for Designing Teaching and Learning.’ In my last post I discussed the appropriateness of the classroom design model for a digital age. In this post, I explore the same issue for the ADDIE model.

What is ADDIE?

ADDIE stands for:

Analyse

  • identify all the variables that need to be considered when designing the course, such as learner characteristics, learners’ prior knowledge, resources available, etc.  This stage is similar to describing the learning environment outlined in Chapter 5.

Design

  • this stage focuses on identifying the learning objectives for the course and how materials will be created and designed (for instance, it may include describing what content areas are to be covered and a storyboard outlining what will be covered in text, audio and video and in what order), and deciding on the selection and use of technology, such as an LMS, video or social media

Develop

  • the creation of content, including whether to develop in-house or outsource, copyright clearance for third party materials, recording videos or audio, loading of content into a web site or LMS, etc.

Implement

  • this is the actual delivery of the course, including any prior training or briefing of learner support staff, and student assessment

Evaluate

  • feedback and data is collected in order to identify areas that require improvement and this feeds into the design, development and implementation of the next iteration of the course.

The interactive infographic above provides an in-depth, step-by-step approach to the design of learning, with lots of online resources to draw on. There have been many books written about the ADDIE model (see for instance, Morrison, 2010; Dick and Carey, 2004).

Where is ADDIE used?

This is a design model used by many professional instructional designers for technology-based teaching. ADDIE has been almost a standard for professionally developed, high quality distance education programs, whether print-based or online. It is also heavily used in corporate e-learning and training. There are many variations on this model (my favourite is ‘PADDIE’, where planning and/or preparation are added at the start). The model is mainly applied on an iterative basis, with evaluation leading to re-analysis and further design and development modifications.

One reason for the widespread use of the ADDIE model is that it is extremely valuable for large and complex teaching designs. ADDIE ‘s roots go back to the Second World War and derive from system design, which was developed to manage the hugely complex Normandy landings.

The Open University in the United Kingdom heavily uses ADDIE to manage the design of complex multi-media distance education courses. When the OU opened in 1971 with an initial intake of 20,000, it used radio, television, specially designed printed modules, text books, reproduced research articles in the form of selected readings that were mailed to students, and regional study groups, with teams of often 20 academics, media producers and technology support staff developing courses, and with delivery and learner support provided by an army of regional tutors and senior counsellors. Creating and delivering its first courses without systematic instructional design model would have been impossible, and in 2014, with over 200,000 students, the OU still employs a strong instructional design model based on ADDIE.

Although ADDIE and instructional design in general originated in the USA, the Open University’s success in developing high quality learning materials influenced many more institutions that were offering distance education on a much smaller scale to adopt the ADDIE model, if on a more modest scale. As distance education courses became increasingly developed as online courses, the ADDIE model continued, and is now being used by instructional designers in many institutions for the re-design of large lecture classes, hybrid learning, and for fully online courses.

What are the benefits of ADDIE?

One reason it has been so successful is that it is heavily associated with good quality design, with clear learning objectives, carefully structured content, controlled workloads for faculty and students, integrated media, relevant student activities, and assessment strongly tied to desired learning outcomes. Although these good design principles can be applied with or without the ADDIE model, it is a model that allows these design principles to be identified and implemented on a systematic and thorough basis. It is also a very useful management tool, allowing for the design and development of large numbers of courses to a standard high quality.

What are the limitations of ADDIE?

The ADDIE approach can be used with any size of teaching project, but works best with large and complex projects. Applied to courses with small student numbers and a deliberately simple or traditional classroom design, it becomes expensive and possibly redundant, although there is nothing to stop an individual teacher following this strategy when designing and delivering a course.

A second criticism is that the ADDIE model is what might be called ‘front-end loaded’ in that it focuses heavily on content design and development, but does not pay as much attention to the interaction between instructors and students during course delivery. It has been criticised by constructivists for not paying enough attention to learner-instructor interaction, and for privileging more behaviourist approaches to teaching.

Another criticism is that while the five stages are reasonably well described in most descriptions of the model, it does not provide guidance on how to make decisions within that framework. For instance, it does not provide guidelines or procedures for deciding how to choose between different technologies, or what assessment strategies to use. Instructors have to go beyond the ADDIE framework to make these decisions.

The over-enthusiastic application of the ADDIE model can and has resulted in overly complex design stages, with many different categories of workers (faculty, instructional designers, editors, web designers) and consequently a strong division of labour, resulting in courses taking up to two years from initial approval to actual delivery. The more complex the design and management infrastructure, the more opportunities there are for cost over-runs and very expensive programming.

My main criticism though is that the model is too inflexible for the digital age. Adamson (2012) states:

The systems under which the world operates and the ways that individual businesses operate are vast and complex – interconnected to the point of confusion and uncertainty. The linear process of cause and effect becomes increasingly irrelevant, and it is necessary for knowledge workers to begin thinking in new ways and exploring new solutions.

In particular knowledge workers must deal with situations and contexts that are volatile, uncertain, complex and ambiguous (what Adamson calls a VUCA environment). This certainly applies to teachers working with ever changing technologies, very diverse students, a rapidly changing external world that puts pressure on institutions to change.

If we look at course design, how does a teacher respond to rapidly developing new content, new technologies or apps being launched on a daily basis, to a constantly changing student base? For instance, even setting prior learning outcomes is fraught in a VUCA environment, unless you set them at an abstract ‘skill’ level such as thinking flexibly, networking, and information retrieval and analysis. Students need to develop the key knowledge management skills of knowing where to find relevant information, how to assess, evaluate and appropriately apply such information. This means exposing them to less than certain knowledge and providing them with the skills, practice and feedback to assess and evaluate such knowledge then apply that to solving real world problems.

This means designing learning environments that are rich and constantly changing, but enable students to develop and practice the skills and acquire the knowledge they will need in a VUCA world. I would argue that while the ADDIE model has served us well in the past, it is too pre-determined, linear and inflexible to handle this type of learning. I will discuss more flexible models later in this chapter.

Over to you

1. Have I given enough information about what ADDIE is, by using the infographic, or do I need to cover this more fully in the text? Do I need to say something about rapid course development here?

2. What are your views on the ADDIE model? Is it a useful model for designing teaching in a digital age? Do you agree with my criticisms of the model?

3. Any suggestions about other, more flexible models that could be used?

What’s next

So far I have done drafts of the following (as blogs)

  • What is a design model?
  • The classroom model
  • Classroom models in online learning
    • LMSs
    • lecture capture
  • ADDIE

Still to come:

  • Competency-based learning,
  • Connectivist models, including Communities of practice and cMOOCs
  • Flexible design models
  • PLEs
  • AI approaches.
  • Conclusion

My next post in this series then will be on the appropriateness of competency-based learning for teaching in a digital age.

References

Adamson, C. (2012) Learning in a VUCA world, Online Educa Berlin News Portal, November 13

Dick, W., and Carey, L. (2004). The Systematic Design of Instruction. Allyn & Bacon; 6 edition Allyn & Bacon

Morrison, Gary R. (2010) Designing Effective Instruction, 6th Edition. New York: John Wiley & Sons 

Is the classroom model appropriate for teaching in a digital age?

Listen with webReader
© University of Science and Arts Oklahoma

© University of Science and Arts Oklahoma

Chapter 5 of my open textbook, ‘Teaching in a Digital Age‘, is now published. In Chapter 5, I developed the concept of a learning environment.

I am now working on Chapter 6, ‘Models for Designing Teaching and Learning.’

What is to be covered in Chapter 6

This may change as I get into the writing but my plan at the moment is to cover the following topics:

  • What is a design model?
  • The classroom model
  • Classroom models in online learning
    • LMSs
    • lecture capture
  • ADDIE
  • Competency-based learning,
  • Communities of practice (inc. cMOOCs)
  • Flexible design models
  • PLEs
  • AI approaches.
  • Conclusion

In this post I introduce the concept of a design model and discuss the appropriateness of the classroom design model for a digital age. My next post, which follows almost immediately, does the same for the ADDIE model.

Purpose of the chapter

At the end of this chapter the reader should be able to:

  1. Describe key models or approaches to the design of teaching and learning
  2. Analyse each model in terms of its value for teaching in a digital age
  3. Decide which model or combination of models will fit best with their own teaching
  4. Use the model as a basis for designing their own teaching

What is a design model?

By a design model, I mean the organized steps taken to convert a desired learning environment into teaching and learning activities. Project management is a typical example of a design model, in that it presents a framework for taking a plan or goal and turning it into action. In project management, there are certain steps to be followed which are relatively independent of whatever project is being implemented.

However, there are many different kinds of approaches to design implementation besides project management. I intend to examine several of the most common design models that can be used in teaching, and in particular to examine them for their suitability for teaching in a digital age.

The classroom design model

Classroom old 2

Some design models are so embedded in tradition and convention that we are often like fish in water – we just accept that this is the environment in which we have to live and breath. The classroom model is a very good example of this. In a classroom based model, learners are organised in classes that meet on a regular basis at the same place at certain times of the day for a given length of time over a given period (a term or semester).

This is a design decision that was taken more than 150 years ago. It was embedded in the social, economic and political context of the 19th century. This context included:

  • the industrialization of society which provided ‘models’ for organizing both work and labour, such as factories and mass production
  • the movement of people from rural to urban occupations and communities, with increased density resulting in larger institutions
  • the move to mass education to meet the needs of industrial employers and an increasingly large and complex range of state-managed activities, such as government, health and education
  • voter enfranchisement and hence the need for a better educated voting public
  • over time, demand for more equality, resulting in universal access to education.

The large urban school, college or university, organized by age stratification, learners in groups, and regulated units of time was an excellent fit for such a society. In effect, we still have a predominantly factory model of educational design, which in large part remains our default design model even today.

However, over the span of 150 years, our society has slowly changed. Many of these factors or conditions no longer exist, while others persist, but often in a less dominant way than in the past. Thus we still have factories and large industries, but we also have many more small companies, greater social and geographical mobility, and above all a massive development of new technologies that allow both work and education to be organized in different ways. This is not to say that the classroom design model is inflexible. Teachers for many years have used a wide variety of teaching approaches within this overall model.

I don’t want to devote much space to the classroom design model, as we are all so familiar with it, and there is so much invested in the ‘default’ model that it is impractical to rip everything up and start with something completely different. Nevertheless, we have at least the seeds of change already showing. ‘Flipped’ classrooms where students get lectures on video and come to class for discussion and the re-design of large lecture classes are moves to modify the default model, while fully online programs and MOOCs are a manifestation of more radical change by offering education at any time and any place.

The real danger though is that we fail to grasp the opportunities that are now available to us, because we are so comfortable and familiar with the classroom design model. Even worse is trying to force the old default model on to new developments, when what is needed is a totally different approach if we are to meet the needs of a digital age. I give two examples below of forcing new technologies into the old classroom design model.

Old wine in new bottles: classroom-type online learning

When commercial movies were first produced, they were basically a transfer of previous music hall and vaudeville acts to the movie screen. Then along came D.W. Griffith’s ‘Birth of a Nation’, which transformed the design of movies, by introducing techniques that were unique to cinema at the time, such as panoramic long shots, panning shots, realistic battle scenes, and what are now known as special effects.

Learning management systems

Most learning management systems, such as Blackboard, Desire2Learn and Moodle, are in fact a replication of a classroom design model. They have weekly units or modules, the instructor selects and presents the material to all students in the class at the same time, a large class enrollment can be organized into smaller sections with their own instructors, there are opportunities for (online) discussion, students work through the materials at roughly the same pace, and assessment is by end-of-course tests or essays.

The main design differences are that the content is primarily text based rather than oral, the online discussion is asynchronous rather than synchronous, and the course content is available at any time from anywhere with an Internet connection. These are important differences, and skilled teachers and instructors can modify or adapt LMSs to meet different teaching or learning requirements (as they can in physical classrooms), but the basic organizing framework of the LMS remains the same as for a physical classroom.

Nevertheless, the LMS is still an advance over online designs that merely put lectures on the Internet or load up pdf copies of Powerpoint lecture notes, as is still the case unfortunately in many online programs. Good online design should take account of the special requirements of online learners, so the design needs to be different from that of a classroom model.

Lecture capture

This technology, which automatically records a classroom lecture, was originally designed to enhance the classroom model by making lectures available for repeat viewings online at any time for students regularly attending classes – in other words, a form of homework. Flipped classrooms are an attempt to exploit more fully this potential, but the biggest impact has been the use of lecture capture for ‘instructionist’ massive open online courses (MOOCs), such as those offered by Coursera, Udacity and edX. However, even this type of MOOC is really a basic classroom design model. The main differences are that the classroom is open to anyone (but then in principle so are many university lectures), and MOOCs are available to unlimited numbers at a distance. These are important differences again, but the design of the teaching – lectures delivered in chunks – has not changed markedly.

‘Instructionist’ MOOCs have resulted in some important design changes to the classroom model, such as using computer-marked assignments to test students or give feedback, and the use of peer review (both often used also in physical classroom design of course), but the predominant design model of instructionist MOOCs is that of an admittedly massive classroom.

The limitations of the classroom design model

Old wine can still be good wine, whether the bottle is new or not. What matters is whether classroom design meets the changing needs of a digital age. Just adding technology to the mix, or delivering the same design online, does not automatically result in meeting changing needs. It is important then to look at the design that makes the most of the educational affordances of new technologies, because unless the design changes significantly to take full advantage of the potential of the technology, the outcome is likely to be inferior to that of the physical classroom model which it is attempting to imitate.

The second danger of just adding new technology to the classroom design is that we may just be increasing cost, both in terms of technology and the time of instructors, without changing outcomes. Thus even if the new technology, such as lecture capture and computer-based multiple-choice questions organised in a MOOC, result in helping more students memorise better or learn more content, for example, this may not be sufficient to meet the higher level skills needed in a digital age.

Education is no exception to the phenomenon of new technologies being used at first merely to reproduce earlier design models before they find their unique potential. However, changes to the basic design model are needed if the demands of a digital age and the full potential of new technology are to be exploited in education.

Over to you

1. Do I manage to make clear what I mean by design ‘models’? If not, how can this be made clearer – or is the concept not helpful in the first place?

2. Do you agree that the classroom design model is a product of the 19th century and needs to changed for teaching in a digital age? Or is there still enough flexibility in the classroom model for our times?

3. To what extent do you feel you have to teach in a certain way because of the classroom model – or are you able to work flexibly within this model?

4. Do you agree that LMSs are basically a classroom model delivered online, or are they a unique design model in themselves. If so, what makes them unique?

What’s next?

My next post looks at the appropriateness of the ADDIE model for teaching in a digital age.

Is there a Canadian market for American online programs?

Listen with webReader
© Wikipedia, 2014

Image: © Wikipedia, 2014

Identifying the market

I got a phone call last week from an international consultancy company contracted on behalf of a major U.S. state university, wanting to know if I thought there was a Canadian market for U.S. online degrees. Since I didn’t get paid for my advice, I’m willing to share it with anyone who’s interested. (I do find it annoying that universities will pay large sums to general business consultants such as Price Waterhouse, KPMG, Accenture, etc., who then expect professionals in the field to do their work for them for free.)

So I will share with you the short answer I gave to the consultants, then the long one which only you have access to!

The short answer is yes, but only in niche markets. This means doing careful market research (not just ringing up a few experts), developing a strong business plan, keeping tuition fees relatively low compared with those at the top end of the market in the USA, and being nimble, in that the market will not last long if they are successful, because Canadian institutions will eventually try to take that market back.

Now for the long answer.

Why Canada is not an easy market for foreign online programs

I don’t have any figures, but there are certainly some Canadian students already taking online degrees from U.S. institutions (if you are one of them, I’d be interested in your experience.) Why would they do that, you might ask? Well,

  • if they want to get a job in the USA or work for an American company in Canada
  • if there is no Canadian university offering the program either face-to-face locally or online
  • because they think a big name U.S. university will do a better job than their local Canadian universities.

I’m not saying they are right to think that, but the whole question of comparative quality in post-secondary educations, either in Canada or the USA, is so fraught and subjective that potential students still have a very hard job making those kinds of decisions.

However, for that very reason, better the devil you know. In general, I would argue that most potential students in Canada will look to one of their local universities as their first choice, for several reasons:

  • Cost. Tuition fees are generally lower in Canada in absolute terms, but also Canadians may be eligible for grants and scholarships for study, which probably will not apply to online programs offered by a program from the USA (although again there are exceptions.)
  • Quality. Nearly all Canadian universities are publicly funded and accredited by the provincial government. There are of course differences in status, but the system is simple enough for most Canadians to be aware of these differences, and the variation between institutions in Canada is nowhere near as great as in the much larger and much more diverse US higher education system. In particular, distance and online education in the USA has been tarnished much more than in Canada by private, for profit diploma mills, making many Canadians suspicious then of any online programs offered from the USA
  • Access. In general, most Canadians can get access to the programs they need from one or more of their local universities. 51% of Canadians go on from high school to university, and 60% to some form of publicly-funded post-secondary education. There are already roughly one million course enrollments a year in online courses from Canadian post-secondary institutions, or roughly about 20-25 per cent of all course enrollments

So the Canadian market for post-secondary education in general is what marketers call ‘mature’ and even online learning, while not mature, is certainly not in its infancy in Canada. Many Canadian universities already offer a range of online programs, both for credit and for non-credit.

For instance, there are several online/distance MBAs (Athabasca University, Queen’s/Cornell, Laurentian, for instance) and UBC has been running several very successful online masters programs in education, creative writing and health sciences. This is a market where institutions can more than cover their costs within the current range of government-regulated tuition fees; indeed UBC has been able to add new research faculty as a result of some their online masters programs.

The Canadian Virtual University (CVU), which is a consortium of 11 universities from across Canada, lists over 2,000 online courses from its members, including a total of 57 online masters programs. There are 16 online masters programs in Ontario, Canada’s largest province (with 20 universities). Of the online masters currently available from Ontario institutions, one third are in education, one third in business or leadership, and one third in health or social work (see Ontario Online Learning Portal for Students). There are a few other universities in Canada that are not members of the CVU or in Ontario that also offer online programs, and the number grows each year. Thus there is already a significant Canadian university (and college) ‘presence’ in the online learning area.

Furthermore once a U.S. program starts attracting large numbers of Canadian students, this is likely to draw a response from the Canadian institutions. This happened in British Columbia in the 1990s, when a large number of teachers started taking distance education masters’ programs from Washington State institutions such as Gonzaga University, because there was little available in British Columbia. It took a few years but the B.C. institutions eventually responded and the number of teachers taking out of province online programs is now very much reduced.

Lastly there are more subtle cultural barriers. Canada is not so different from the USA as Canadians like to believe, but there are differences, and they are important. Our laws and government regulations for instance are different. We have different iconic references, different national symbols, a different racial mix. This means content does not always transfer as seamlessly as you would think for countries sharing a similar language and continent. This is not an insurmountable obstacle, but it does require a certain sensitivity to ‘Canadian-ness’ that is not always there in the USA. However, it is probably too costly to adapt existing materials for a specifically Canadian market.

So there are several reasons why some caution would be needed in marketing online programs from the USA for a Canadian market (or anywhere else, such as the U.K. or Australia).

Why there is still a market for foreign online programs in Canada

But Canadian universities should not be complacent. Canadian institutions have been slower than their counterparts in the USA in entering the online professional masters’ market. There are many gaps in the availability of post-graduate online programs in Canada, especially in the STEM (science, technology, engineering, math) areas.

For instance, in Ontario there are no online masters’ programs in food and agriculture, humanities (except Design from OCAD), social sciences, math, science, law or engineering. The situation would be similar or worse in the other provinces. In particular, despite the CVA, which covers only 11 of the 80 or so universities in Canada, there is no national portal for Canadian online courses, so potential students have a hard time looking for online programs offered out of province.

Secondly, should Harvard, Stanford or MIT start offering for-credit online programs, there would certainly be a large take-up in Canada of such programs. For the still prestigious, Tier 1 research state universities, such as the University of California, Arizona State University or Penn State, brand recognition in Canada would be less compelling if there are suitable Canadian alternatives, although for graduate programs, many Canadian students might still be interested.

Lastly, good marketing is likely to attract potential students in Canada who may well not be aware that similar programs are already available in Canada where they do exist. Even as a so-called expert in Canadian online learning, I was unaware of most of the programs listed in the CVA and Contact North portals. Canadian universities are still remarkably provincially focused when it comes to marketing online programming. Which brings me to my next point.

Why don’t Canadian institutions market online programs in the USA?

Well, some do, but not many, and there are good reasons for this.

First the U.S. accreditation system is byzantine and bizarre, and totally ill-adapted to the move to online, distance education, but without accreditation from one of the regional accreditation agencies in the U.S., any Canadian online program would not be recognized for financial assistance, credit transfer or by many employers. Indeed (although the federal regulations are currently under discussion) it may be illegal to offer programs to students within a state without approval from that state (and to get approval you may need a ‘physical presence.’) Despite the very large cost in getting regional accreditation, some specialist Canadian institutions, such as Athabasca University, have done so. I’d be interested to know though if they feel it was worth it, in terms of the numbers of U.S. students it has attracted. (Yes, you are correct: the U.S.A and Canada are partners in a free trade association called NAFTA, but it doesn’t apply to education – or forestry, for that matter.)

Second, the reverse is true regarding brand recognition and local preferences. Many U.S. citizens don’t even know where Canada is, let alone know whether the University of Waterloo is a bona fide institution and not a diploma mill. (I once shared an airport shuttle bus in Los Angeles with two American businessmen complaining they needed a passport to go to Toronto. ‘Anyone would think it was in another country,’ one said. I’m not making it up, and for British Columbian’s it may well be.)

So Canada, and Canadian institutions, do not have a strong enough brand image for most Americans, although some institutions, such as the University of Windsor, have done an excellent job in building partnership with institutions just across the border in Michigan and Detroit – but not, so far, in online learning.

Why these are the wrong questions

The real question should be (for both Canadian and U.S. online program developers): Is there an international market for online professional masters’ programs, to which the answer is a resounding ‘yes.’

It really doesn’t matter what country a student is in. If the program being offered meets a need, and the student can afford it, then they will take it, whether it comes from the USA, Canada, Australia, Spain, India or Brazil. Brand recognition helps. Canadian public institutions produce high quality, government accredited online programs. There is no distinction in the degree transcripts whether the program is offered at a distance or on campus. With its rather monolithic public higher education system, it would have an advantage if it could only market a Canadian rather than a local or provincial brand (but then it has the same problem in attracting international students to on-campus programs).

U.S. states trying to regulate whether students can take a program from out of state are like King Canute trying to stop the tide from coming in. All you can do is say ‘Caveat Emptor’ – buyer beware.

One way round the ‘not invented here’ attitude to out of state or out of country programs though is to develop a partnership with local institutions of a similar ‘status’ level. The University of British Columbia developed a very successful partnership with Tec de Monterrey in Mexico for a graduate program in educational technology. UBC developed most of the content, but the program was available in Spanish from Tec de Monterrey and in English from UBC. UBC’s offering of the program drew almost 30 per cent of its enrollments initially from English-speaking students from other countries all round the world. Similarly, Tec de Monterrey marketed its program in South America as well as in Mexico. Queen’s University in Canada has partnered with Cornell University in the USA to market a joint online MBA. These partnerships help to get round the accreditation issues.

Admission of international students to an online program can be more of a problem, as can issues of security in terms of exams and assessment, but these are not insuperable problems. If the institution already has an admission policy for on-campus international students this should apply just as well to online students from another country. Local proctoring (again using a partner institution) can help with examinations, but even better is to use authentic assessments where students have to apply their knowledge locally or within their own work or social context. This form of assessment is ideal for professional masters ‘ programs.

Furthermore, it helps when initially planning an online program to take into consideration that there is probably an international market (indeed, this helps with marketing the program even locally). This means choosing content carefully so it can work in a wide variety of international contexts, and to take advantage of the different perspectives from the international students.

Lastly, when developing a business model, the inclusion of international students can make the difference between loss and profit, because in well-designed online programs, the marginal cost of each additional student should more than cover the delivery costs. UBC found it did not need to charge a premium tuition fee for international students, because the business model depended on getting enough enrollments at the government-regulated tuition fee (for domestic students) to break even. The international students were an important contributor to breaking even without having to charge them a premium price. So pricing policy is also an important factor.

In conclusion

Yes, there is a market for international students in online programs, and there will be a market in Canada for specific professional masters offered by well-recognised and legitimate U.S. universities, provided they fill a gap not currently being met by Canadian institutions – and at the moment there are many such gaps.

However, it may be a mistake to focus on marketing to specific countries. Institutions will (even in Canada) respond relatively quickly to what they may feel as foreign incursions into their market. The world market (especially for programs in the English language) is large enough to attract more than enough high quality students to credit-based programs – just look at the response to MOOCs, which don’t offer formal qualifications. The trick is good marketing, making sure the materials take into consideration an international market and are not focused on specifically local or national issues, and having a good business plan. And partnership with foreign institutions would be a wise move. So while not easy, it is feasible, but if the reason is solely to make money, then it will, ironically, be a harder sell. Students will pick up on that remarkably quickly.

 

 

 

 

 

 

Adapting student assessment to the needs of a digital age

Listen with webReader

Assessment 2

The story so far

Chapter 5 of my open textbook, ‘Teaching in a Digital Age’ is about the design of teaching and learning, which I am currently writing and publishing as I go.

I started Chapter 5 by suggesting that instructors should think about design through the lens of constructing a comprehensive learning environment in which teaching and learning will take place. I have been working through the various components of a learning environment, focusing particularly on how the digital age affects the way we need to look at some of these components.

I briefly described some of the key components of an effective learning environment in a series of blog posts:

In this post, I examine the assessment of students as a key component, and how assessment methods need to be adapted to meet the needs of a digital age. This is the last component I’m discussing, but it will be followed by a final post that discusses the value of designing teaching and learning through the lens of a comprehensive learning environment.

Learner assessment

‘I was struck by the way assessment always came at the end, not only in the unit of work but also in teachers’ planning….Assessment was almost an afterthought…

Teachers…are being caught between competing purposes of …assessment and are often confused and frustrated by the difficulties that they experience as they try to reconcile the demands.’

Earle, 2003

Learner assessment in a digital age

Because assessment is a huge topic, it is important to be clear that the purpose of this section is (a) to look at one of the components that constitute an effective and comprehensive learning environment, and (b) briefly to examine the extent to which assessment is or should be changing in a digital age. Assessment will be a recurring theme in this book, so in this section the treatment is deliberately cursory.

Probably nothing drives the behaviour of students more than how they will be assessed. Not all students are instrumental in their learning, but given the competing pressures on students’ time in a digital age, most ‘successful’ learners focus on what will be examined and how they can most effectively (i.e. in as little time as possible) meet the assessment requirements. Therefore decisions about methods of assessment will in most contexts be fundamental to building an effective learning environment.

The purpose of assessment

There are many different reasons for assessing learners. It is important to be clear about the purpose of the assessment, because it is unlikely that one single assessment instrument will meet all assessment needs. Here are some reasons (you can probably think of many more):

  1. to improve and extend students’ learning
  2. to assess students’ knowledge and competence in terms of desired learning goals or outcomes
  3. to provide the teacher/instructor with feedback on the effectiveness of their teaching and how it might be improved
  4. to provide information for employers about what the student knows and/or can do
  5. to filter students for further study, jobs or professional advancement
  6. for institutional accountability and/or financial purposes.

I have deliberately ordered these in importance for creating an effective learning environment. In terms of the needs of a digital age, assessment needs to focus on both developing and assessing skills. This means that continuous or formative assessment will be as important as summative or ‘end-of-course’ assessment.

A question to be considered is whether there is a need for assessment of learning in the first place. There may be contexts, such as a community of practice, where learning is informal, and the learners themselves decide what they wish to learn, and whether they are satisfied with what they have learned. In other cases, learners may not want or need to be formally evaluated or graded, but do want or need feedback on how they are doing with their learning. ‘Do I really understand this?’ or ‘How am I doing compared to other learners?’

However, even in these contexts, some informal methods of assessment by experts, specialists or more experienced participants could help other participants extend their learning by providing feedback and indicating the level of competence or understanding that a participant has achieved or has yet to accomplish. Lastly, students themselves can extend their learning by participating in both self-assessment and peer assessment, preferably with guidance and monitoring from a more knowledgeable or skilled instructor.

Methods of assessment

The form the assessment takes, as well as the purpose, will be influenced by the instructors’ or examiners’ underlying epistemology: what they believe constitutes knowledge, and therefore how students need to demonstrate their knowledge. The form of assessment should also be influenced by the knowledge and skills that students need in a digital age, which means focusing as much on assessing skills as knowledge of content.

There is a wide range of possible assessment methods. I have selected just a few to illustrate how technology can change the way we assess learners in ways that are relevant to a digital age:

  • computer-based multiple-choice tests: good for testing ‘objective’ knowledge of facts, ideas, principles, laws, and quantitative procedures in mathematics, science and engineering etc., and are cost-effective for these purposes. This form of testing though tends to be limited  in assessing high-level intellectual skills, such as complex problem-solving, creativity, and evaluation, and therefore less likely to be useful for developing or assessing many of the skills needed in a digital age.
  • written essays or short answers: good for assessing comprehension and some of the more advanced intellectual skills, such as critical thinking, but are labour intensive, open to subjectivity, and are not good for assessing practical skills. Experiments are taking place with automated essay marking, using developments in artificial intelligence, but so far automated essay marking still struggles with reliably identifying valid semantic meaning (for a balanced and more detailed account of the current state of machine grading, see Mayfield, 2013Parachuri, 2013).
  • project work: either individual but more commonly group-based, project work encourages the development of authentic skills that require understanding of content, knowledge management, problem-solving, collaborative learning, evaluation, creativity and practical outcomes. Designing valid and practical project work needs a high level of skill and imagination from the instructor.
  • e-portfolios (an online compendium of student work): enables self-assessment through reflection, knowledge management, recording and evaluation of learning activities, such as teaching or nursing practice, and recording of an individual’s contribution to project work (as an example, see  the use of e-portfolios in Visual Arts and Built Environment at the University of Windsor.); usually self-managed by the learner but can be made available or adapted for formal assessment purposes or job interviews
  • simulations, educational games (usually online) and virtual worlds: facilitate the practice of skills, such as complex and real time decision-making, operation of (simulated or remote) complex equipment, the development of safety procedures and awareness, risk taking and assessment in a safe environment, and activities that require a combination of manual and cognitive skills (see the training of Canadian Border Service officers at Loyalist College, Ontario). Currently expensive to develop, but cost-effective with multiple use, where it replaces the use of extremely expensive equipment, where operational activities cannot be halted for training purposes, or  where available as open educational resources.
Virtual world border crossing, Loyalist College, Ontario

Virtual world border crossing, Loyalist College, Ontario

It can be seen that some of these assessment methods are both formative, in helping students to develop and increase their competence and knowledge, as well as summative, in assessing knowledge and skill levels at the end of a course or program.

In conclusion

Nothing is likely to drive student learning more than the method of assessment. At the same time, assessment methods are rapidly changing and are likely to continue to change. Assessment in terms of skills development needs to be both ongoing and continuous as well as summative. There is an increasing range of digitally based tools that can enrich the quality and range of student assessment. Therefore the choice of assessment methods, and their relevance to other components, are vital elements of any effective learning environment.

Over to you

Your views, comments and criticisms are always welcome. In particular:

  • are there other methods of assessment relevant to a digital age that I should have included?
  • there is still a heavy reliance on computer-based multiple-choice tests in much teaching, mainly for cost reasons. However, although there are exceptions, in general these really don’t assess the high level conceptual skills needed in a digital age. Are there other methods that are equally as economical, particularly in terms of instructor time, that are more suitable for assessment in a digital age? For instance, do you think automated essay grading is a viable alternative?
  • would it be helpful to think about assessment right at the start of course planning, rather than at the end? Is this feasible?

Or any other comments on assessment as a critical component of a learning environment, please!

Next up

Why thinking in terms of a comprehensive learning environment is necessary but not sufficient when designing a course or program.

 

References

Earle, L. (2003) Assessment as Learning Thousand Oaks CA: Corwin Press

Mayfield, E. (2013) Six ways the edX Announcement Gets Automated Essay Grading Wrong, e-Literate, April 8

Parachuri, V. (2013) On the automated scoring of essays and the lessons learned along the way, vicparachuri.com,  July 31