August 2, 2015

Thinking about theory and practice in online learning

Listen with webReader
Taking the float plane to Victoria: always a wonderful experience

Taking the float plane to Victoria: always a wonderful experience

I ran a short face-to-face workshop yesterday on ‘Thinking about Theory and Practice’ for about a dozen students taking the Masters of Arts in Learning and Technology at Royal Roads University  My online open textbook, Teaching in a Digital Age, is being used in this program and the instructors asked me to run a workshop on this topic, as students struggle with the relationship between epistemology, theories of learning, and methods of teaching.

The exercise

I’m not surprised that students struggle with this, as the relationships are by no means clear. I started by asking them to define different epistemologies. I then asked them what the connection was between different epistemologies and different learning theories. Then I asked them to choose from about 18 different methods or approaches to teaching (all covered in my book) and try to place them in relationship to theories of learning, as in Figure 1 below.

Figure 1: Thinking about theory and practice

Figure 1: Thinking about theory and practice

I also raised questions about whether constructivism and connectivism are epistemologies, or theories of learning, or both.

This was meant as a heuristic exercise, to get students arguing about and discussing the relationship between epistemology, theory, and practice, and why it is important to think about this in terms of learning design.

I ended my session with the following questions:

  • Constructivism and connectivism: are they epistemologies or learning theories?
  • Is there a direct relationship between epistemology, theory and practice?
  • How well do different teaching methods ‘fit’ with a specific learning theory?
  • Does technology change the nature of knowledge? If so, is connectivism an ‘adequate’ epistemology for a digital age?

Following my workshop, in the afternoon the students were divided into two teams to formally debate the motion (chosen by the instructors):

Connectivism should be adopted as the learning theory for educating students in our digital culture.

Both the workshop and the debate resulted in very thoughtful and forceful, sometimes impassioned, discussion.

Outcomes

It is impossible to capture the richness of the discussions in a short blog (I am hoping that the MALAT team will make an edited recording of the sessions available online). Different participants will have come away from the two sessions with different conclusions. Although I am fairly confident about discussing theories of learning and methods of teaching, I am not a trained or qualified philosopher, so I hesitate to tell students what the truth is in this area (OK, so I’m a relative constructivist).

However, here are some of my conclusions:

  • the most important is that I believe that connectivism is more of an epistemology than a theory of learning. Indeed it is an epistemology that relies on other theories of learning to explain how learning occurs in networks, although it has established conditions that make for ‘effective’ networks (see, for instance, Downes, 2007). In this sense it can be seen as an overall belief system about the importance of networks for sustaining and creating knowledge, but the mechanisms by which learning occurs in networks still need to be identified or worked out, or explained in terms of existing theories, such as constructivism.  This does not mean that over time, particular ways of learning and creating new knowledge through networking will not be identified, but more importantly, it would seem to make sense that we should be making use of networks and social media in education, since we are all becoming increasingly immersed in a connectivist world, and learning how to adapt and thrive in such a world probably requires using connections and networks for teaching and learning;
  • similarly, I am uncomfortable with defining constructivism as an epistemology. It is a strong theory in terms of explaining how learning occurs, but it takes its philosophical roots from other more general epistemologies. I would need to be a philosopher to define accurately what these would be, but constructivism is strongly influenced by philosophers such as John Stuart Mill (free will), Jean Jacques Rousseau (the Natural Human), and Jean Piaget (‘genetic’ epistemology);
  • although there is some relationship between epistemologies and theories of learning, they are not isomorphic, in the sense that a single theory of learning derives solely from one epistemological position. For instance, cognitive theories of learning draw heavily on both objectivist approaches (e.g. brain research) and more subjective or reflective approaches, such as constructivism;
  • there is even less isomorphism between theories of learning and methods of teaching, because methods of teaching are driven primarily by context. For instance, in a digital age, trades apprentices increasingly need both manual and cognitive learning. The learning of manual or mechanical skills through an apprenticeship model may be behaviourist in approach, but cognitive apprenticeship may draw much more heavily on a constructivist approach. Nevertheless some teaching methods, such as lectures or xMOOCs, are generally more towards the objectivist spectrum, while cMOOCs are more towards the connectivist spectrum (even though in practice they may include other approaches, such as more objectivist webinars, and support from teachers or experts through constructivist forms of discussion);
  • different subject areas tend to favour different epistemological positions, such as science favouring more objectivist approaches to teaching, and arts more subjective and interpretive approaches. However, it is still possible to teach science in a constructivist way – for instance through problem or inquiry-based learning – and arts in a more objectivist way (for instance, Mrs. Thatcher wanted British school children to learn the facts about British history, rather than discuss imperialism or racism and their legacies), although purists will argue that students will not become ‘true’ scientists or historians if the teaching does not reflect the ‘core’ epistemological nature of the subject area.

However, I’m a ‘relativist’ on all these points and open to be persuaded.

Does it matter?

Isn’t this all terribly abstract and philosophical? Nothing seems clear and definite, so how does thinking about these things help to teach better?

Well, if you are going to be an instructional designer, you will come across instructors and subject experts who may have a fundamentally different epistemological position from you. It will really help if you understand their position and how to take this into account when designing courses.

Second, there is nothing more practical than a good theory. If you have a theory that is convincing to you in terms of explaining how learners best learn, this should drive your teaching practice. It may not tell you exactly what to do as a teacher, but it should enable you to work out for yourself what to do – and more importantly, what learners need to do. But this theory needs to fit with your overall epistemological position about the nature of knowledge in your subject area.

Third, teaching is a pragmatic profession. It may take several different approaches, depending on the context and above all on the learner. In some contexts, such as safety compliance, employers don’t want workers questioning the process; they need to learn exactly what to do in a particular circumstance (behaviourism rules). In others, where problem-solving is essential, rote learning is not going to help dealing with a new or unanticipated danger.  Having a range of options in terms of teaching approaches for a range of different kinds of learners and contexts is more likely to produce results than slavishly following one particular method.

Lastly, all this uncertainty and choice illustrates why teaching and learning are not well defined activities that can be easily mechanised. Humans are better than machines at dealing with uncertainty and fuzzy or ambiguous circumstances, but only if they have a deep understanding of the options available to them and the circumstances in which each option is likely to succeed. This means thinking carefully about epistemology and theories of learning as well as various methods of teaching.

Galiano Island, on the way to Victoria

Galiano Island, on the way to Victoria. Vancouver Island is in the background.

Some (further) thoughts about ‘agile’ learning design

Listen with webReader
Peter Rawsthorne's model of agile learning design

Peter Rawsthorne’s model of agile learning design (see references at end)

In my online, open textbook, Teaching in a Digital Age, I felt I needed a separate section on agile learning design, to capture some of the innovative teaching that is happening online. I based the section in the book very much on ETEC 522, ‘Ventures in Learning Technology‘, which is part of the University of British Columbia’s Master in Educational Technology. The course is designed and taught by two very innovative adjunct professors, David Voigt and David Porter, supported by Jeff Miller, a brilliant instructional designer.

As soon as I finished the book, I discovered that there was in fact a small but significant literature on agile learning design, and that there were other people ‘out there’ practising agile learning design.

So when I was approached by the British Columbia Educational Technology Users’ Group (ETUG) to do a ‘Tuesday’s ETUG Lunch and Learn’ (T.e.l.l) webinar on any aspect covered in the book, agile learning design was an obvious choice. ETUG is a great community of practice, and there were bound to be several agile learning practitioners in the group. So I prepared a few slides, and then used the webinar as an opportunity to have a professional ‘chat’ about agile learning design. Here’s what ensued (a recording of the whole webinar will be available from ETUG shortly and I will add the link as soon as it becomes available).

Defining agile learning design

Well, I did my best to define it in both Scenario F and Section 4.7 ‘Agile Design': Flexible Designs for Learning’. Originally I started to describe this teaching method as flexible design, but because flexible learning has a broader and more widely used meaning, almost at the last draft stage I changed ‘flexible’ to ‘agile’, as this represented better what I was trying to get at. However, after I finished the book, I discovered that ‘agile learning design’ has a history emanating from software design, as can be seen by this diagram by Jennifer Bertram of Bottom Line Performance (2012):

© Jennifer Bertram, Bottom Line performance, 2012

© Jennifer Bertram, 2012

However, I felt that even the Bertram diagram was too ‘systematized’ to capture the ‘open-ness’ of the agile design process being used in ETEC 522 and other ‘lightweight’ design models in online learning. So from my perspective, the Bertram model is just one of many possible agile design approaches.

Designing for a VUCA world

© C. Adamson, 2012

© C. Adamson, 2012

In my book, I draw on Claire Adamson’s description of the kind of world in which our students now need to learn and live. In particular, teachers and instructors need to prepare students for a world that is:

  • volatile
  • uncertain
  • complex
  • ambiguous.

VUCA requires a strategy for coping with unavoidable changes and events that may arise. Agile learning design enables both instructors and learners to operate and teach and learn in such an environment.

When to use agile learning design?

The contexts in which there is a need for agile learning design could include the following:

  • areas where the subject matter is particularly dynamic, or where examples that illustrate more abstract contexts are frequently occurring. Subject areas that are about, or strongly influenced by, digital technologies, for instance, or political science, economics, or environmental studies, where examples and new thinking are constantly developing, need an agile design that enables changes in the subject area or the external environment to be quickly incorporated into the teaching and learning;
  • where the course or program has very diverse students with very different needs. Agile learning design allows the instructor to take into account the various needs of students and to design the course or program accordingly. Since the students and their diversity are likely to be different on each offering of the course, the design needs to change from offering to offering;
  • where appropriate teaching and learning tools are under constant change and development. For instance, any course that uses social media to enable student networking will need to integrate new tools and applications as they develop;
  • where the main goal is to enable students to develop appropriate skills to cope with a VUCA world, in whatever field they may be studying. This will mean presenting constantly changing and challenging course content, methods and tools, but within a framework that enables students to develop the skills needed to cope with such an environment.

It can be seen then that agile learning design has great potential for developing the knowledge and skills that students will need in a digital age.

Teaching economics? Would agile design enable you to include the Greek crisis? Image: Getty Pictures, 2015

Teaching economics? Would agile design enable you to include the Greek crisis as it develops? Image: Getty Pictures, 2015

Guidelines for agile design

Trying to set guidelines for agile learning design is a little like trying to establish rules for managing chaos. Nevertheless, successful agile designers need to be guided by a set of pedagogically sound principles, otherwise the course or program will quickly get out of hand, or students will feel lost and confused. Here are some suggestions, although there are many other possible guidelines that will need to be identified through greater experience from using such designs:

  • clearly defined and measurable broad learning goals that are communicated to and understood by the learners; these are likely to focus on learners covering and understanding certain core content and developing specific skills and will usually be determined by the instructor in advance of the course;
  • sub-goals or topics, negotiated with learners – particularly important for very diverse students within a course;
  • core learning materials and tools chosen in advance by the instructor; learners will be responsible for discovering and analyzing other learning materials and will be free to incorporate or negotiate the use of other tools; for instance, the instructor may decide that everyone will use a common course ‘platform’ such as WordPress, and assessment will be through a single e-portfolio software, but students may also use other tools that can be linked to WordPress and/or their e-portfolios; these decisions may vary across different offerings of the course;
  • assessment based on pre-determined criteria linked to the broad learning outcomes set for the course; again there may be room for some negotiation of assessment criteria between instructor and learners;
  • vision: a clear idea of what the overall goals, methods, and assessment for the course will be, and an open, flexible approach to achieving these goals; this is probably the most important requirement from the instructor.

Some agile learning designers may find even these guidelines to be too restrictive.

Conditions for success

We need more research and evaluation on agile learning design to determine the conditions for success, but the following are likely to be critical:

  • skilled, confident instructors supported by instructional designers with a strong pedagogical background;
  • learners will need careful preparation and orientation to a style or method of teaching with which they will be unfamiliar; it will be particularly important to stress the development of key skills that will carry over into work and life after graduation;
  • there needs to be a wealth of appropriate and relevant high quality open learning resources and digital tools that students can access and use;
  • constant and on-going communication between instructors and students, feedback, and evaluation will all be necessary to enable the course and methods are adapted as appropriate;
  • there will need to be sufficient minimum structure and content to pass any institutional or professional course approval process; the focus should be though on broad learning goals, core materials, and clear assessment criteria, rather than on detailed content;
  • at this stage, it is difficult to see how an agile design could be scaled up to large numbers of learners for a single instructor, although a team teaching approach may both strengthen the teaching and enable larger numbers of students to participate successfully.

In conclusion

I find agile learning design to be one of the most exciting and potentially powerful means of developing the knowledge and skills needed in a digital age. Even among the limited number of participants in the ETUG webinar, there were at least two who were engaged in agile learning design. However, more experimentation, applications and evaluation are needed, and it is important that we do not converge too quickly on ‘best practices’ in this design method until it has been explored and applied more generally.

I would particularly appreciate hearing from anyone ‘out there’ who has been using agile learning design methods and what they believe are the conditions for success.

References

Adamson, C. (2012) Learning in a VUCA world, Online Educa Berlin News Portal, November 13

Bertram, J. (2013) Agile Learning Design for Beginners New Palestine IN: Bottom Line Performance

Rawsthorne, P. (2012) Agile Instructional Design St. John’s NF: Memorial University of Newfoundland: http://www.rawsthorne.org/bit/docs/RawsthorneAIDFinal.pdf

What can past history tell us about the Athabasca University ‘crisis’?

Listen with webReader
Any merger needs to resolve incompatible union collective agreements

Any merger needs to resolve incompatible union collective agreements

It’s not just the Greeks who are having problems financially, even though they are getting all the headlines. In earlier posts I commented on Athabasca University’s so-called impending ‘insolvency’, as the president put it. As with all crises, the actual ‘end’ is never certain until it happens, so perhaps there’s still time for the Alberta government and Athabasca University to learn from history.

Questions from Wayne Burnett

Wayne Burnett, one of readers of this post, has asked some pretty good questions about what we can learn from the past that might help Athabasca in its current struggles. I originally replied to his comment with another comment, but feel the discussion needs a post of its own.

Wayne asked:

I would be interested in your comments (or the observations of your readers) on:

  1. What makes AU unique, from a student perspective? That’s the best argument for increased government support. What is it that students get from AU that they cannot get from the online initiatives at bricks and mortar universities?
  2. What has been the experience as BCOU was merged into TRU? Did the student experience change? Were there cost savings?
  3.  I don’t see the Feds getting involved (as they would be asked to help out TRU, Télé-université, and maybe others) but is there a possibility of seeking an arrangement with Saskatchewan and Manitoba, given that there is already some co-operation in higher education in the Western provinces? Does the UKOU get funding from the Scotland and Northern Ireland governments?

Cheers, Wayne

My response

Great questions, Wayne. Fancy a job as President at AU?!

I’ll do my best to provide a personal answer to Wayne’s questions, but each one is probably better answered by others.

1. What makes AU unique, from a student perspective?

This is a question for the Board and senior administration at Athabasca and it’s negligent to the point of irresponsible that they have not come out with a vision statement that sets this out clearly for government and for their own staff.

It isn’t actually hard to do, either.

  1. The first answer is that AU provides open access, enabling those who do not have the necessary qualifications for conventional universities to attempt higher level studies.
  2. Alberta needs more trained and qualified workers and has been depending on immigrants from outside Alberta, who need opportunities for continuing and higher education but do not have the qualifications for entry to conventional universities and cannot study full-time.
  3. Alberta also has a large and fast growing aboriginal population that is under-educated and desperately needs alternative routes to post-secondary education.
  4. None of the conventional universities in Alberta offer full undergraduate degrees at a distance, and there are very few fully online post-graduate degrees from the other universities.

I could go on, but AU needs not only to state that these are its main target groups, because they are under-served by the conventional institutions, but also has a plan of action for meeting these needs, which would require some substantial changes to the current curriculum and program offerings, for instance.

2. What has been the experience as BCOU was merged into TRU?

Again, this is best answered by former BCOU students and possibly by the OL division at TRU, but here’s my two cents worth.

Initially, it was pretty disastrous for most BCOU students. The BC government had no plan for the 16,000 or so students enrolled in the BCOU through the Open Learning Agency when they closed the OLA in 2003. They tried to get BCIT to take it on (OLA’s campus/building was near to the BCIT campus), but because of the unique union agreements for part-time BCOU faculty/tutors, BCIT did not want to touch it, nor did SFU.

This resulted in a period of nearly seven years when these 16,000 students were in limbo, until eventually TRU was forced or decided to take on these students. Again, however, because of the union agreements for BCOU part-time staff, because TRU had recently been changed from a college to a university, and because the ‘open’ students received less grant from government than the on-campus students, many of the campus faculty and administration were hostile to, or reluctant to acknowledge the validity of, ‘open’ or distance learning.

As a result TRU has to this day maintained strict apartheid between the campus and the open parts of its operation. Although in recent years the atmosphere has improved considerably, and a new administration is now much more supportive of the OL division, 12 years on, enrolments in the TRU OL division are just getting back to where they were when BCOU was closed down.

Perhaps more importantly, like AU, the OL division has not had the funds or the institutional commitment to make the major changes in its teaching model needed as a result of developments in online learning. However, if there are any BCOU students reading this, please let us have your views on this.

3. Is there a possibility of seeking an arrangement with Saskatchewan and Manitoba?

Well, there is already a co-operative of Canadian universities called the Canadian Virtual University, which includes the University of Manitoba, and Thompson Rivers University (TRU) in British Columbia (as well, as, interestingly, Mount Royal University in Alberta). There is automatic transfer of credits between Alberta and BC post-secondary institutions already (I actually went to an announcement about this by the then BC Minister of Advanced Education when embarrassingly he referred to Athabasca University as BC’s new open university, much to the chagrin of the TRU delegation.) So there are already opportunities for economies of scale by sharing courses from other institutions. The issue is whether this has been fully exploited at Athabasca, by using courses from other institutions rather than providing a complete program from within AU. I’m not in a position to answer that question.

The issue though isn’t so much about Saskatchewan or Manitoba, since the overall numbers of potential AU students from either province is likely to be low, but Ontario. Currently Ontario students make up 40 per cent of AU’s enrolments. What’s not clear is how much this will change now that the Council of Ontario Universities has established its own ‘Ontario Online.’

Although this will result in more online courses available from Ontario universities, it does not necessarily guarantee fully online programs. Even more importantly, Ontario Online still requires students to meet the qualifications for entry to Ontario universities before students can take their online courses. However, don’t expect Ontario to give money to Alberta to support Ontario students who want access to Athabasca.

What the Federal government could do, though, is to offer student aid to lifelong learners without a degree wanting to take further online qualifications from recognised institutions anywhere in Canada , which would then enable these Ontario students to be supported at Athabasca, as could students from all over Canada. Since there’s an election coming and none of the parties has stated its higher education policy yet……

4. Does the UKOU get funding from the Scotland and Northern Ireland governments [as well as from the government of England and Wales]? 

Sorry, I don’t know the answer to this question. Can anyone from the UKOU help with this? (My cynical answer would be that it’s equal treatment from all three governments: no funding at all, these days.)

What lessons can be drawn?

Here’s what I take away from this situation, although I’m sure readers will draw other conclusions:

1. No unique/non-conventional institution can survive without:

  • being clear about what makes it unique, and continuously identifying its uniqueness in changing circumstances;
  • having a clear strategy and plans to meet that unique mandate;
  • being nimble enough to adapt rapidly to changing external factors, without losing its unique advantages.

2. Closing or even merging a unique institution will usually leave a large gap in educational provision, and students enrolled in such a unique institution will suffer as a result of such closures or mergers, no matter how much a government may wriggle to mitigate such effects. Any re-organisation or merger must resolve incompatible union agreements to stand a chance of future success.

3. Although I didn’t discuss this explicitly with regards to the closure of the OLA, good leadership of unique institutions is even more important than for conventional institutions; it is essential that the leadership of such institutions wins and maintains the trust and confidence of government, and that requires constant attention and communication of the unique role and value of the institution. Once that trust is lost, it is almost impossible to regain, especially if its uniqueness is fading or under challenge.

4. Open and distance learning transcend provincial, state or even national boundaries. It is counter-productive to try to limit open and distance education to just state or provincial boundaries. Government and institutions need to develop business strategies that support and enable cross-state and cross-provincial activities in open and distance learning, for instance, through:

  • two-tier fee systems,
  • collaborative programming such as the CVU,
  • self-financing through tuition fees.

4. Nevertheless, in a provincial post-secondary education system such as Canada’s, it is in reality impossible to get financial support from other provincial governments for residents taking courses from an institution in another province. However, Federal policies regarding student financial aid could help institutions with a student enrolment footprint larger than their province. The Federal government should have a strategy for supporting lifelong learning, for economic reasons alone, and Federal student financial aid should support such a cross-provincial strategy.

So, Wayne, yes, there are lessons to be learned from the past here, but it would be extraordinary in Canadian higher education if these lessons ever get applied to rational decision-making.

Over to you

I’d love to hear from BCOU students, AU students, or open learning faculty/tutors at TRU about this:

  • What would you recommend to the Alberta government and/or Athabasca University, from your experience?
  • Most of all, what advice would you give to current or potential AU students?

 

Independent reviews of Teaching in a Digital Age now published

Listen with webReader
Image: computer lab at SUNT Purchase campus, © Wikipedia

Image: computer lab at SUNY Purchase campus, © Wikipedia

I have now received the three independent reviews I requested for my open, online textbook for faculty and instructors, called ‘Teaching in a Digital Age‘.

These are now published, alongside and as part of the book, as Appendix 4.

The process used to obtain the reviews can be seen here: The independent review process.

A review from a faculty perspective by Professor James Mitchell, of Drexel University, can be seen here.

A review from an open and distance education perspective, by Sir John Daniel, can be seen here.

A review from a digital learning perspective, by Leanora Zefi and the team at Digital Education Strategies, Ryerson University, can be seen here.

If you are doing or have done a review of Teaching in a Digital Age for an academic journal or other publication, I’d appreciate it if you could let me know, so I can link it to the book.

Obtaining independent reviews for an open textbook: what criteria to use?

Listen with webReader
Image: © Wikipedia Commons

Image: © Wikipedia Commons

What is the issue?

One of the questions I had to ask myself as a self-publishing author of Teaching in a Digital Age was whether I needed my book to be independently reviewed before publication. If so, would the same criteria need to be used as if I was publishing commercially?

What is the usual process in academic publishing?

Usually, before publishing an academic book or a textbook, commercial publishers will seek independent reviews at two stages of the process: when an author submits a proposal for a book, and then when the first complete draft is sent to the publisher. Of course, as well as external reviewers, the publishing company will have an in-house specialist editor who will be the main person in the decision-making process, and but even then an editor will usually take the final proposal to an internal committee or even a board meeting for final approval. Each of these stages can take up to three months, sometimes longer for the second stage, much longer if the author is required to make substantial changes before publication. Lastly, after the book is published, it may be reviewed, again independently, in academic journals specializing in the field.

Although this lengthy approval and review process can be very frustrating for an author, the process does ensure that the author gets a lot of feedback, and above all it is part of the quality control process, which is one reason why books count so much in the academic tenure and promotion process. However, the main purpose of external reviews in the publishing process is to ensure that there is a market for the book that is large enough to at least cover costs and hopefully generate a profit for the publisher, to provide quotes or endorsements that will help sell the book, and also to some extent to protect the publisher’s ‘brand’ or reliability.

There are also disadvantages of course with this process. It limits the acceptance of any publication that is outside the accepted norms within a discipline, thus perhaps inhibiting or delaying progress in a field, and, as happened to me once (with my first book), if a ‘rival’ academic with very different views is asked to be a reviewer, a perfectly good book can be unfairly trashed (although to be fair to the publisher, they still went ahead and published, on the basis of the other two reviews they received.)

In general, I have to say that with my 12 commercially published book, as an author I found the external review process, and above all, in two or three cases the feedback from the publisher’s editor, to be extremely valuable and helpful, and this review process resulted in far better books.

Open publishing

Self-published books need not follow any of this process, although open textbooks, such as those from OpenStax or the BCcampus open textbook project, are nearly always independently reviewed by faculty in the jurisdiction where these books may be adopted.

However, my book is somewhat different. It was written from scratch for a different market, faculty and instructors, rather than students, and it is not part of the BC government’s open textbook project that BCcampus manages. Although BCcampus offered me essential technical services, they were not responsible for editing or reviewing the book. (I was fortunate to be well enough known to BCcampus for them to put a lot of trust in me.)

What did I do?

Because the book was to be an open textbook, and I have a blog which is read within the community of practice in which I work, I was able to test early drafts of chapters and get some feedback on an ad hoc and voluntary basis. I also hired an instructional designer/editor to proof read and assess each draft chapter. I also sent drafts to other specialists in the field where I described in detail their work, asking for feedback and comments. I then published each chapter when I thought it was ready, and the Centre for Digital Education at Ryerson University also offered to provide systematic feedback as I published.

As a result I got a lot of useful feedback and comments that influenced the final version of the book, but nevertheless I was a bit shaken when I received an e-mail from a student who wanted to quote me in her graduate thesis, but was advised not to by her supervisor because the examiners might not accept references to a book that had not been independently reviewed.

As a result, after the book was published, and with no guarantee that it would be picked up and reviewed in an academic journal, I decided to obtain three independent reviews, and, as with the BCcampus textbooks, I would publish these reviews as received alongside the book.

Note though that I have obtained the external reviews after, not before, publication, because I felt it was more important to publish and be damned and thus get out the book as soon as possible, and because if there are major changes needed, that can still be done.

Criteria for choosing reviewers

I had three main criteria in mind: independence, qualification, and availability/willingness.

Independence was the most difficult. I had to invite someone who could be as objective as possible. This meant looking for reviewers who were professionals in the digital learning, instructional design, online learning or open education area, but who had not been closely associated with me during my 40 years working in the field. These reviewers should be people within the field who would be seen as being objective and sufficiently ‘distant’ from me and my career.

In terms of qualification, I needed reviewers who were also experts in the field of digital teaching and learning. This was the easiest of the three criteria to meet, but this had to be combined with independence, and this is where it started to get tricky.

Also, because the book is also targeted at faculty and instructors, I wanted a reviewer who is a mainline faculty member interested in teaching and learning but who did not know my previous work, and who would judge it strictly from a faculty or instructor perspective.

The third criterion, availability and willingness, was also important. The amount of work involved in reviewing a 500 page textbook is quite significant. Usually publishers pay a small fee for external reviewers, which no way compensates for the work involved, but at least it helps sweeten the pot. However, if I paid the reviewers as an author, that may be seen as unduly influencing the independence of the reviewer. In any case, I’m not getting revenues from the book, so any payment would have to come out of my own resources. As it turned out, none of the reviewers I approached requested or even mentioned a fee. Nevertheless I realised I was asking a lot of the reviewers with very little to offer them in return (other than a free read).

My choice of reviewers

The mainstream faculty member turned out to be the easiest of my choices. I published each chapter when it was ready, and after publishing the first chapter I received a string of comments from Dr. James Mitchell, Professor and Director of the Architectural & Environmental Engineering Program at Drexel University, Pennsylvania. I had never been in touch with him before and had never visited Drexel, but it was clear he was interested in changing the teaching in his department, and had some good points and questions to raise, so he was my first choice for reviewer.

Secondly, staff at the Centre for Digital Learning at Ryerson University, Toronto, had been tracking the development of the book and also providing feedback. This is a fairly new Centre and I did not know any of the staff there and had not done any consultancy work at Ryerson, so they seemed an obvious second choice to write a review of the whole book once it was finished.

Lastly, I wrote to a distinguished scholar of open learning at a British university, asking this person to be a reviewer, but they did not reply, so I was wondering who else to approach when I received an e-mail from Sir John Daniel, former Vice-Chancellor of the U.K. Open University, former President of the Commonwealth of Learning, and former President and Provost of several Canadian universities. He is also a scholar of open education with several (properly!) published books to his credit. He informed me that he was writing a review of the book for an academic journal and was looking forward to reading the book, and I therefore asked him if I may use his book review also as an external review of the book, which he agreed to. John Daniel is of course someone I have known and respected for many years, but we have never worked directly together.

So I now have my three reviewers, and I am extremely grateful for their willingness to do this.

Criteria for the review

BCcampus sends out a set of criteria to reviewers when they are reviewing books for the BCcampus open textbook project. In order to ensure consistency between the three reviews, I took the BCcampus guidelines and amended it to the slightly different context of my own book, and sent the guidelines to each reviewer. I will publish this as a separate blog post, as this post is already too long. 

Next steps

I now have two of the three reviews and the third is expected shortly. These will be published ‘as is’ with some context as an appendix to the book. If I also come across other reviews of the book from academic journals, I will add these (good or bad) to the book appendices.

Questions for my blog readers

The need for independent reviews for an open textbook has raised a lot of questions for me. If it had not been for the e-mail from the graduate student, frankly I don’t think I would have bothered. Since an open textbook is free and easily accessible, I was more than happy to let readers make their own judgements about the value of the book. It’s not as if you are asking someone to pay a large amount of money for something which they are then disappointed to read after they have paid their money. I also felt awkward about asking someone to read a 500 page book then write a critical review without being able to offer anything in return.

On the other hand, I want the book to be recognized and used by graduate students, and their committees and examiners. I want faculty in particular who read it to be assured that it has been properly peer reviewed. There is no reason why an open textbook cannot be as good if not better than any other book published more traditionally. But, as with distance education, online learning and open education, you have to be twice as good as the alternatives to be recognized. So if it takes external reviews to be accepted, so be it.

But I would really like to get your views on this. In particular:

1. Do you think it is important for open textbooks to be externally reviewed, before or after publication?

2. Is the process I have followed appropriate, or is it flawed? What would you have done or preferred?

3. Would you be happy to use an open textbook in your course if it had not been externally reviewed?

Next

I will share the guidelines I sent to the reviewers, and I will also ask for your feedback on these, so that the guidelines for review can be used by other authors of open textbooks.

I will also post on this blog each of the three external reviews when they are all in.