April 26, 2018

‘Making Digital Learning Work’: why faculty and program directors must change their approach

Completion rates for different modes of delivery at Houston Community College

Bailey, A. et al (2018) Making Digital Learning Work Boston MA:The Boston Consulting Group/Arizona State University

Getting blended learning wrong

I’ve been to several universities recently where faculty are beginning to develop blended or ‘hybrid’ courses which reduce but do not eliminate time on campus. I must confess I have mixed feelings about this. While I welcome such moves in principle, I have been alarmed by some of the approaches being taken.

The main strategy appears to be to move some of the face-to-face lectures online, without changing either the face-to-face or the online lecture format. In particular there is often a resistance to asynchronous approaches to online learning.  In one or two cases I have seen, faculty have insisted that students watch the Internet lectures live so that there can be synchronous online discussion, thus severely limiting the flexibility of ‘any time, any place’ for students.

Even more alarming, academic departments seem to be approaching the development of new blended learning programs the same way as their on-campus programs – identify faculty to teach the courses and then let them loose without any significant faculty development or learning design support. Even worse, there is no project management to ensure that courses are ready on time. Why discuss the design of the online lectures when you don’t do that for your classroom lectures? 

Trying to move classroom lectures online without adaptation is bound to fail, as we saw from the early days of fully online learning (and MOOCs). I recognise that blended or hybrid learning is different from fully online learning, but it is also different from face-to-face teaching. The challenge is to identify what the added value is of the face-to-face component, when most teaching can be done as well or better, and much more conveniently for students, online, and how to combine the two modes of delivery to deliver better learning outcomes more cost-effectively.  In particular, faculty are missing the opportunity to change their teaching method in order to get better learning outcomes, such as the development of high-level intellectual skills.

The real danger here is that poorly designed blended courses or programs will ‘fail’ and it is ‘blended learning’ that is blamed, when really it’s ignorance of best teaching practices on the part of faculty, and program directors especially. The problem is that faculty, and particularly senior faculty such as Deans and program directors, don’t know what they don’t know, which is why the report, ‘Making Digital Learning Work’ is so important. The report provides evidence that digital learning needs a complete change in culture and approaches to course and program development and delivery for most academic departments. Here’s why.

The report

The Arizona State University Foundation and Boston Consulting, funded by the Melinda and Bill Gates Foundation, conducted a study of the return on investment (ROI) of digital learning in six different institutions. The methodology focused on six case studies of institutions that have been pioneers in post-secondary digital education:

  • Arizona State University
  • University of Central Florida
  • Georgia State University
  • Houston Community College
  • The Kentucky Community and Technical College System
  • Rio Salado Community College.

These are all large institutions (over 30,000 students each) and relatively early adopters of online learning. 

The study had three aims:

  • define what ROI means in terms of digital education, and identify appropriate metrics for measuring ROI
  • assess the impact of digital learning formats on institutions’ enrolments, student learning outcomes, and cost structures
  • examine how these institutions implemented digital learning, and identify lessons and promising practices for the field.

The study compared results from three different modes of delivery:

  • face-to-face courses
  • mixed-modality courses, offering a mix of online and face-to-face components, with the online component typically replacing some tradition face-to-face teaching (what I would call ‘hybrid learning)
  • fully online courses.

The ROI framework

The study identified three components of ROI for digital learning:

  • impact on student access to higher education
  • impact on learning and completion outcomes
  • impact on economics (the costs of teaching, administration and infrastructure, and the cost to students).

The report is particularly valuable in the way it has addressed the economic issues. Several factors were involved:

  • differences in class size between face-to-face and digital teaching and learning
  • differences in the mix of instructors (tenured and adjunct, full-time and part-time)
  • allocation of additional expenses such as faculty development and learning design support
  • impact of digital learning on classroom and other physical capacity 
  • IT costs specifically associated with digital learning.

The report summarised this framework in the following graphic:

While there are some limitations which I will discuss later, this is a sophisticated approach to looking at the return on investment in digital learning and gives me a great deal of confidence in the findings.

Results

Evidence from the six case studies resulted in the following findings, comparing digital learning with face-to-face teaching.

Digital learning resulted in:

  • equivalent or improved student learning outcomes
  • faster time to degree completion
  • improved access, particularly for disadvantaged students
  • a better return on investment (at four of the institutions): savings for online courses ranged from $12 to $66 per credit hour.

If you have problems believing or accepting these results then I recommend you read the report in full. I think you will find the results justified.

Conditions for success

This is perhaps the most valuable part of the report, because although most faculty may not be aware of this, those of us working in online learning have been aware for some time of the benefits of digital learning identified above. What this report makes clear though are the conditions that are needed for digital learning to succeed:

  • take a strategic portfolio approach to digital learning. This needs a bit of unpacking because of the terminology. The report argues that the greatest potential to improve access and outcomes while reducing costs lies in increasing the integration of digital learning into the undergraduate experience through mixed-modality (i.e. hybrid learning). This involves not just one single approach to course design but a mix, dependent on the demands of the subject and the needs of students. However, there should be somewhat standard course design templates to ensure efficiency in course design and to reduce risk.
  • build the necessary capabilities and expertise to design for quality in the digital realm. The experience of the six institutions emphasises that significant investment needs to be made in instructional design, learning sciences and digital tools and capacity (and – my sidebar – faculty need to listen to what instructional designers tell them)
  • provide adequate student support that takes account of the fact that students will often require that support away from the campus (and 24/7)
  • fully engage faculty and provide adequate faculty development and training by fostering a culture of innovation in teaching
  • tap outside vendors strategically: determine the strategic goals first for digital learning then decide where outside vendors can add value to in-house capacity
  • strengthen analytics and monitoring: the technology provides better ways to track student progress and difficulties

My comments on the report

This report should be essential reading for anyone concerned with teaching and learning in post-secondary education, but it will be particularly important for program directors. 

It emphasises that blended learning is not so much about delivery but about achieving better learning outcomes and increased access through the re-design of teaching that incorporates the best of face-to-face and online teaching. However this requires a major cultural change in the way faculty and instructors approach teaching as indicated by the following:

  • holistic program planning involving all instructors, instructional designers and probably students as well
  • careful advanced planning, and following best practices, including project management and learning design
  • focusing as much on the development of skills as delivering content
  • identifying the unique ‘affordances’ of face-to-face teaching and online learning: there is no general formula for this but it will require discussion and input from both content experts and learning designers on a course by course basis
  • systematic evaluation and monitoring of hybrid learning course designs, so best (and poor) practices can be identified

I have a few reservations about the report:

  • The case study institutions were carefully selected. They are institutions with a long history of and/or considerable experience in online learning. I would like to see more cases built on more traditional universities or colleges that have been able successfully to move into online and especially blended learning
  • the report did not really deal with the unique context of mixed-modularity. Many of the results were swamped by the much more established fully online courses. However, hybrid learning is still new so this presents a challenge in comparing results.

However, these are minor quibbles. Please print out the report and leave it on the desk of your Dean, the Provost, the AVP Teaching and Learning and your program director – after you’ve read it. You could also give them:

Bates, A. and Sangra, A. (2011) Managing Technology in Higher Education San Francisco: Jossey-Bass/John Wiley

But that may be too much reading for the poor souls, who now have a major crisis to deal with.

What I would like to see in online learning in 2018: 1: a theory of classroom affordances

Prediction is difficult, especially about the future, so I won’t waste your time in suggesting what technologies are likely to take off in 2018. Instead, I’d rather focus on what I would like to see happen in 2018.

A research-based theory of classroom affordances

a. The challenge

With more and more teaching and learning occurring online, every instructor is now faced with the question: what is best done face-to-face and what is best done online? From a student’s point of view, what can the institution offer educationally on campus that they cannot get online? I am suggesting that we do not yet have a sufficiently powerful research-based theory that can realistically answer these questions.

b. What we know

Those of us working in online learning are well aware of the assumption made by many instructors that the classroom experience is inherently superior to any form of online learning. We are also aware of how often this assumption has proved wrong, with for instance student-student and student-instructor interactions online often being just as or more effective than in classrooms.

With the development of video, simulations, games-based learning and remote labs, even forms of experiential learning such as scientific and engineering experiments, manual operations and familiarity with tools can be developed as effectively online as in labs, workshops or classrooms. 

However, the differences between the effectiveness of online learning and face-to-face learning usually are dependent as much on the context or the circumstances of learning as on inherent qualities of what is to be taught or the medium of teaching. It is clear there are some circumstances where we now know online learning is preferable to face-to-face teaching (e.g. where learners have difficulty accessing physical classrooms, either because they are working or because it means a two hour commute) and where face-to-face teaching is more practical than online learning (e.g. where students need to handle and use heavy equipment). 

c. The need for a theory – and research questions

Nevertheless, there are other circumstances where either it doesn’t matter in terms of learning effectiveness whether it is done face-to-face or online, or where indeed there are significant differences in certain circumstances, but we don’t yet know what these are because we have not tested or challenged them.

So we need research-based evidence that can answer the following research question:

Under what conditions and for what purposes is it better to learn in a face-to-face context rather than online? And when and how should they be used to complement each other when both are readily available?

Can we produce a theory from such evidence that would enable a set of rules or criteria that instructors could use to make such a decision? What research would be needed to develop or test such a theory?

d. Is there no current theory we could use or build on?

There are plenty of theories of how learning best takes place¹, plenty of theories that are used to support best practices in face-to-face teaching², and similarly a few theories that suggest best practices in online learning and teaching³. What we don’t have is theory about the differences (if any) between face-to-face and online learning in specific circumstances or conditions, backed by reliable research evidence, when both are available in practice.

One potentially promising line of enquiry could be built around the research on the pedagogical affordances of different media: what kinds of learning can specific media support or help develop? If we treat face-to-face teaching as a medium, what are its pedagogical affordances: what can it do better than other media? (see Norman, 1988 and Chapter 7 of Teaching in a Digital Age)

However, the issue in deciding what to do online or face-to-face is usually not only pedagogical but as much to do with cost, instructor convenience, and a lack of imagination of how things could be done differently. Also the context is critically important. An effective theory will need to incorporate all these factors.

Note that most research on differences between online learning and face-to-face teaching at a meta level results in no significant differences overall. The factors or conditions that lead to differences often cancel each other out and are ‘controlled’ or eliminated from the studies to ensure ‘comparability.’ Thus – surprise, surprise – good quality online learning could be better than poor quality face-to-face teaching, and vice versa. Thus the conditions in which each is used is essential for evaluating their effectiveness. Furthermore these meta studies are looking at replacing face-to-face teaching with online learning or more recently blended learning, not at what the unique teaching characteristics of each mode may be, and in what conditions.

However it is precisely these ‘conditions’ that we should be researching to answer the research questions outlined above. When does online learning work better than face-to-face teaching and vice-versa? In other words, do not assume that it does not matter whether we teach online or face-to-face because the research shows no statistical differences, but instead let’s focus on identifying those specific conditions that actually do lead to significant differences, especially when both are equally available to instructors and students.

e. What about the SECTIONS model?

The SECTIONS model I have proposed in my open, online textbook, Teaching in a Digital Age, provides a set of questions that instructors should ask before finalising decisions on the choice of a particular medium or technology for teaching, partly based on their pedagogical affordances (T for Teaching and I for Interaction) but also on other factors such as student access, costs, and security. If we think of face-to-face teaching as just another teaching medium, could not the SECTIONS model be applied to answering the research questions in 1. c above? 

This could be one starting point perhaps for such a theory, but it will need much more research to test and validate it. In Chapter 7, I looked at all media except face-to-face teaching, because I was unaware of relevant research that could identify the unique features of face-to-face teaching when online learning could also be used.

Furthermore, face-to-face teaching is not monolithic, but can vary enormously – as can other media – and also can incorporate other media, so probably more research is needed to establish the conditions where face-to-face teaching is superior. 

f. What about Teaching in a Digital Age?

If you have read my online open textbook, you might think that this provides a theoretical basis for choosing between face-to-face and online learning. Certainly it does discuss a number of different educational theories and looks at several different teaching methods. It also suggests guidelines based on research and best practices for choosing between different modes of delivery and different media (except face-to-face teaching as a medium).

But the book is not written as a particular theory of teaching and does not provide enough theory to identify what to do regarding the ‘either online or face-to-face when I can use both’ decision within a specific teaching context. It is more a set of guidelines derived from existing theory and best practice. Someone else needs to move this work further.

g. Next steps

  1. Acknowledge and have recognized the significance of the research questions. This is an extremely important issue for research in education. We know from the National Survey of Online and Distance Learning in Canadian Post-secondary Education that the move to blended and hybrid learning is growing rapidly. Every instructor will soon face the question of what should be done in class and what online, but we have few answers at the moment that go beyond beliefs or prejudice;
  2. build these research questions into doctoral programs in education, so we have a growing body of evidence on the research questions and students and supervisors thinking about the issue and developing hypotheses and research evidence to support them;
  3. develop a national program of research into this issue so that there is a significant mass of study and research that will likely lead to some practical and useful answers in different subject domains.

I should make it clear I have no intention or wish to lead this research because I am trying to reduce my work commitments as I grow older. It is my privilege to pose such questions but not my responsibility to answer them! I just hope though someone else will pick up the gauntlet I have thrown down.

Over to you

This is meant as a ‘thought piece’ to stimulate thinking around a particular issue that I think is important. However, you may have different views on this that I hope you will share, in particular:

  1. Is this really an important issue? Do we really need research on this? Why not let instructors experiment and find out what works best for them without the need for any formal research?
  2. Is the question: ‘What should be done online and what face-to-face under what conditions?’ a question suitable for research? Are there other, better questions that should be asked?
  3. What existing theories could help with this question? Do we need yet another theory – or just a few more hypotheses that can be tested within existing theoretical frameworks? If so which one(s)?

Footnotes

¹ See, for instance, Chapter 2, Teaching in a Digital Age

²See for instance, Chapter 3, Teaching in a Digital Age

³ See for instance Chapter 4, Teaching in a Digital Age

References

Norman, Donald (1988). The Design of Everyday Things. New York: Basic BooksISBN 978-0-465-06710-7.

How to keep up with new technology in online learning

I was recently asked if I would answer a couple of questions from students in Royal Roads University’s course ‘Leveraging Technology in Higher Education’ in their MA in Higher Education Administration and Leadership.

With the permission of their instructor, Irwin de Vries, and the students, I am sharing my response to the two questions they raised.

Question 1

1. How can an institution make sense of all the new developments, such as what the NMC highlights every year, and incorporate that successfully into their institutional planning?

What a good question! It’s a question I personally struggle with. One could spend every waking moment these days trying to keep up with the latest apps, devices, and waves of tech innovation. Indeed, the fear of not being able to do this forced me into premature retirement – how could I keep up with everything and still play golf whenever I wanted?  (Golf won.)

However, it turns out that while the technology is forever changing, there are a number of ‘coping’ strategies, based on more fundamental principles or theories that do not change so rapidly.

Prediction is very difficult, especially if it’s about the future

First, the New Media Consortium has a poor track record in accurately predicting technology trends in higher education, mainly in terms of timelines (far too optimistic regarding the application of a particularly technology) but also often in terms of whether a technology in fact turns out to be useful for teaching or learning. Go back for instance to their 2008 report: grassroots video (iTunes, Möbius, etc.), the collaborative web (Google Docs), data mashups, within one or two years, etc.

These tools are often very useful outside of the teaching/learning process but don’t necessarily adapt easily for teaching and learning. More often these tools might have been useful but were not used or were ignored by instructors because they did not meet the immediate needs of the instructors (or the perceived needs of the students).

Institutional vs individual choice

Second, institutional decision-making is based mainly around IT network technology, classroom equipment, and ‘universally used’ commercially licensed software or technology such as word processing (which is one reason why LMSs and webinar technology are so heavily used – the institution pays for and maintains them), but emerging technologies these days are more end-user focused and low cost, so technologies are now being adopted and decided by individual instructors and especially by students, rather than the institution. These low cost technologies tend to be based on mobile phones or tablets and free or low-cost apps. It is only when a technology really takes off in teaching does it make sense for the institution to ‘block buy’ a license if it is a commercial product.

More importantly, it doesn’t make sense for institutions to make institution-wide decisions for most teaching technologies, partly because of wide variations in subject discipline needs, but mainly because with constantly emerging technologies, it’s better for the grassroots instructors and students to adopt as appropriate, hence ensuring more innovation in teaching and learning.

For instructors, usually technology adoption enables them to solve a teaching problem, such as not enough interaction with students, students not attending in bad weather or with long commutes, difficult concepts to teach abstractly, etc. Since the teaching problems often vary from instructor to instructor, it is best to leave such decisions to them. However, instructors can be ‘nudged’ by instructional designers/learning technology support staff, who should be constantly looking for potential new applications of technology, and for faculty who may be interested in trying them.

Lastly, for instructors or instructional designers who want to make a fully considered decision about the best choice or mix of educational media, there is my own SECTIONS model which attempts to identify key factors that should influence choice of media. However, even if this approach is used, in most cases it will be influenced by an instructor’s gut feeling or intuition about what will work best within a particular context -which is more likely to be right than wrong.

The SECTIONS model

The one exception I would make to the decentralization approach to technology selection is where an institution has a strong strategy or plan for digital learning. In this case, part of the strategy might be to combine the choice of a technology (such as tablets) with a plan for faculty development in how to use the technology, based on a clear pedagogical approach. This allows a large step forward to be made. The Justice Institute of BC’s digital learning strategy for the University of Guadalajara in Mexico is a good example. This helps the majority of faculty with the adoption of technology in a consistent and high quality manner. In this case the strategy depended on a clear pedagogical basis for the choice of technology made at an institutional level. However, this is the exception rather than the rule.

Conclusion

As with most educational decisions, context is all important. Instructors and to some extent students are closest to the action and hence are usually in a better position to make an appropriate choice than an institution trying to cover all possible positions.

However, there are guidelines that can be adopted to avoid being swayed by the media hype over the latest technology. Does it solve a problem I’m having? Will it help students to develop the knowledge and skills they will need in the future? Is it easy to use? Is it cheap or free for students? Not rocket science, by any means, but it is surprising how little such obvious questions are asked, especially by the media, when a new technology appears on the horizon.

My response to the second question will follow.

DIY online courses on Facebook

Hughes, M. (2017) Facebook is letting Groups create online learning courses – what could possibly go wrong?  The Next Web, 1 June

It has been a depressing week for me, with terrorist bombings, Trump’s environmental blunder, trade wars and political uncertainty in British Columbia. We are currently without an effective government but that seems not to be a problem for anyone.

However, I did have one good laugh this week that I want to share with you. This came from Matthew Hughes blog (Matthew is at the University of Roehampton, in the UK). He is reporting on news (fake news?) that Facebook is enabling anyone running a Facebook Group to create their own online courses. As he says, tongue in cheek: what could possibly go wrong from an organization with such high standards of credibility?

  • Well, we might begin by looking at the problems that collaborative MOOCs have been facing, especially moderating discussions. Who and how will that moderation be done?
  • Then there’s the likely make-up of the participants  – will this just be more opinion swapping between people with the same world view? 
  • Who will accredit or validate the knowledge in such courses? The learners themselves, probably – and look again at the problems with that in many MOOCs.
  • how will the data that Facebook collects through such Groups be used? You will never know.

This is not to argue that good quality online courses will be impossible through Facebook Group – but once the necessary quality standards are applied, how will it then look different from any other online course delivered on other platforms? What’s the added value?

My point here is that technology alone is never a solution to an educational issue. Do the benefits of the technology (the added value) outweigh the negatives – in this case Big Brother watching you?

One last point while I’m on about Facebook. A Facebook account is a true zombie. You cannot kill it. I ‘signed out’ of Facebook within six months of joining it in 2007, for privacy reasons primarily, and have never used it since, but I still get e-mail messages from them asking me to look at someone I know who has just posted something on Facebook, and I keep getting messages from Facebook to log back in. GO AWAY, FACEBOOK!!!! You are not wanted here.

Webinar on choosing modes of delivery and the role of face-to-face teaching in an online world

Why get the bus to campus when you can study online?

Why get the bus to campus when you can study online?

On Tuesday I gave another in the Contact North series of webinars designed around my open, online textbook for faculty and instructors, Teaching in a Digital Age.

This focused on Chapter 9 of the book, but with a different twist from last year’s webinar on the same topic, this year’s webinar focused particularly on the move to blended learning, and the need to redefine the role of campus-based teaching when so much can now be done online.

You can download a recording of the webinar from here: https://contactnorth.webex.com/contactnorth/lsr.php?RCID=760bef531b9a8fcf59f5480dd57401ff. However, make sure you have the WebEx ARF player downloaded in order to play the recording – see the download instructions on the above web page if the ‘play’ button doesn’t load the recording.

Also note that the presentation doesn’t start until two minutes into the recording because the introduction was accidentally muted.