August 14, 2018

Learning analytics, student satisfaction, and student performance at the UK Open University

There is very little correlation between student satisfaction and student performance. Image: Bart Rienties. Click on image to see the video.

Rienties, B. and Toetenel, L. (2016) The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules, Computers in Human Behaviour, Vol. 60, pp.333-341

Li, N. et al. (2017) Online learning experiences of new versus continuing learners: a large-scale replication study, Assessment and Evaluation in Higher Education, Vol. 42, No. 4, pp.657-672

It’s never too late to learn

It’s been a hectic month with two trips from Vancouver to Ontario and back and one to the UK and back, a total of four keynotes, two panel sessions and two one day consultancies. By the time I got to the end of the month’s travels, I had learned so much that at a conference in Toronto I had to go to my room and lie down  – I just couldn’t take any more!

At my age, it takes time to process all this new information, but I will try to summarise the main points of what I learned in the next three posts.

Learning analytics at the Open University

The Open University, with over 100,000 students and more than 1,000 courses (modules), and most of its teaching online in one form or another, is an ideal context for the application of learning analytics. Fortunately the OU has some of the world leaders in this field. 

At the conference on STEM teaching at the Open University that I attended as the opening keynote, the closing keynote was given by Bart Rienties, Professor of Learning Analytics at the Institute of Educational Technology at the UK Open University. Rienties and his team linked 151 modules (courses) and 111,256 students with students’ behaviour, satisfaction and performance at the Open University UK, using multiple regression models. 

His whole presentation (40 minutes, including questions) can be accessed online, and is well worth viewing, as it provides a clear summary of the results published in the two detailed papers listed above. As always, if you find my summary of results below of interest or challenging, I strongly recommend you view Bart’s video first, then read the two articles in more detail. Here’s what I took away.

There is little correlation between student course evaluations and student performance

This result is a bit of a zinger. The core dependent variable used was academic retention (the number of learners who completed and passed the module relative to the number of learners who registered for each module). As Rientes and Toetenel (p.340) comment, almost as an aside, 

it is remarkable that learner satisfaction and academic retention were not even mildly related to each other….Our findings seem to indicate that students may not always be the best judge of their own learning experience and what helps them in achieving the best outcome.’

The design of the course matters

One of the big challenges in online and blended learning is getting subject matter experts to recognise the importance of what the Open University calls ‘learning design.’ 

Conole (2012, p121) describes learning design as:

a methodology for enabling teachers/designers to make more informed decisions in how they go about designing learning activities and interventions, which is pedagogically informed and makes effective use of appropriate resources and technologies. LD is focussed on ‘what students do’ as part of their learning, rather than the ‘teaching’ which is focussed on the content that will be delivered.

Thus learning design is more than just instructional design.

However, Rienties at al. comment that ‘only a few studies have investigated how educators in practice are actually planning and designing their courses and whether this is then implemented as intended in the design phase.’ 

The OU has done a good job in breaking down some of the elements of learning design. The OU has mapped the elements of learning design in nearly 200 different courses. The elements of this mapping can be seen below (Rientes and Toetenal, 2016, p.335):

Rientes and Toetenel then analysed the correlations between each of these learning design elements against both learner satisfaction and learner performance. What they found is that what OU students liked did not match with learner performance. For instance, students were most satisfied with ‘assimilative’ activities, which are primarily content focused, and disliked communication activities, which are primarily social activities. However, better student retention was most strongly associated with communication activities, and overall, with the quality of the learning design.

Rientes and Toetenel conclude:

although more than 80% of learners were satisfied with their learning experience, learning does not always need to be a nice, pleasant experience. Learning can be hard and difficult at times, and making mistakes, persistence, receiving good feedback and support are important factors for continued learning….

An exclusive focus on learner satisfaction might distract institutions from understanding the impact of LD on learning experiences and academic retention. If our findings are replicated in other contexts, a crucial debate with academics, students and managers needs to develop whether universities should focus on happy students and customers, or whether universities should design learning activities that stretch learners to their maximum abilities and ensuring that they eventually pass the module. Where possible, appropriate communication tasks that align with the learning objectives of the course may seem to be a way forward to enhance academic retention.

Be careful what you measure

As Rientes and Toetenel put it:

Simple LA metrics (e.g., number of clicks, number of downloads) may actually hamper the advancement of LA research. For example, using a longitudinal data analysis of over 120 variables from three different VLE/LMS systems and a range of motivational, emotions and learning styles indicators, Tempelaar et al. (2015) found that most of the 40 proxies of simple” VLE LA metrics provided limited insights into the complexity of learning dynamics over time. On average, these clicking behaviour proxies were only able to explain around 10% of variation in academic performance.

In contrast, learning motivations, emotions (attitudes), and learners’ activities during continuous assessments (behaviour) significantly improved explained variance (up to 50%) and could provide an opportunity for teachers to help at-risk learners at a relatively early stage of their university studies.

My conclusions

Student feedback on the quality of a course is really important but it is more useful as a conversation between students and instructors/designers than as a quantitative ranking of the quality of a course.  In fact using learner satisfaction as a way to rank teaching is highly misleading. Learner satisfaction encompasses a very wide range of factors as well as the teaching of a particular course. It is possible to imagine a highly effective course where teaching in a transmissive or assimilative manner is minimal, but student activities are wide, varied and relevant to the development of significant learning outcomes. Students, at least initially, may not like this because this may be a new experience for them, and because they must take more responsibility for their learning. Thus good communication and explanation of why particular approaches to teaching have been chosen is essential (see my comment to a question on the video).

Perhaps though the biggest limitation of student satisfaction for assessing the quality of the teaching is the often very low response rates from students, limited evaluation questions due to standardization (the same questions irrespective of the nature of the course), and the poor quality of the student responses. This is no way to assess the quality of an individual teacher or a whole institution, yet far too many institutions and governments are building this into their evaluation of teachers/instructors and institutions.

I have been fairly skeptical of learning analytics up to now, because of the tendency to focus more on what is easily measurable (simple metrics) than on what students actually do qualitatively when they are learning. The focus on learning design variables in these studies is refreshing and important but so will be analysis of student learning habits.

Finally, this research provides quantitative evidence of the importance of learning design in online and distance teaching. Good design leads to better learning outcomes. Why then are we not applying this knowledge to the design of all university and college courses, and not just online courses? We need a shift in the power balance between university and college subject experts and learning designers resulting in the latter being treated as at least equals in the teaching process.

References

Conole, G. (2012). Designing for learning in an open world. Dordrecht: Springer

Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: learning analytics in a data-rich context. Computers in Human Behavior, 47, 157e167. http://dx.doi.org/10.1016/j.chb.2014.05.038.

 

‘Making Digital Learning Work’: why faculty and program directors must change their approach

Completion rates for different modes of delivery at Houston Community College

Bailey, A. et al (2018) Making Digital Learning Work Boston MA:The Boston Consulting Group/Arizona State University

Getting blended learning wrong

I’ve been to several universities recently where faculty are beginning to develop blended or ‘hybrid’ courses which reduce but do not eliminate time on campus. I must confess I have mixed feelings about this. While I welcome such moves in principle, I have been alarmed by some of the approaches being taken.

The main strategy appears to be to move some of the face-to-face lectures online, without changing either the face-to-face or the online lecture format. In particular there is often a resistance to asynchronous approaches to online learning.  In one or two cases I have seen, faculty have insisted that students watch the Internet lectures live so that there can be synchronous online discussion, thus severely limiting the flexibility of ‘any time, any place’ for students.

Even more alarming, academic departments seem to be approaching the development of new blended learning programs the same way as their on-campus programs – identify faculty to teach the courses and then let them loose without any significant faculty development or learning design support. Even worse, there is no project management to ensure that courses are ready on time. Why discuss the design of the online lectures when you don’t do that for your classroom lectures? 

Trying to move classroom lectures online without adaptation is bound to fail, as we saw from the early days of fully online learning (and MOOCs). I recognise that blended or hybrid learning is different from fully online learning, but it is also different from face-to-face teaching. The challenge is to identify what the added value is of the face-to-face component, when most teaching can be done as well or better, and much more conveniently for students, online, and how to combine the two modes of delivery to deliver better learning outcomes more cost-effectively.  In particular, faculty are missing the opportunity to change their teaching method in order to get better learning outcomes, such as the development of high-level intellectual skills.

The real danger here is that poorly designed blended courses or programs will ‘fail’ and it is ‘blended learning’ that is blamed, when really it’s ignorance of best teaching practices on the part of faculty, and program directors especially. The problem is that faculty, and particularly senior faculty such as Deans and program directors, don’t know what they don’t know, which is why the report, ‘Making Digital Learning Work’ is so important. The report provides evidence that digital learning needs a complete change in culture and approaches to course and program development and delivery for most academic departments. Here’s why.

The report

The Arizona State University Foundation and Boston Consulting, funded by the Melinda and Bill Gates Foundation, conducted a study of the return on investment (ROI) of digital learning in six different institutions. The methodology focused on six case studies of institutions that have been pioneers in post-secondary digital education:

  • Arizona State University
  • University of Central Florida
  • Georgia State University
  • Houston Community College
  • The Kentucky Community and Technical College System
  • Rio Salado Community College.

These are all large institutions (over 30,000 students each) and relatively early adopters of online learning. 

The study had three aims:

  • define what ROI means in terms of digital education, and identify appropriate metrics for measuring ROI
  • assess the impact of digital learning formats on institutions’ enrolments, student learning outcomes, and cost structures
  • examine how these institutions implemented digital learning, and identify lessons and promising practices for the field.

The study compared results from three different modes of delivery:

  • face-to-face courses
  • mixed-modality courses, offering a mix of online and face-to-face components, with the online component typically replacing some tradition face-to-face teaching (what I would call ‘hybrid learning)
  • fully online courses.

The ROI framework

The study identified three components of ROI for digital learning:

  • impact on student access to higher education
  • impact on learning and completion outcomes
  • impact on economics (the costs of teaching, administration and infrastructure, and the cost to students).

The report is particularly valuable in the way it has addressed the economic issues. Several factors were involved:

  • differences in class size between face-to-face and digital teaching and learning
  • differences in the mix of instructors (tenured and adjunct, full-time and part-time)
  • allocation of additional expenses such as faculty development and learning design support
  • impact of digital learning on classroom and other physical capacity 
  • IT costs specifically associated with digital learning.

The report summarised this framework in the following graphic:

While there are some limitations which I will discuss later, this is a sophisticated approach to looking at the return on investment in digital learning and gives me a great deal of confidence in the findings.

Results

Evidence from the six case studies resulted in the following findings, comparing digital learning with face-to-face teaching.

Digital learning resulted in:

  • equivalent or improved student learning outcomes
  • faster time to degree completion
  • improved access, particularly for disadvantaged students
  • a better return on investment (at four of the institutions): savings for online courses ranged from $12 to $66 per credit hour.

If you have problems believing or accepting these results then I recommend you read the report in full. I think you will find the results justified.

Conditions for success

This is perhaps the most valuable part of the report, because although most faculty may not be aware of this, those of us working in online learning have been aware for some time of the benefits of digital learning identified above. What this report makes clear though are the conditions that are needed for digital learning to succeed:

  • take a strategic portfolio approach to digital learning. This needs a bit of unpacking because of the terminology. The report argues that the greatest potential to improve access and outcomes while reducing costs lies in increasing the integration of digital learning into the undergraduate experience through mixed-modality (i.e. hybrid learning). This involves not just one single approach to course design but a mix, dependent on the demands of the subject and the needs of students. However, there should be somewhat standard course design templates to ensure efficiency in course design and to reduce risk.
  • build the necessary capabilities and expertise to design for quality in the digital realm. The experience of the six institutions emphasises that significant investment needs to be made in instructional design, learning sciences and digital tools and capacity (and – my sidebar – faculty need to listen to what instructional designers tell them)
  • provide adequate student support that takes account of the fact that students will often require that support away from the campus (and 24/7)
  • fully engage faculty and provide adequate faculty development and training by fostering a culture of innovation in teaching
  • tap outside vendors strategically: determine the strategic goals first for digital learning then decide where outside vendors can add value to in-house capacity
  • strengthen analytics and monitoring: the technology provides better ways to track student progress and difficulties

My comments on the report

This report should be essential reading for anyone concerned with teaching and learning in post-secondary education, but it will be particularly important for program directors. 

It emphasises that blended learning is not so much about delivery but about achieving better learning outcomes and increased access through the re-design of teaching that incorporates the best of face-to-face and online teaching. However this requires a major cultural change in the way faculty and instructors approach teaching as indicated by the following:

  • holistic program planning involving all instructors, instructional designers and probably students as well
  • careful advanced planning, and following best practices, including project management and learning design
  • focusing as much on the development of skills as delivering content
  • identifying the unique ‘affordances’ of face-to-face teaching and online learning: there is no general formula for this but it will require discussion and input from both content experts and learning designers on a course by course basis
  • systematic evaluation and monitoring of hybrid learning course designs, so best (and poor) practices can be identified

I have a few reservations about the report:

  • The case study institutions were carefully selected. They are institutions with a long history of and/or considerable experience in online learning. I would like to see more cases built on more traditional universities or colleges that have been able successfully to move into online and especially blended learning
  • the report did not really deal with the unique context of mixed-modularity. Many of the results were swamped by the much more established fully online courses. However, hybrid learning is still new so this presents a challenge in comparing results.

However, these are minor quibbles. Please print out the report and leave it on the desk of your Dean, the Provost, the AVP Teaching and Learning and your program director – after you’ve read it. You could also give them:

Bates, A. and Sangra, A. (2011) Managing Technology in Higher Education San Francisco: Jossey-Bass/John Wiley

But that may be too much reading for the poor souls, who now have a major crisis to deal with.

Online learning for beginners: 8. Won’t online learning be more work?

Books lots! 2

This is the eighth in a series of a dozen blog posts aimed at those new to online learning or thinking of possibly doing it. The previous seven are:

More work?

The short answer is, yes, of course, at least in the short term. This is because online teaching is the same as any other skill. When you first start, you have to learn a lot, and do things you haven’t done before. For instance, as I discussed in earlier posts, you have to think carefully about why you are using online learning, talk to colleagues and work with other professionals such as instructional and web designers, master the technology, such as video recording or a learning management system, and basically re-think and re-design your teaching. This will take time, and your first online course will undoubtedly be more work and more challenging than your most recent face-to-face course.

However, in the long run, there is no reason why online teaching should be more work than face-to-face teaching, all other things being equal (which, of course, they never are in teaching). As always, there are important conditions to be met, if you don’t want to be swamped with extra work. So let’s look at what these conditions are.

Re-design your teaching

In a previous post in this series, I warned against trying to move your face-to-face teaching online, by just recording lectures. Although this may seem to be a time saver when developing an online course, it can cause a lot more work down the line. There will always be some students who don’t understand parts of a lecture and if something isn’t clear all students may have the same problem. When this happens, watch the e-mails or phone calls or even tweets roll in from students – or watch the course completion rate take a dive.

The answer is to use what is called ‘learning design’ or course design: setting clear learning outcomes or objectives for the course, breaking down the learning into manageable chunks of time for the students, providing appropriate learning activities for online learners, for instance, online discussion forums, and ensuring that assessment and feedback is continuous throughout the course, all the time thinking of the context in which the online learner is working. The trick is to move much of the work of finding, analyzing and applying content, and development of skills such as independent thinking, critical thinking, and problem solving, from you to the students, but under your guidance.

For most instructors, this means spending a good deal of time preparing the course in advance of its actual delivery. This means having the weekly modules that students study ready well in advance of the opening of the course. Although there will always be the need for minor changes to content in subsequent years, the bulk of the design and development of the course is done in the first year of its offering.

Most instructors in fact find a time shift when they move to online teaching. The more time you put into the development of the course in the first year, the less time you find yourself spending on content delivery during the delivery of the course, because it is already there. Multiply this over several offerings of the course and the time shift can lead to either significant time savings for you, or, more likely, your spending your time better in working directly, if online, with students, such as monitoring and contributing to online discussion of the course content.

For this reason, many institutions now offer funding to enable you to ‘buy yourself out’ of a face-to-face class for one or two semesters in order to prepare your first online course. Once you have some experience in this more traditional form of online learning, you can move to more ‘agile’ designs later, but that is another matter altogether. The first time out, you and your students need a clear structure and framework for the course.

Also, it is at this stage of course development that working with other professionals such as an instructional designer and web designer is most valuable. They should be able to provide the necessary advice and above all a framework and timetable for your work in designing the course.

Managing class size

I mentioned earlier that online teaching should not be more work, all things being equal. However, sometimes the aim is to use online learning in order to handle large classes or take extra students. These pressures may be coming from the administration rather than from you – or alternatively you may be concerned about the quality of the teaching of large face-to-face classes when many are delivered not by you but by teaching assistants who have barely more content knowledge than those they are teaching and who in particular may not have good lecturing skills.

The general rule for the most appropriate numbers for an individual instructor to teach online is pretty much the same as for face-to-face teaching. Once the instructor:student ratio goes over 1:30, it becomes harder to individualize the teaching and the instructor’s work load increases, unless the course is focused mainly on quantitative or ‘objective’ outcomes that can be automatically assessed, through, for instance, computer marked assignments. It is generally the marking that leads to overload when classes get beyond 30 per instructor.

However, because with online learning the content is available at any time and any place for students, there is some scope for scaling up the teaching to handle larger numbers. In particular, if the teaching content on the course is well developed by a top quality professor or instructor, all students will receive the same quality of content instruction. This means then that learner support and student assessment (marking) can be supported by contract sessional instructors as class size increases.

The availability of funding for hiring additional sessional instructors will depend on the business model behind the online program. If you are merely moving students from an existing face-to-face course to an online course, then there will be no extra money from tuition fees. However, if the online program is attracting new students paying additional tuition fees, then the extra funding can be used to hire more sessional instructors. In many North American universities, the tuition fee once an online course is developed more than covers the cost of additional sessional instructors, even with ‘steps’ of 30 students (i.e. for every additional 30 students you hire another sessional). Much of this of course will depend on faculty agreements, but from your point of view, re-design of a large face-to-face course by moving it online can not only improve the quality but also enable you to manage your own workload better.

What I would advise against is the use of graduate students as teaching assistants for online courses. The re-design of online courses requires instructors who can go beyond the ‘recorded’ content of an online course and can push students in online discussion groups, for instance, to challenge ideas and go deeper than just the formal online content. This requires sessional instructors with a good understanding of content and good inter-personal teaching skills to handle the extra students as class size increases.

In summary then, managing your workload as online class size increases requires several conditions:

  • ‘core’ content of high quality that does not need to be changed a lot from year to year;
  • learning/course design that provides a strong structure for students so that it is clear what they need to do when studying;
  • professional instructional design and web/media design support;
  • flexibility to hire additional, well-qualified sessional instructors as class size increases.

In the end, this may mean moving to a team approach to teaching large online classes. In some cases, the senior instructor’s responsibility may not involve direct teaching at all, but being responsible for the curriculum/content, setting learning outcomes, designing assessments, and supervising the learning support and assignment marking provided by sessional instructors.

Shifting from content to skills development

What online learning can do is enable you, as an instructor or teacher, to move away from ‘sage-on-the-stage’, where you are responsible for choosing and delivering content, and assessing how well students have comprehended this content, to ‘guide-on-the-side’, where students find, analyze and apply content, and develop higher level skills through practice, often working with other students online, through discussion or project work, but always under your guidance, or under the guidance of a team of sessional instructors that you monitor.

Implications

  1. Such changes inevitably mean more work, and more challenges, initially, in moving to online learning, but the benefits in both the quality of what your students learn, and the quality of your own engagement with students, can be substantial.
  2. There are also strategies for managing your workload when teaching online, so that over time you can balance better your teaching, research and administrative responsibilities.
  3. But online learning is not something to be undertaken lightly. You need to do it professionally, or it will be both more work and very frustrating.

Follow-up

For more on the design of online courses, see:

Later posts in this series will go into more detail about providing support for online learners that will also enable you to manage better your workload.

Up next

How can I do online learning well?

Your turn

If you have comments, questions or plain disagree, please use the comment box below.

Get the students to do the work!

Get the students to do the work!

Online learning for beginners: 7. Why not just record my lectures?

Why not just put your lecture capture lessons online? Image: MediaCore, 2014

Why not just put your lecture capture lessons online?
Image: MediaCore, 2014

This is the seventh in a series of a dozen blog posts aimed at those new to online learning or thinking of possibly doing it. The other six are:

I gave a short answer to ‘Why not just record my lectures?’ in the fourth post in this series, but it deserves a fuller answer. It is natural that faculty and instructors want to use an approach to teaching that is not only familiar and comfortable, but has been used for hundreds of years, so has passed the test of time. However, there are several reasons why recorded classroom lectures are not a good idea for online learning, at least not as the main form of delivering online courses.

Start with the students

When designing online courses, you need to start by thinking about the context of the online learner. An online learner is usually studying in an isolated situation, without other students or the instructor physically present. There are many ways to overcome the isolation of the online learner (dealt with in later posts), but giving them recordings of 50 minute classroom presentations is not one of them.

In a classroom context there are many interactive cues or contexts – such as the response of other students, the look on students faces – that result in slight but important adjustments on the instructor’s part, and which help maintain student concentration and interest. Even if a live class was present at the time of the recording, these cues are usually lacking when students are studying a recorded video at home, in the library, or on the bus.

There is also research evidence that suggests for every hour of presentation, online students need to spend between two to three hours of additional time going over the recording, stopping and starting, to ensure they have fully understood. This is a good benefit of recording compared to even a live lecture, but it also ups the student workload, especially if there is other work to be done, such as readings, assignments, and practical work. Managing student workload is a key factor in ensuring high completion rates for online students.

Lastly, even when recorded lectures are strongly integrated with other activities, such as subsequent classroom discussions or assignments, students often skimp on the video preparation, either skimming the video or not watching it at all. The more isolated the student, the more likely this is to happen.

The changing nature of learning in a digital age

One of the main reasons for moving to online learning is to help develop the knowledge and skills needed in a knowledge-based society and particularly in a digital age. These new forms of knowledge – such as Internet-based sources and rapidly changing content – and in particular the skills required to master these forms of knowledge, such as knowledge management, independent learning and use of digital media – are not handled well through lectures. In particular the lecturer is doing the knowledge management, the modelling and the organization of content, not the students.

In other words lectures require a more passive approach from the learner which is not suitable for isolated learners who need to be active and engaged in their learning, as much for motivational reasons as for developing the knowledge and skills needed. (The same could also be true for classroom based students, incidentally.) Indeed, one of the principle reasons for moving to online teaching is to move away from the limitations of lecture-based classes, and to exploit the benefits of online study.

Video as a teaching medium

Asking ‘Why can’t I move my lectures online?’ is really the wrong question. It assumes that what I’m doing in the classroom will work equally well on video for online students. The right question though should be: ‘What is the best use of video for students studying online?’

In media terms, a recorded lecture is mainly a talking head, with, if students are lucky, textual illustration (e.g. Powerpoint slides). There has been a great deal of research on the best mix of voice, images, and text in video for teaching (see, for instance Mayer, 2009). To incorporate the factors that make the use of video effective for learning, the type of lecture usually delivered in a classroom would need to be considerably redesigned to make it more effective for remote learners.

In addition, there are many other, more creative and relevant ways than lectures for using video for teaching, such as demonstrations of equipment, experiments or processes, animation, and examples drawn from the real world to illustrate abstract concepts.

Successful uses of video for lectures

It could be argued that MOOCs, the Khan Academy, and TED talks are all examples of the successful use of lectures on video. However, they are not the typical classroom lecture delivered three times a week over a 13 week semester.

I have heard instructors say that their MOOC lectures are much better than their classroom lectures, because they put more time into the presentation. MOOC developers have learned to adapt the 50 minute lecture to better fit the online format, with shorter, 10-15 minutes videos, and shorter courses. This is fine for non-credit programming but does not fit the Carnegie-based 13 week semester model for credit programs. Costs for producing successful MOOC lectures run over $100,000 a lecture, production costs that are not sustainable for moving large numbers of classroom lectures online.

Sal Khan is an inspired lecturer who uses voice over combined with on-screen digital notes. His technique is not the same as recording a classroom lecture with whiteboard notes. For a start, the audio and screen quality is much higher, but it is also the technique of constructing teaching in appropriate chunks of recorded time that requires considerable thought and preparation. This is not to say that classroom lecturers could not do this, but it would require once again redesigning the classroom lecture.

Lastly, TED talks require a great deal of preparation and rehearsal, and again are much shorter than the typical classroom lecture.

So, yes, recorded video can work online, but it needs to be designed specifically to suit the mode of delivery. There are also other ways to design online learning that do not necessarily require so much work, and other uses of video for teaching that are more appropriate.

What are the alternatives?

Too many to list them all here, but one is to use an online learning management system, such as Blackboard, Moodle or D2L. These provide a weekly structure for ‘lessons’, organize content in the form of text or online readings, provide a forum for discussion on course topics, provide regular online activities and assignments, and could include links to short videos. Indeed, a short introductory video to a topic by the instructor is often a good idea, providing a personal link between you and your students.

I will discuss other possible online learning environments in later posts.

Implications

  1. A talking head delivering 50 minute lectures is in general not a good way to teach online learners.
  2. It is better in the long run to sit down with an instructional designer and build a course from scratch that is appropriate for an online learning environment, rather than try to force your classroom teaching online.
  3. Video is a good medium to use for online learning, but only if it exploits its unique pedagogical benefits.
  4. Talking heads are therefore useful only in particular contexts, and not as a way to deliver a whole course or program online.
  5. Developing quality video for online learning requires a professional approach involving lecturer, instructional designer and a multimedia or video producer.

Follow up

For a critique of the limitations of classroom lectures based on research by Donald Bligh, see Chapter 3.3 Transmissive lectures: learning by listening in Teaching in a Digital Age.

For a good summary of best design principles for developing video/multimedia for learning, based on research on the learning effectiveness of video, see the University of British Columbia’s Design Principles for Multimedia.

For a discussion of the pedagogical potential of video, see Chapter 7.4.2, Presentational features in Teaching in a Digital Age

If you want to follow up on the research and theory on which this post is based see:

  • Bligh, D. (2000) What’s the Use of Lectures? San Francisco: Jossey-Bass
  • Mayer, R. E. (2009). Multimedia Learning (2nd ed). New York: Cambridge University Press.
  • McKeachie, W. and Svinicki, M. (2006) McKeachie’s Teaching Tips: Strategies, Research and Theory for College and University Teachers Boston/New York: Houghton Mifflin

Up next

Won’t online learning be more work?

Your turn

If you have comments, questions or plain disagree, please use the comment box below.

Online learning for beginners: 4. ‘What kinds of online learning are there?’

©acreelman.blogspot.com, 2013

©acreelman.blogspot.com, 2013

This is the fourth of a series of a dozen blog posts aimed at those new to online learning or thinking of possibly doing it. The other three are:

In the third post, I pointed out that MOOCs were just one of the many different types of online learning. In this post, I will provide more detail about the various approaches to online learning, and will also provide a personal evaluation of each approach in terms of quality. This post will be a little longer than normal, as there are not only many approaches to online learning, but the field is also rapidly changing and developing.

Different approaches to online learning

In the first post, ‘What is online learning’? I pointed out that there is a continuum of teaching, from no use of online learning through blended learning, to fully online (or distance) learning. However, even within these categories, there are different possible approaches:

1. Online class notes

Approach

Students access Powerpoint slides and pdfs from a class web site which may be a part of an institution’s learning management system (see below) or it may be just a web site created by the instructor or made available by the institution. Usually the same slides or notes that are given to students taking an on-campus class are put up on the web site for online students, often on a weekly basis.  Online students access the relevant documents, and take the same assessments or exams as on-campus students, either remotely, in the form of computer-marked assignments, or on campus. If online students have questions, they can usually e-mail the instructor. Students usually work individually, although if a learning management system is available, there may be voluntary online discussion between students through the LMS’s discussion forum or social media.

Evaluation

This method is often used by novice online instructors. It requires, on the surface, little extra work for the instructor, once the materials are loaded.

The main problem is that such an approach is not adapted to the needs of online learners, who usually need more support than this model provides. The Powerpoint slides or pdfs do not allow for student interaction with the learning materials (unless they are re-written to do this). If there is a problem with the materials, in terms of the content not being clear, every student is likely to have the same difficulty. Instructors in this model therefore often find that they are overwhelmed with e-mail. If there are not activities (other than reading) scheduled for every week, students tend to get behind. Coming on-campus to do assignments or exams is also a problem for students who have chosen the online option because they have difficulty in getting to campus on a scheduled basis. Students in such courses often feel isolated and unsupported, and therefore such courses usually have much higher non-completion rates. And in the end, instructors find that this approach ends up being a lot more work than they anticipated.

2. Recorded lectures

Approach

The increased availability of technology such as lecture capture, which records classroom lectures on digital video and stores them for later downloading over the Internet, and desk-top cameras, has resulted in many instructors offering online courses built around recorded lectures. The lectures are usually the same as those for on-campus classes. Many MOOCs, as well as courses for credit, use recorded lectures as the main form of delivery.

Evaluation

This approach is again convenient for instructors, especially if they are giving a face-to-face lecture anyway and have technical help in recording and storing the lectures. However this approach suffers from many of the same problems as the class notes method above. An additional problem is that if the recording is of a normal 50 minute lecture, students often suffer from what is known as cognitive overload. Although students viewing a recorded lecture have the opportunity to stop and replay material, this can mean that a 50 minute lecture may take up several hours for an online student. MOOC designers, and TED talk designers, have realised this and often they have limited a single video to 10-20 minutes in length. Nevertheless this does not work so well in a full credit program with maybe 39 lectures over a 13 week semester. Providing transcripts of the lectures is not only time consuming and adds costs, but again increases the cognitive load for students. Lastly, there is considerable research that questions the value of lectures as a teaching method.

3. Webinars

Approach

These are ‘live’ sessions usually consisting of a lecture delivered over the Internet, supported by Powerpoint slides with opportunities for live online chat for the participants. Webinars can be recorded and made available for online access at another time. Again, ‘good’ webinars tend to be broken up into smaller 5-10 segments of presentation followed by either online voice or more commonly (for group management reasons) text comments and questions contributed by participants to which the lecturer responds.

Evaluation

Webinars come closer to mirroring a live face-to-face class than either class notes or recorded lectures, and need relatively little adaptation or change for instructors. While webinars tend to be more interactive than recorded lectures, again it is difficult to cover a whole curriculum through webinars alone. Also participants need to be available at a set time, which restricts the flexibility or availability for online students, although the availability of the recording can offset that to some extent. Webinars using a lecture format also suffer from the same pedagogical limitations for online students as recorded lectures.

4. Instructionally-designed online courses based on a learning management system

These are probably the most common form of online courses for credit and more importantly, they have proved themselves with high completion rates and quality learning.

Approach

A whole science of instructional design has been developed since the 1940s based on pedagogical theory, research on how students learn, the appropriate use of technology, and the evaluation of learning outcomes, and this approach has been applied systematically to the design of fully online and increasingly blended courses. Usually an instructor will work with a professional instructional designer to redesign a classroom course or even a new course for use by online, distance learners. The instructor will be asked to define desired learning objectives, or learning outcomes, the content will be chosen to support the development of such objectives, and organised into ‘blocks’ of study (weekly or more) so that the whole curriculum can be covered over the semester. Assessment will be linked to the desired learning objectives. Sometimes objectives are determined through an analysis of the assessment requirements for equivalent face-to-face classes, if these are not already formally defined. Decisions will be made about which media (text, audio, video, computing) to use in terms of their appropriateness for meeting the defined learning objectives. Particular attention is paid to providing regular student activities, and managing student and instructor workload. Online learning management systems are often used to provide a structure for the course, opportunities for instructor-monitored student discussion, and online assessment tools.

Evaluation

This approach has been used very successfully with the design of fully online courses, usually leading to high completion rates and good quality learning outcomes. In some cases, it has also been successfully applied to blended courses. It is from this approach that many of the best practices in online learning have been identified. It means working in a team, often consisting of a senior faculty member, and for large classes, sessional or contract instructors and/or teaching assistants, an instructional designer, and other technical support staff, such as web designers, that can be called upon as necessary. However, this approach appears initially to be more costly for an institution, and more work for an instructor. It can take up to two years to design and develop a large fully online course, although courses for small classes (less than 40) can be designed in a  much shorter period. However, if the course or program attracts new students, tuition and other revenues can offset many of the additional costs, for instance, paying for release time for faculty to work on course design and development.

This is an interactive infographic. To see more detail on each of the five stages, click on each stage in the graphic © Flexible Learning Australia, 2014

This is an interactive infographic. To see more detail on each of the five stages, click on each stage in the graphic
© Flexible Learning Australia, 2014

5. Designs based on open education and emerging technologies

Approach

This is a bit of a rag-bag category for a small but growing number of online course designs that seek fully to exploit specific characteristics of new media and open educational approaches. These might include:

  • connectivist MOOCs‘ that focus on the contributions of all participants in an extended online network;
  • courses built around social media tools such as blogs, wikis, and e-portfolios;
  • approaches that exploit open educational resources, such as open textbooks and content freely available over the Internet;
  • courses built around emerging technologies, such as virtual worlds, gaming, and augmented reality.

Common features of such courses are increased activity and choices for learners, more diversity in course designs, and ‘agile’ or quick design and development. In such courses, students are often encouraged to seek, analyse, evaluate and apply content to real world issues or contexts, rather than the instructor being primarily responsible for content choice and delivery.

Evaluation

The main rationale for such courses is as follows:

  • they are more appropriate for developing the skills and knowledge learners need in a digital age;
  • they are more active and engaging for learners, resulting in deeper learning;
  • they make better use of new technologies by exploiting their unique teaching potential;
  • these approaches usually result in quicker and relatively low-cost course development and delivery compared with the instructional design approach;
  • they are transforming teaching into a more modern, relevant methodology that better suits today’s learners.

However, such approaches require highly confident and effective instructors with experience in using new technology for teaching, combined with the team approach described earlier. Above all instructors need to have a good grasp of both pedagogy and technology, as well as subject expertise. Direct instructional design and technology support is also essential. Most of these approaches are so new that there is relatively little research on their effectiveness. They are therefore a high risk activity for an instructor, especially those with little experience of online teaching.

This is a very abbreviated description of fast-developing, constantly changing approaches to online learning. You are especially encouraged to do the follow-up reading below.

Implications

  1. It is generally a mistake to merely transport your classroom teaching to an online environment. Online students work in different contexts and have different needs to students in face-to-face classes. Online courses need to be redesigned to accommodate the unique requirements of online learners.
  2. There is a strong body of knowledge about how to design online courses well. You ignore this at your peril. Consequences of ignoring best practices may include poor learning results, a much heavier work-load than anticipated, and dissatisfied students and superiors.
  3. It is best to work in a team. Instructional designers have knowledge about teaching online that most instructors lack. While you will always be in control of content selection, assessment and overall teaching approach, instructional designers need to be listened to as equals.
  4. New technologies have the promise of radically changing teaching, making it more relevant, more engaging for students, and more exciting and challenging for an instructor.

Follow-up

This is a very simplified account of the different kinds of online learning. For a more extensive coverage, see:

For more on the effectiveness of lectures, see:

For more on cognitive load and online learning design, see:

For more on instructional design, see:

For more on designs based on open education and emerging technologies see:

For more on emerging technologies in online learning see:

Up next

When should I use online learning? (This will be much shorter, I promise!)

Your turn

If you have comments, questions or plain disagree, please use the comment box below.