July 16, 2018

Learning analytics, student satisfaction, and student performance at the UK Open University

There is very little correlation between student satisfaction and student performance. Image: Bart Rienties. Click on image to see the video.

Rienties, B. and Toetenel, L. (2016) The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules, Computers in Human Behaviour, Vol. 60, pp.333-341

Li, N. et al. (2017) Online learning experiences of new versus continuing learners: a large-scale replication study, Assessment and Evaluation in Higher Education, Vol. 42, No. 4, pp.657-672

It’s never too late to learn

It’s been a hectic month with two trips from Vancouver to Ontario and back and one to the UK and back, a total of four keynotes, two panel sessions and two one day consultancies. By the time I got to the end of the month’s travels, I had learned so much that at a conference in Toronto I had to go to my room and lie down  – I just couldn’t take any more!

At my age, it takes time to process all this new information, but I will try to summarise the main points of what I learned in the next three posts.

Learning analytics at the Open University

The Open University, with over 100,000 students and more than 1,000 courses (modules), and most of its teaching online in one form or another, is an ideal context for the application of learning analytics. Fortunately the OU has some of the world leaders in this field. 

At the conference on STEM teaching at the Open University that I attended as the opening keynote, the closing keynote was given by Bart Rienties, Professor of Learning Analytics at the Institute of Educational Technology at the UK Open University. Rienties and his team linked 151 modules (courses) and 111,256 students with students’ behaviour, satisfaction and performance at the Open University UK, using multiple regression models. 

His whole presentation (40 minutes, including questions) can be accessed online, and is well worth viewing, as it provides a clear summary of the results published in the two detailed papers listed above. As always, if you find my summary of results below of interest or challenging, I strongly recommend you view Bart’s video first, then read the two articles in more detail. Here’s what I took away.

There is little correlation between student course evaluations and student performance

This result is a bit of a zinger. The core dependent variable used was academic retention (the number of learners who completed and passed the module relative to the number of learners who registered for each module). As Rientes and Toetenel (p.340) comment, almost as an aside, 

it is remarkable that learner satisfaction and academic retention were not even mildly related to each other….Our findings seem to indicate that students may not always be the best judge of their own learning experience and what helps them in achieving the best outcome.’

The design of the course matters

One of the big challenges in online and blended learning is getting subject matter experts to recognise the importance of what the Open University calls ‘learning design.’ 

Conole (2012, p121) describes learning design as:

a methodology for enabling teachers/designers to make more informed decisions in how they go about designing learning activities and interventions, which is pedagogically informed and makes effective use of appropriate resources and technologies. LD is focussed on ‘what students do’ as part of their learning, rather than the ‘teaching’ which is focussed on the content that will be delivered.

Thus learning design is more than just instructional design.

However, Rienties at al. comment that ‘only a few studies have investigated how educators in practice are actually planning and designing their courses and whether this is then implemented as intended in the design phase.’ 

The OU has done a good job in breaking down some of the elements of learning design. The OU has mapped the elements of learning design in nearly 200 different courses. The elements of this mapping can be seen below (Rientes and Toetenal, 2016, p.335):

Rientes and Toetenel then analysed the correlations between each of these learning design elements against both learner satisfaction and learner performance. What they found is that what OU students liked did not match with learner performance. For instance, students were most satisfied with ‘assimilative’ activities, which are primarily content focused, and disliked communication activities, which are primarily social activities. However, better student retention was most strongly associated with communication activities, and overall, with the quality of the learning design.

Rientes and Toetenel conclude:

although more than 80% of learners were satisfied with their learning experience, learning does not always need to be a nice, pleasant experience. Learning can be hard and difficult at times, and making mistakes, persistence, receiving good feedback and support are important factors for continued learning….

An exclusive focus on learner satisfaction might distract institutions from understanding the impact of LD on learning experiences and academic retention. If our findings are replicated in other contexts, a crucial debate with academics, students and managers needs to develop whether universities should focus on happy students and customers, or whether universities should design learning activities that stretch learners to their maximum abilities and ensuring that they eventually pass the module. Where possible, appropriate communication tasks that align with the learning objectives of the course may seem to be a way forward to enhance academic retention.

Be careful what you measure

As Rientes and Toetenel put it:

Simple LA metrics (e.g., number of clicks, number of downloads) may actually hamper the advancement of LA research. For example, using a longitudinal data analysis of over 120 variables from three different VLE/LMS systems and a range of motivational, emotions and learning styles indicators, Tempelaar et al. (2015) found that most of the 40 proxies of simple” VLE LA metrics provided limited insights into the complexity of learning dynamics over time. On average, these clicking behaviour proxies were only able to explain around 10% of variation in academic performance.

In contrast, learning motivations, emotions (attitudes), and learners’ activities during continuous assessments (behaviour) significantly improved explained variance (up to 50%) and could provide an opportunity for teachers to help at-risk learners at a relatively early stage of their university studies.

My conclusions

Student feedback on the quality of a course is really important but it is more useful as a conversation between students and instructors/designers than as a quantitative ranking of the quality of a course.  In fact using learner satisfaction as a way to rank teaching is highly misleading. Learner satisfaction encompasses a very wide range of factors as well as the teaching of a particular course. It is possible to imagine a highly effective course where teaching in a transmissive or assimilative manner is minimal, but student activities are wide, varied and relevant to the development of significant learning outcomes. Students, at least initially, may not like this because this may be a new experience for them, and because they must take more responsibility for their learning. Thus good communication and explanation of why particular approaches to teaching have been chosen is essential (see my comment to a question on the video).

Perhaps though the biggest limitation of student satisfaction for assessing the quality of the teaching is the often very low response rates from students, limited evaluation questions due to standardization (the same questions irrespective of the nature of the course), and the poor quality of the student responses. This is no way to assess the quality of an individual teacher or a whole institution, yet far too many institutions and governments are building this into their evaluation of teachers/instructors and institutions.

I have been fairly skeptical of learning analytics up to now, because of the tendency to focus more on what is easily measurable (simple metrics) than on what students actually do qualitatively when they are learning. The focus on learning design variables in these studies is refreshing and important but so will be analysis of student learning habits.

Finally, this research provides quantitative evidence of the importance of learning design in online and distance teaching. Good design leads to better learning outcomes. Why then are we not applying this knowledge to the design of all university and college courses, and not just online courses? We need a shift in the power balance between university and college subject experts and learning designers resulting in the latter being treated as at least equals in the teaching process.

References

Conole, G. (2012). Designing for learning in an open world. Dordrecht: Springer

Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: learning analytics in a data-rich context. Computers in Human Behavior, 47, 157e167. http://dx.doi.org/10.1016/j.chb.2014.05.038.

 

‘Humans Wanted’: online learning and skills development

Royal Bank of Canada (2018) Humans Wanted Toronto ON: Royal Bank of Canada

I have at last got hold of a full copy of this report that came out a couple of weeks ago. Much to my surprise, I found the report essential reading for anyone in education, mainly because it is relatively specific about the skills that the Canadian job market will need between 2018 and 2021, and the results were not quite what I expected to see.

Conclusions from the report

I can’t better the summary in the report itself:

1. More than 25% of Canadian jobs will be heavily disrupted by technology in the coming decade. Fully half will go through a significant overhaul of the skills required.

2. An assessment of 20,000 skills rankings across 300 occupations and 2.4 million expected job openings shows an increasing demand for foundational skills such as critical thinking, co-ordination, social perceptiveness, active listening and complex problem solving.

3. Despite projected heavy job displacement in many sectors and occupations, the Canadian economy is expected to add 2.4 million jobs over the next four years, all of which will require this new mix of skills.

4. Canada’s education system, training programs and labour market initiatives are inadequately designed to help Canadian youth navigate this new skills economy.

5. Canadian employers are generally not prepared, through hiring, training or retraining, to recruit and develop the skills needed to make their organizations more competitive in a digital economy.

6. Our researchers identified a new way of grouping jobs into six “clusters,” based on essential skills by occupation rather than by industry.

7. By focusing on the foundational skills required within each of these clusters, a high degree of mobility is possible between jobs.

8. Digital fluency will be essential to all new jobs. This does not mean we need a nation of coders, but a nation that is digitally literate.

9. Global competencies like cultural awareness, language, and adaptability will be in demand.

10. Virtually all job openings will place significant importance on judgment and decision making and more than two thirds will value an ability to manage people and resources.

So, no, automation is not going to remove all work for humans, but it is going to change very much the nature of that work, and it is in this sense that technology will be disruptive. Workers will be needed in the future but they will need to be very different workers from the past.

This has massive implications for teaching and learning and the bank is in my view correct in arguing that Canada’s education system is inadequately designed to help Canadian youth navigate this new skills economy.

What skills will be in demand?

Not the ones most of us would have thought that a bank would identify:

© Royal Bank of Canada, 2018

You will see that the most in demand skills will be active listening, speaking, critical thinking and reading comprehension, while the least important skills include science, programming and technology design.

In other words, ‘soft skills’ will be most needed for human work. While this may seem obvious to many educators, it is refreshing to hear this from a business perspective as well.

Methodology

How did the Royal Bank not only identify these skills and their importance, but also how did it put actual numbers in terms of workers against these skills?

The data were derived from an interesting application of big data: an analysis of the skills listed on the web in ‘future-oriented’ job advertisements through media such as LinkedIn, combined with more qualitative interviews with employers, policy-makers, educators and young people.

What does this mean for teaching and learning?

There are several challenges I see:

  • first, getting teachers and instructors to accept that these (and other) skills need to be taught within any subject domain;
  • second, as these skills are not likely to be developed within a singe course, identifying how best to teach these skills at different ages, throughout a program of study, and indeed throughout life;
  • third, codifying these skills in terms of appropriate teaching and assessment methods; too often educators claim they are teaching these skills but if so, it is often implicit or not clear how or even if students acquire these skills.
  • we need to determine how best digital technology/e-learning can support the development of skills. For instance well-designed digital learning can enable skills practice and feedback at scale, freeing teachers and instructors to focus on what needs to be done on a face-to-face basis.

It’s not just about work

The Royal Bank has done a very good job in identifying work-force skills, but these are not the only skills needed in a digital age. Equally if not more important are the skills we need as humans in handling everyday life in a digital age. Examples would be:

  • a wide range of non-work oriented digital literacy skills, such as managing our digital identities (see UBC’s Digital Tattoo as an excellent example) so we as individuals have at least some control over the technology and how it is used
  • understanding the organization and power structures of digital companies and digital technologies: one example might be understanding how to identify and challenge algorithmic decision-making, for instance
  • teaching the important non-digital skills necessary in a digital society (for instance, mindfulness, or social awareness and conduct in both real and online environments).

Identifying such skills and finding ways to integrate the development of such skills within the curriculum is a major challenge but essential if we are to not only survive but thrive as humans in a digital world. We are just getting started on this, but it’s none too soon. In the meantime, the Royal Bank has done a good job in making the discussion about 21st century skills more concrete and practical.

Online learning for beginners: 8. Won’t online learning be more work?

Books lots! 2

This is the eighth in a series of a dozen blog posts aimed at those new to online learning or thinking of possibly doing it. The previous seven are:

More work?

The short answer is, yes, of course, at least in the short term. This is because online teaching is the same as any other skill. When you first start, you have to learn a lot, and do things you haven’t done before. For instance, as I discussed in earlier posts, you have to think carefully about why you are using online learning, talk to colleagues and work with other professionals such as instructional and web designers, master the technology, such as video recording or a learning management system, and basically re-think and re-design your teaching. This will take time, and your first online course will undoubtedly be more work and more challenging than your most recent face-to-face course.

However, in the long run, there is no reason why online teaching should be more work than face-to-face teaching, all other things being equal (which, of course, they never are in teaching). As always, there are important conditions to be met, if you don’t want to be swamped with extra work. So let’s look at what these conditions are.

Re-design your teaching

In a previous post in this series, I warned against trying to move your face-to-face teaching online, by just recording lectures. Although this may seem to be a time saver when developing an online course, it can cause a lot more work down the line. There will always be some students who don’t understand parts of a lecture and if something isn’t clear all students may have the same problem. When this happens, watch the e-mails or phone calls or even tweets roll in from students – or watch the course completion rate take a dive.

The answer is to use what is called ‘learning design’ or course design: setting clear learning outcomes or objectives for the course, breaking down the learning into manageable chunks of time for the students, providing appropriate learning activities for online learners, for instance, online discussion forums, and ensuring that assessment and feedback is continuous throughout the course, all the time thinking of the context in which the online learner is working. The trick is to move much of the work of finding, analyzing and applying content, and development of skills such as independent thinking, critical thinking, and problem solving, from you to the students, but under your guidance.

For most instructors, this means spending a good deal of time preparing the course in advance of its actual delivery. This means having the weekly modules that students study ready well in advance of the opening of the course. Although there will always be the need for minor changes to content in subsequent years, the bulk of the design and development of the course is done in the first year of its offering.

Most instructors in fact find a time shift when they move to online teaching. The more time you put into the development of the course in the first year, the less time you find yourself spending on content delivery during the delivery of the course, because it is already there. Multiply this over several offerings of the course and the time shift can lead to either significant time savings for you, or, more likely, your spending your time better in working directly, if online, with students, such as monitoring and contributing to online discussion of the course content.

For this reason, many institutions now offer funding to enable you to ‘buy yourself out’ of a face-to-face class for one or two semesters in order to prepare your first online course. Once you have some experience in this more traditional form of online learning, you can move to more ‘agile’ designs later, but that is another matter altogether. The first time out, you and your students need a clear structure and framework for the course.

Also, it is at this stage of course development that working with other professionals such as an instructional designer and web designer is most valuable. They should be able to provide the necessary advice and above all a framework and timetable for your work in designing the course.

Managing class size

I mentioned earlier that online teaching should not be more work, all things being equal. However, sometimes the aim is to use online learning in order to handle large classes or take extra students. These pressures may be coming from the administration rather than from you – or alternatively you may be concerned about the quality of the teaching of large face-to-face classes when many are delivered not by you but by teaching assistants who have barely more content knowledge than those they are teaching and who in particular may not have good lecturing skills.

The general rule for the most appropriate numbers for an individual instructor to teach online is pretty much the same as for face-to-face teaching. Once the instructor:student ratio goes over 1:30, it becomes harder to individualize the teaching and the instructor’s work load increases, unless the course is focused mainly on quantitative or ‘objective’ outcomes that can be automatically assessed, through, for instance, computer marked assignments. It is generally the marking that leads to overload when classes get beyond 30 per instructor.

However, because with online learning the content is available at any time and any place for students, there is some scope for scaling up the teaching to handle larger numbers. In particular, if the teaching content on the course is well developed by a top quality professor or instructor, all students will receive the same quality of content instruction. This means then that learner support and student assessment (marking) can be supported by contract sessional instructors as class size increases.

The availability of funding for hiring additional sessional instructors will depend on the business model behind the online program. If you are merely moving students from an existing face-to-face course to an online course, then there will be no extra money from tuition fees. However, if the online program is attracting new students paying additional tuition fees, then the extra funding can be used to hire more sessional instructors. In many North American universities, the tuition fee once an online course is developed more than covers the cost of additional sessional instructors, even with ‘steps’ of 30 students (i.e. for every additional 30 students you hire another sessional). Much of this of course will depend on faculty agreements, but from your point of view, re-design of a large face-to-face course by moving it online can not only improve the quality but also enable you to manage your own workload better.

What I would advise against is the use of graduate students as teaching assistants for online courses. The re-design of online courses requires instructors who can go beyond the ‘recorded’ content of an online course and can push students in online discussion groups, for instance, to challenge ideas and go deeper than just the formal online content. This requires sessional instructors with a good understanding of content and good inter-personal teaching skills to handle the extra students as class size increases.

In summary then, managing your workload as online class size increases requires several conditions:

  • ‘core’ content of high quality that does not need to be changed a lot from year to year;
  • learning/course design that provides a strong structure for students so that it is clear what they need to do when studying;
  • professional instructional design and web/media design support;
  • flexibility to hire additional, well-qualified sessional instructors as class size increases.

In the end, this may mean moving to a team approach to teaching large online classes. In some cases, the senior instructor’s responsibility may not involve direct teaching at all, but being responsible for the curriculum/content, setting learning outcomes, designing assessments, and supervising the learning support and assignment marking provided by sessional instructors.

Shifting from content to skills development

What online learning can do is enable you, as an instructor or teacher, to move away from ‘sage-on-the-stage’, where you are responsible for choosing and delivering content, and assessing how well students have comprehended this content, to ‘guide-on-the-side’, where students find, analyze and apply content, and develop higher level skills through practice, often working with other students online, through discussion or project work, but always under your guidance, or under the guidance of a team of sessional instructors that you monitor.

Implications

  1. Such changes inevitably mean more work, and more challenges, initially, in moving to online learning, but the benefits in both the quality of what your students learn, and the quality of your own engagement with students, can be substantial.
  2. There are also strategies for managing your workload when teaching online, so that over time you can balance better your teaching, research and administrative responsibilities.
  3. But online learning is not something to be undertaken lightly. You need to do it professionally, or it will be both more work and very frustrating.

Follow-up

For more on the design of online courses, see:

Later posts in this series will go into more detail about providing support for online learners that will also enable you to manage better your workload.

Up next

How can I do online learning well?

Your turn

If you have comments, questions or plain disagree, please use the comment box below.

Get the students to do the work!

Get the students to do the work!

Online learning for beginners: 5. When should I use online learning?

Knowledge-based industries include entertainment, such as video games design

Most subject disciplines now require students to know how technology influences their field of study

This is the fifth of a series of a dozen blog posts aimed at those new to online learning or thinking of possibly doing it. The other four are:

This question ‘When should I use online learning?’ is difficult to answer in a short post because there are many possible reasons, and as always in education, the answers are absolutely dependent on the specific context in which you are working, but the reasons can be classified under three main headings: academic, market, and policy/administrative.

Academic reasons

These boil down to relevancy and the changing nature of knowledge in a digital age.

Curriculum requirements

Technology is affecting the content of curriculum in nearly all subject disciplines. It is increasingly difficult to think of an academic area that is not undergoing profound changes as a result of information and communications technologies (ICTs). For instance, any business program now needs to look at the impact of social media and the Internet on marketing and on the delivery of goods. How are ICTs going to change financial investments and advising? In science and engineering, to what extent would animation, simulations or the use of virtual reality enable better understanding of three-dimensional phenomena, equations or formulae? In humanities and fine arts, to what extent are ICTs changing the way we express ourselves? How do we ensure our students are digitally literate and responsible? How do we prepare our students for a world controlled by massive technology companies who track our every movement and expression? It is difficult to think how these issues can be addressed without students themselves going online to study such issues.

Skills development

Also, the skills that our students will need to develop in a digital age will often best be achieved through the use of ICTs. In Chapter 1.2 of Teaching in a Digital Age, I give more detailed examples of such skills. Many of these skills are not only best developed by, but may not even be possible without, students spending an extensive period studying online.

However, I want to focus on two ‘core’ 21st century skills: independent learning and knowledge management. In a knowledge-based society, students will need to go on learning throughout life and outside the formal academic curriculum. Jobs are constantly changing as the knowledge base changes, and even our social lives are increasingly dominated by technological change. Independent learning – or self-learning – is a skill that itself can be taught. Online learning in particular requires self-discipline and independent learning, because the instructor is often not physically ‘there’. Thus gradually introducing learners to online learning can help build their independent learning skills.

Perhaps the overarching ’21st century skill’ though is knowledge management: how to find, analyse, evaluate, apply and communicate knowledge, especially when much of this knowledge is Internet-based or located, and constantly undergoing change. Students then need many opportunities to practice such skills, and online learning often provides a means by which this can be done in a cost-effective manner.

Whether we like it or not, an understanding and management of the use of ICTs is becoming critical in almost any subject area. Students will need to go online to study such phenomena, and to practice core 21st century skills. To do this students will need to spend much more time than at present studying online. (Again, though, we need to ensure that the balance between online and face-to-face time is also properly managed.)

Market reasons

Not only is knowledge undergoing rapid change, so are demographics. In most economically advanced societies, the population is aging. Over time, this will mean fewer younger students coming straight from high school, and more lifelong learners, perhaps already with post-secondary qualifications, but wanting to upgrade or move to a new profession or job and hence needing new knowledge and skills.

Also, with mass education, our students are increasingly diverse, in culture, languages and prior knowledge. One size of teaching does not fit all. We need ways then to individualise our programs. In particular, there are many pedagogical problems with very large lecture classes. They do not meet the needs of an increasingly diverse student population. Online learning is one way to allow students to work at different speeds, and to individualise the learning with online options enabling some choice in topics or level of study.

The changing population base offers opportunities as well as challenges. For instance, your area of research may be too specialised to offer a whole course or program within your current catchment area, but by going online you can attract enough students nationally or globally to make the effort worthwhile. These will be new students bringing in extra tuition revenues that can cover the full costs of an online masters degree, for instance. At the same time, online learning will enable critically important areas of academic development to reach a wider audience, helping create new labour markets and expand new areas of research.

Policy/administrative

We all know the situation where a President or Vice Chancellor has gone to a conference and come back ‘converted’. Suddenly the whole ship is expected to make an abrupt right turn and head off in a new direction. Unfortunately, online learning often leads to enthusiastic converts. MOOCs are a classic example of how a few elite universities suddenly got the attention of university leaders, who all charged off in the same direction.

Nevertheless, there can also be good policy reasons for institutional leadership wanting to move more to blended or flexible learning, for instance. One is to improve the quality of teaching and learning (breaking up large lecture classes is one example); another reason is to expand the reach of the university or college beyond its traditional base, for demographic and economic reasons; a third is to provide more flexibility for full-time students who are often working up to 15 hours a week to pay for their studies.

These policy shifts provide an excellent opportunity then to meet some of the academic rationales mentioned earlier. It is much easier to move into online learning if there is institutional support for this. This will include often extra money for release time for faculty to develop online courses, extra support in the way of instructional and media design, and even better chances of promotion or tenure.

Implications

  1. It can be seen that while market and policy reasons may be forcing you towards online learning, there are also excellent and valid academic reasons for moving in this direction.
  2. However, the extent to which online learning is a solution will depend very much on the particular context in which it will be used. It is essential that you think through carefully where it best fits within your own teaching context: blended learning for undergraduate students; masters programs for working professionals; skills development for applied learning; or all of these?
  3. Online learning is not going to go away. It will play a larger role in teaching in even the most campus-based institutions. Most of all, your students can benefit immensely from online learning, but only if it is done well.

Follow-up

Chapter 1, Fundamental Change in Education, of Teaching in a Digital Age, is basically a broader rationale for the use of online learning

Chapters 3 and 4 look at ways to individualise learning; see in particular:

Up next

‘How do I start?’

Your turn

If you have comments, questions or plain disagree, please use the comment box below.

 

Online learning for beginners: 2. Isn’t online learning worse than face-to-face teaching?

Distance education: anyone sitting more than 10 rows from the front

Distance learning: anyone sitting more than 10 rows from the front

The short answer to this question is: no, online learning is neither inherently worse – nor better – than face-to-face teaching; it all depends on the circumstances.

The research evidence

There have been thousands of studies comparing face-to-face teaching to teaching with a wide range of different technologies, such as televised lectures, computer-based learning, and online learning, or comparing face-to-face teaching with distance education.

With regard to online learning there have been several meta-studies. A meta-study combines the results of many ‘well-conducted scientific’ studies, usually studies that use the matched comparisons or quasi-experimental method (Means et al., 2011; Barnard et al., 2014). Nearly all such ‘well-conducted’ meta-studies find no or little significant difference in the modes of delivery, in terms of the effect on student learning or performance. For instance, Means et al. (2011), in a major meta-analysis of research on blended and online learning for the U.S. Department of Education, reported:

In recent experimental and quasi-experimental studies contrasting blends of online and face-to-face instruction with conventional face-to-face classes, blended instruction has been more effective, providing a rationale for the effort required to design and implement blended approaches. When used by itself, online learning appears to be as effective as conventional classroom instruction, but not more so.

However, the ‘no significant difference’ finding is often misinterpreted. If there is no difference, then why do online learning? I’m comfortable teaching face-to-face, so why should I change?

This is a misinterpretation of the findings, because there may indeed within any particular study be large differences between conditions (face-to-face vs online), but they cancel each other out over a wide range of studies, or because with matched comparisons you are looking at only very specific, strictly comparable conditions, that never exist in a real teaching context.

For instance the ‘base’ variable chosen is nearly always the traditional classroom. In order to make a ‘scientific’ comparison, the same learning objectives and same treatment (teaching) is applied to the comparative condition (online learning). This means using exactly the same kind of students, for instance, in both conditions. But what if (as is the case) online learning better suits non-traditional students, or will achieve better learning outcomes if the teaching is designed differently to suit the context of online learning?

Asking the right questions

Indeed, it is the variables or conditions for success that we should be examining, not just the technological delivery. In other words, we should be asking a question first posed by Wilbur Schramm as long ago as 1977:

What kinds of learning can different media best facilitate, and under what conditions?

In terms of making decisions then about mode of delivery, we should be asking, not which is the best method overall, but:

What are the most appropriate conditions for using face-to-face, blended or fully online learning respectively? 

So what are the conditions that best suit online learning?

There are a number of possible answers:

  • learners:
    • fully online learning best suits more mature, adult, lifelong learners who already have good independent learning skills and for work and family reasons don’t want to come on campus
    • blended learning or a mix of classroom and fully online courses best suits full time undergraduate students who are also working part-time to keep their debt down, and need the flexibility to do part of their studies online
    • ‘dependent’ learners who lack self-discipline or who don’t know how to manage their own learning probably will do better with face-to-face teaching; however independent learning is a skill that can be taught, so blended learning is a safe way to gradually introduce such students to more independent study methods
  • learning outcomes:
    • embedding technology within the teaching may better enable the development of certain ’21st century skills’, such as independent learning, confidence in using information technologies within a specific subject domain, and knowledge management
    • online learning may provide more time on task to enable more practice of skills, such as problem-solving in math
    • redesign of very large lecture classes, so that lectures are recorded and students come to class for discussion and questions, making the classes more interactive and hence improving learning outcomes

Even this is really putting the question round the wrong way. A better question is:

What are the challenges I am facing as an instructor (or my learners are facing as students) that could be better addressed through online learning? And what form of online learning will work best for my students?

Quality

However, the most important condition influencing the effectiveness of both face-to-face and online teaching is how well it is done. A badly designed and delivered face-to-face class will have worse learning outcomes than a well designed online course – and vice versa. Ensuring quality in online learning will be the topic of the last few blogs in this series.

Implications

  1. Don’t worry about the effectiveness of online learning. Under the right conditions, it works well.
  2. Start with the challenges you face. Keep an open mind when thinking about whether online learning might be a better solution than continuing in the same old way.
  3. If you think it might be a solution for some of your problems, start thinking about the necessary conditions for success. The next few blog posts should help you with this.

Follow up

Here is some suggested further reading on the effectiveness of online learning:

Up next

‘Aren’t MOOCs online learning?’ (to be posted later in the week July 18-22, 2016)

Comparing modes: horses for courses

Comparing modes: horses for courses