April 19, 2014

Innovation, quality and digital resources: the LINQ 2013 conference

Listen with webReader

No – it’s not Anita Ekberg in front of the Trevi fountain – it’s Rory McGreal!

The LINQ 2013 conference, held in May in Rome, Italy, addressed the issue of innovation, quality and digital resources (in particular, OERs – open educational resources). The conference was co-organized by the University of Duisberg-Essen and the U.N. Food and Agricultural Organization (FAO).

The focus of the conference, according to Christian Stracke, one of the main conference organizers, was on the interaction between the internet, globalization, innovation and quality in addressing ‘post-crisis’ educational needs.


Keynote speakers on the first day included:

To see an abstract of a paper and to access copies of the slides, click on the title of the paper above.

The second day was given over mainly to parallel sessions. For the full program, click here. For the slides accompanying each presentation, click on the title of the talk.


It’s impossible for me to do justice to the richness of the presentations, especially as I was pretty jet-lagged the whole time, but here are the main points I took away from the conference (for my comments on each of these points, see below):

  • Europe has a serious employment crisis, especially for young people. António Silva Mendes, from the European Commission, pointed out that in Europe the 7.5 million unemployed youth between the ages of 15-24 and the 6.5 million unemployed people between the ages of 25-29 represent a loss of 153 billion euros or nearly 1.5% of GDP. He argued that the EC was doing its best to ensure as much as possible that skills offered by educational and training institutions match the needs of the labour market. In particular there is a need to improve the quality of vocational teachers, as this has the biggest impact on quality outcomes. The EC is looking at ways to open up education through the use of OERs and improving ‘digital competencies’, and helping create online communities of practice for vocational teachers. It is also aiming to build partnerships between employers, educational and training providers and other agencies to help better match training to desired learning outcomes.
  • Jay Cross talked about how the nature of work has changed and the implications for the education and training of adults.He listed ten interesting core skills now needed in the workforce. He claimed that adult workers get 70% of their learning experientially, 20% from others, and 10% from formal education (not sure how he came to those figures – I must have been asleep at that point). He ended with a strong plea to focus as much on happiness at work and in life as on formal training, as this is more likely to lead to in depth learning.
  • Lettmayr provided data showing that those with higher levels of education and training had lower levels of unemployment, that there are many adults in the workforce (or unemployed) with low levels of education/training, and the older you are, the less training you get (tell me about it). Thus not enough attention is paid to adult learning.
  • Ignasi Labastida provided a fascinating presentation on the effect of copyright laws on the sharing of information in Europe (apparently the laws are much more restrictive there than in Canada, especially regarding fair use/dealing). He argued for greater use of open licenses such as those operated by the Creative Commons, but there was currently too much fear, concern for protecting individual academic’s rights, etc., that was inhibiting more open approaches to sharing educational materials in Europe (although I’m not sure that it’s that much different here in North America).
  • Rory McGreal was in good form, as usual. In educational creationism the lecture model in which the professor reads and the students listen (or sleep) is posited as the ideal form as Plato and the other ancient Greek idealists might have posited. On the other hand, educational Darwinists recognize that evolutionary change is driven by real world developments. As more and more of the world’s knowledge becomes accessible to anyone with an Internet connexion, the need to physically go to a special place, whether it be a classroom or a reading room is no longer a requirement. Rory then went on to give his usual impassioned speech about the need for free education through open resources and open education and a blistering attack on copyright. He then looked at the opportunities offered by new technologies such as mobile learning to make educational available to all.
  • Fred Mulder briefly covered the development of open education initiatives from the first open universities through OpenCourseWare, OERs and finally MOOCs. However, open education requires open services and open teaching efforts as well as OERs. He introduced the concept of 5COE (not sure what that is) then described the pan-European OpenupEdMOOCs being developed by the European Association of Distance Teaching Universities.
  • Miguel-Angel Sicilia finally gave a presentation on interlinking open educational resources through the use of metadata (technological conventions to make data available to machines). By this time jet lag had set in and I was fast asleep (my apologies.)

Ancient gate in Travetere, Rome

Evaluating the quality of digital resources

I actually changed my talk after the program was published, to focus on the evaluation of the quality of digital resources and open education, as this seemed to fit better the theme of the program. (That’s a problem of agreeing to speak and giving a title and abstract several months before being fully aware of the aims of the conference – that was my fault, not the organizers – so ignore the program title and abstract and focus on the slides).

I in fact used my ACTIONS model to assess the comparative quality of MOOCs, OERs, Open Universities, and ‘closed’ online credit programs. I then tried to assess each of these type of initiative, in a qualitative way, by looking how well example programs within each of these categories matched the following criteria:

  • access
  • costs
  • teaching functions
  • interaction
  • organization
  • novelty
  • speed and security

I ‘rigged’ the scoring to show that ‘closed’ credit online programs could still score higher than ‘open’ forms of education.

My main point is that access, although important, is not enough. Learning has to take place, and quality standards need to be met. (To clarify, open education is highly desirable, but it needs to be good quality open education). By focusing solely on open digital resources, we may forget that there are other critical elements that are needed for education to be successful.


The conference was also used to launch the International Council for Open Research and Open Education, an attempt to bring together two worlds with similar values that have up to now operated relatively independently. The aim is to set up a non-profit organization with no membership fees, relying entirely on volunteer contributions. More details can be found here


This interesting conference brought together mainly European and North American proponents of open education. In North America, ‘open’ education has been heavily pushed by private foundations, such as the Hewlett Foundation, while in Europe the European Commission and the Open Universities are leading the charge. In both cases, though, it is more a push or supply-driven approach. There was almost no discussion or presentations on models for incorporating or supporting OERs in credit-based programming for instance. Course design around OERs was hardly mentioned. Data about actual take-up (other than ‘hits’ to web sites) was lacking.

Also (apart from my session) there was little discussion about how to measure the quality of OERs – or even whether this was important – despite the theme of the conference. But is it enough for materials to be open? We need to get beyond the library concept of free access to resources (which of course is valuable in its own right), and talk about how such valuable resources can best be used.

Lastly, I have to say something about the need to match learning outcomes to the needs of employers. The problem in Europe is NOT a lack of qualified workers; there are just not enough jobs available. Sure the jobs that are available go to the highest qualified, but the problem is not educational, it’s economic. Turning out even more highly trained workers for jobs that don’t exist is a recipe for revolution. Austerity may bring short-term gains to the economic elite, but in the long run it could destroy the whole system. Are you listening Angela?

Graffiti in Rome

Discussing drop-out rates in MOOCs

Listen with webReader

Rivard, R. (2013) Measuring the MOOC drop-out rate Inside Higher Education, March 8

A good, balanced discussion about how ‘success’ in MOOCs should be measured, with some data about completion rates.

MOOCs forcing traditional academics to re-think their teaching

Listen with webReader

Lecture capture at MIT: we can do better

Rivard, R. (2013) Learning how to teach, Inside Higher Education, March 5

This is perhaps the most encouraging item of news about MOOCs so far (and also illustrates that edX is taking a more thoughtful approach to MOOCs than Coursera.)

The Provost of Harvard, Alan Garber, noted that because of the wide exposure that MOOCs offer to academics, they are wanting to ensure that their teaching matches or exceeds that elsewhere. “Our faculty are extraordinarily successful,” Garber said. “They are used to winning. And they don’t want to lose this game.”

At the same time, the Director of edX, Anant Argawal, admitted that there is certain learning sciences research that many faculty, including himself, had long ignored as they focused on their own disciplinary fields. “To me, these papers should be must-reads,” he said.

However, it is a pity they still haven’t discovered the mass of prior research on online learning, but at least there is a recognition here that MOOCs can and should improve their pedagogy.

Who benefits from online learning?

Listen with webReader

Di Xu & Shanna Smith Jaggars (2013) Adaptability to Online Learning: Differences Across Types of Students and Academic Subject Areas New York: Community College Research Center, Columbia University

The study

Using a dataset containing nearly 500,000 courses taken by over 40,000 community and technical college students in Washington State, this study examines how well students adapt to the online environment in terms of their ability to persist and earn strong grades in online courses relative to their ability to do so in face-to-face courses. 

The hypothesis

Some populations of students—for example, those with more extensive exposure to technology or those who have been taught skills in terms of time-management and self-directed learning—may adapt more readily to online learning than others. In addition, some academic subject areas may lend themselves to highquality online learning experiences more readily than others

The methodology

Primary analyses were performed on a dataset containing 51,017 degree-seeking students who initially enrolled in one of Washington State’s 34 community or technical colleges during the fall term of 2004. These first-time college students were tracked through the spring of 2009 for 19 quarters of enrollment, or approximately five years.  The dataset, provided by the Washington State Board of Community and Technical Colleges (SBCTC), included information on student demographics, institutions attended, and transcript data on course enrollments and performance.

The results

  • In descriptive terms, students’ average persistence rate across courses was 94.12 percent, with a noticeable gap between online courses (91.19 percent) and face-to-face courses (94.45 percent). For courses in which students persisted through to the end of the term (N = 469,287), the average grade was 2.95 (on a 4.0-point scale), also with a gap between online courses (2.77) and face-to-face courses (2.98).
  • While all types of students in the study suffered decrements in performance in online courses, some struggled more than others to adapt: males, younger students, Black students, and students with lower grade point averages. 
  • Regardless of a particular student’s own adaptability to the online environment, her performance in an online course may suffer if her classmates adapt poorly. English and social science were two academic subjects that seemed to attract a high proportion of less-adaptable students, thereby introducing negative peer effects.
  • Older students adapted more readily to online courses than did younger students.
  • [Also] students who were more disposed to take online course also tended to have stronger overall academic performance than their peers  



First, it is encouraging to see a detailed quantitative assessment of the types of students taking online courses, and their relative performance. This report needs to be read in full, and carefully. It is good that it is based on a significantly large enough sample that one can have confidence in the generalizability of the results (at least in the U.S. two-year college sector). The study was very well carried out and is a model for quantitative analysis of student differences.

Furthermore, I am not surprised or even concerned about these findings. For instance, from my own experience of online teaching, I would agree that ‘students are not homogeneous in their adaptability to the online delivery format and may therefore have substantially different outcomes for online learning.’ Online learning doesn’t suit everyone, and it is valuable to have some research that helps identify the more ‘at-risk’ online learners.

One can put forward a number of reasons why online students, on average, are likely to struggle compared with face-to-face students. Students who choose an online course are likely on average to have less time for study that those attending regularly on campus. Second, for many online students, the mode of study will be unfamiliar, which means making more adaptation to a different way of learning.

My one quibble is that, although the results are clearly significant statistically (as is almost inevitable in large samples), the differences are quite small (96% vs 91% completion rates, for instance.) Thus I do challenge the authors’ conclusion that ‘most students had difficulty adapting to the online context.’ If 91% complete the course, then most students did not have difficulties sufficient to deter them from completing successfully their courses. That seems a pretty good adaptation level to me.

I also have a concern that these results will be misinterpreted. This should not mean that men, Blacks and young people should be discouraged from taking online courses, but that we should be taking more care to ensure that students who do take online courses are better prepared, with particular attention being paid to those likely to be at most risk. This may mean, for instance, gradually introducing students to online learning in a deliberate way throughout a program. This study does suggest those most likely to be at risk.

Further reading

Lederman, D. (2013) Who benefits from online Ed? Inside Higher Education, February 25


No. 3 aha moment: asynchronous is (generally) better than synchronous teaching

Listen with webReader

The transmitter at Alexandra Palace, London, for the OU's TV and radio programs

In an earlier post, I listed the seven ‘aha’ moments that have been the most seminal ‘discoveries’ in my researching and working in educational technology. This is the third of seven posts that discuss why I believe these ‘discoveries’ to be important, and their implications specifically for online learning. The others to date are:

1.  Media are different.

2. God helps those who help themselves (about educational technology in developing countries).

What was the discovery? (1978)

Everyone learns better from media and technologies that allow them to study anywhere, at any time. In particular the ability to repeat and revise recorded material makes learning much more effective than live, synchronous teaching, for any learner who requires flexibility in accessing educational opportunities.

Which are synchronous and which are asynchronous technologies?

From Bates, A. (in press) Technology, e-Learning and the Knowledge Society, London: Routledge

From the table above, it can be seen that synchronous technologies include both one-way (broadcast) technologies, such as lectures, radio, broadcast television, and Webcasts, and two-way (interactive) technologies such as face-to-face seminars, audio-conferencing, video-conferencing, web conferencing, and virtual worlds. The unifying feature of synchronous technologies is that they take place in real time; thus both teachers and students have to be communicating together at the same time (but not necessarily in the same place.)

Asynchronous technologies include both one-way (broadcast) technologies such as print, audio-cassettes, podcasts, video-cassettes, lecture capture, web sites, DVDs, databases, web streaming including YouTube videos, and xMOOCs, and two-way (interactive) technologies such as written assignments, e-mail, online discussion forums, learning management systems, e-portfolios, blogs, search engines,cMOOCs, and other social media such as Facebook. Synchronous ‘content’ can be made available ‘asynchronously’ through recording.

How did this discovery come about?

To be honest, this insight really came from work by my colleagues at the Open University, Hans Grundin, Duncan Brown, Nicola Durbridge and Stephen Brown. As part of the Audio-Visual Media Research Group, we were tracking student participation in the television and radio broadcasts that accompanied the Open University courses. The latest technology in the early 1970s was the battery-operated radio cassette player (the Sony Walkman did not arrive until 1979). This allowed students to set a timer which would automatically record a radio program on to an audio cassette. The research indicated that increasingly students were recording the radio programs to listen to them later, but more importantly they were rating the cassettes as significantly more useful to their studies than the radio transmission.

There were many reasons for this:

  1. The OU radio programs were often transmitted at difficult times, such as 6.00 am or midnight.
  2. Students could stop, rewind and replay the cassettes.
  3. We found that students were working on the print materials on average roughly a week to ten days behind the recommended schedule. Thus the recorded version was more in synch with their actual study pattern than the broadcast.

As a result the university started up an audio-cassette library service, whereby students could order a cassette if they missed a program and have it mailed to them. Also the university started designing audio-cassettes that were not broadcast but accompanied the printed material that was the core of the studies. Instructors began taking advantage of the ‘affordances’ of the cassette technology, in several ways:

  1. Integrating the cassette very tightly with the printed material. For instance, John Mason, a mathematics instructor, used the audio cassette to talk students through equations and mathematical formulae in the printed text, very similar to the way Salman Khan talks student through a video version in the Khan Academy – but 40 years earlier
  2. Making use of the stop-start cassette facility to build in exercises and activities for students to do, with the feedback/answers later on the cassette tape. (Because you have to search ‘blind’ through an audio-cassette, it prevents students jumping straight to the answer.) For a full list of the ‘affordances’ of audio that were identified through the research of the AVMRG, see: Pedagogical roles for audio in online learning

In the end, the audio cassettes became so popular that by 1980 the BBC/OU almost entirely stopped broadcasting radio programs directly linked to course units .

When the video-cassette recorder arrived in the late 1970s, we found exactly the same pattern. The cassettes were rated more highly than the television broadcasts, and at one time the university was operating a system whereby more than 200,000 audio and video cassettes a year were being shipped out to students.

Why is this significant?

Because it suggests that asynchronous online learning is almost always better for learners requiring flexible learning than classroom teaching or ‘live’ broadcasts. In particular, despite the different ‘affordances’ of different media, there are some common advantages across all asynchronous technologies. In particular, students have greater control over asynchronous technologies, enabling them to fit their learning more easily into the rest of their lives, and also to repeat, and practice, until they can achieve mastery.

However,  there are circumstances where there are advantages in synchronous teaching. One obvious example is teaching oral language skills. Real-time communication in a foreign language is an important competency, so while recordings can help, students will need to practice in real time. There are circumstances where a live lecture or classroom can be more effective, for instance when trying to build a sense of community with a class, to provide an overview or summary of a whole course, or to provide inspiration or motivation to students.

Furthermore, as with all media, there are other variables which may have a large influence on effectiveness. For instance, a well-managed face-to-face seminar is likely to result in greater learning than a poorly managed online discussion forum; quality matters. Students looking for a campus-experience and direct social contact with other students are more likely to benefit from synchronous communication opportunities such as lectures and seminars.

But I woud argue that over a very broad range of circumstances, learners will on balance benefit more from asynchronous technologies, because of the extent to which they can control the pace and place of learning, and this is of particular significance for distance and/or lifelong learners.


This is probably one of the most controversial of my aha moments. There are many instructors for instance who believe very strongly in the advantages of real-time teaching, such as a lecture or seminar. Others swear by webinars (which can of course also be recorded).

Thus your comments on this will be particularly appreciated, particularly if you have research evidence to support your views.


There are 300 research reports from the AVMRG at the Open University. They are now difficult to access, but the Open University library has a complete set of papers, from 1 to 300, preserved within the University Archive. They are catalogued in the main Library catalogue http://voyager.open.ac.uk/index.html where they can be found by searching for a related topic or by searching for “AVMRG”. Visitors to the Library are welcome to access the reports within the University Archive.

Much of the research is summarized in the following books:

Bates, A. (1986) Broadcasting in Education: An Evaluation London: Constables

Bates, A. (2005) Open learning, e-Learning and Distance Education London/New York: Routledge.