September 22, 2017

A better ranking system for university teaching?

Who is top dog among UK universities?
Image: © Australian Dog Lover, 2017 http://www.australiandoglover.com/2017/04/dog-olympics-2017-newcastle-april-23.html

Redden, E. (2017) Britain Tries to Evaluate Teaching Quality Inside Higher Ed, June 22

This excellent article describes in detail a new three-tiered rating system of teaching quality at universities introduced by the U.K. government, as well as a thoughtful discussion. As I have a son and daughter-in-law teaching in a U.K. university and grandchildren either as students or potential students, I have more than an academic interest in this topic.

How are the rankings done?

Under the government’s Teaching Excellence Framework (TEF), universities in England and Wales will get one of three ‘awards’: gold, silver and bronze (apparently there are no other categories, such as tin, brass, iron or dross for those whose teaching really sucks). A total of 295 institutions opted to participate in the ratings.

Universities are compared on six quantitative metrics that cover:

  • retention rates
  • student satisfaction with teaching, assessment and academic support (from the National Student Survey)
  • rates of employment/post-graduate education six months after graduation.

However, awards are relative rather than absolute since they are matched against ‘benchmarks calculated to account for the demographic profile of their students and the mix of programs offered.’ 

This process generates a “hypothesis” of gold, silver or bronze, which a panel of assessors then tests against additional evidence submitted for consideration by the university (higher education institutions can make up to a 15-page submission to TEF assessors). Ultimately the decision of gold, silver or bronze is a human judgment, not the pure product of a mathematical formula.

What are the results?

Not what you might think. Although Oxford and Cambridge universities were awarded gold, so were some less prestigious universities such as the University of Loughborough, while some more prestigious universities received a bronze. So at least it provides an alternative ranking system to those that focus mainly on research and peer reputation.

What is the purpose of the rankings?

This is less clear. Ostensibly (i.e., according to the government) it is initially aimed at giving potential students a better way of knowing how universities stand with regard to teaching. However, knowing the Conservative government in the UK, it is much more likely to be used to link tuition fees to institutional performance, as part of the government’s free market approach to higher education. (The U.K. government allowed universities to set their own fees, on the assumption that the less prestigious universities would offer lower tuition fees, but guess what – they almost all opted for the highest level possible, and still were able to fill seats).

What are the pros and cons of this ranking?

For a more detailed discussion, see the article itself but here is my take on it.

Pros

First this is a more thoughtful approach to ranking than the other systems. It focuses on teaching (which will be many potential students’ initial interest in a university) and provides a useful counter-balance to the emphasis on research in other rankings.

Second it has a more sophisticated approach than just counting up scores on different criteria. It has an element of human judgement and an opportunity for universities to make their case about why they should be ranked highly. In other words it tries to tie institutional goals to teaching performance and tries to take into account the very large differences between universities in the U.K. in terms of student socio-economic background and curricula.

Third, it does provide a simple, understandable ‘award’ system of categorizing universities on their quality of teaching that students and their parents can at least understand.

Fourth, and most important of all, it sends a clear message to institutions that teaching matters. This may seem obvious, but for many universities – and especially faculty – the only thing that really matters is research. Whether though this form of ranking will be sufficient to get institutions to pay more than lip service to teaching remains to be seen.

Cons

However, there are a number of cons. First the national student union is against it, partly because it is heavily weighted by student satisfaction ratings based on the National Student Survey, which thousands of students have been boycotting (I’m not sure why). One would have thought that students in particular would value some accountability regarding the quality of teaching. But then, the NUS has bigger issues with the government, such as the appallingly high tuition fees (C$16,000 a year- the opposition party in parliament, Labour, has promised free tuition).

More importantly, there are the general arguments about university rankings that still apply to this one. They measure institutional performance not individual department or instructor performance, which can vary enormously within the same institution. If you want to study physics it doesn’t help if a university has an overall gold ranking but its physics department is crap or if you get the one instructor who shouldn’t be allowed in the building.

Also the actual quantitative measures are surrogates for actual teaching performance. No-one has observed the teaching to develop the rankings, except the students, and student rankings themselves, while one important measure, can also be highly misleading, based on instructor personality and the extent to which the instructor makes them work to get a good grade.

The real problem here is two-fold: first, the difficulty of assessing quality teaching in the first place: one man’s meat is another man’s poison. There is no general agreement, at least within an academic discipline, as to what counts as quality teaching (for instance, understanding, memory of facts, or skills of analysis – maybe all three are important but can how one teaches to develop these diverse attributes be assessed separately?).

The second problem is the lack of quality data on teaching performance – it just isn’t tracked directly. Since a student may take courses from up to 40 different instructors and from several different disciplines/departments in a bachelor’s program, it is no mean task to assess the collective effectiveness of their quality of teaching. So we are left with surrogates of quality, such as completion rates.

So is it a waste of time – or worse?

No, I don’t think so. People are going to be influenced by rankings, whatever. This particular ranking system may be flawed, but it is a lot better than the other rankings which are so much influenced by tradition and elitism. It could be used in ways that the data do not justify, such as justifying tuition fee increases or decreased government funding to institutions. It is though a first systematic attempt at a national level to assess quality in teaching, and with patience and care could be considerably improved. But most of all, it is an attempt to ensure accountability for the quality of teaching that takes account of the diversity of students and the different mandates of institutions. It may make both university administrations and individual faculty pay more attention to the importance of teaching well, and that is something we should all support.

So I give it a silver – a good try but there is definitely room for improvement. 

Thanks to Clayton Wright for drawing my attention to this.

Next up

I’m going to be travelling for the next three weeks so my opportunity to blog will be limited – but that has been the case for the last six months. My apologies – I promise to do better. However, a four hour layover at Pearson Airport does give me some time for blogging!

Book review: A History of the Open University

The image is of Robin Wilson, an OU math lecturer (and incidentally the son of Prime Minister Harold Wilson, who helped create the OU). The floral shirt was de rigueur in the early 1970s (I had one myself).

The image is of Robin Wilson, an OU math lecturer (and incidentally the son of Prime Minister Harold Wilson, who helped create the OU). The floral shirt was de rigueur in the early 1970s (I had a few myself).

Weinbren, D. (2105) The Open University: A History Manchester: Manchester University Press/The Open University, 274 pp + notes, £18.99, C$31.61, US$22.30 (paperback edition)

Why you should read this book

From the book cover:

This analysis of the Open University’s precedents, personalities, politics and pedagogies contextualises learners’ experiences and illuminates the change in the values of our society, our ideas about learning and our use of a variety of media.

Despite the florid writing in the publisher’s blurb, this is an accurate summary of the importance of this book, which should be read by anyone interested in open learning, distance learning, equality of access to higher education, changing pedagogies, the role of media in teaching and learning, the politics of creating radically new institutions of higher education, how higher education has changed in terms of value and purpose over the last 45 years, and, most important of all, how open learning can truly transform the lives of individuals.

What the book covers

The book is in four parts, which I will briefly summarise.

Part I: Creating a university of the air

This part covers the origins of the university within the socio-political context of Britain in the 1960s and early 1970s. It is a fascinating story in itself, of how a Labour Prime Minister, Harold Wilson, and Jennie Lee, his Under-Secretary of State in the Department of Education and Science, drove through their vision of a technocratic university for the masses, how the original vision was modified from a University of the Air to a multi-media university, and how the university survived a change of government which brought Margaret Thatcher and the Conservatives to power in the year that the OU opened. This is mandatory reading for policy wonks interested in how to bring about radical change in higher education.

Chapter 1 provides a broad overview of the challenges, developments and successes of the university over its 45 years existence. For those without the time to read the whole book, this chapter provides a useful encapsulation of the main points covered later.

Chapter 2 provides a more detailed account of the creation of the university, covering the political, socio-economic, pedagogical, and media components. It should be noted that many commentators believe that the Open University was ‘the most original innovation in 20th century British higher education’ and a ‘national treasure.’ This chapter helps to explain why.

In this part of the book, Weinbren captures well the social and political conditions, and above all the idealism and philosophies, that underpinned the creation and establishment of the Open University.

Part II: The first two decades

It is one thing to create a new institution; it is quite another to make it work. Indeed, the author notes that other attempts at innovation in higher education, such as the UK Open Polytechnic and the OU e-University, failed dismally after being created.

This part looks in detail at the governance and administrative structures, the role of academics, tutors and counsellors, the pedagogical models, the use of media, and the regional structure.

Weinbren points out that the intention from the start was to develop a degree-granting university with the highest possible academic standards:

Jennie Lee was adamant that the OU should be comparable to other universities in terms of its academic standards, rather than merely representing an educational second chance for the marginalised.

One reason for the OU’s relatively quick acceptance by the rest of the UK higher education sector was the high quality of the course materials which were used extensively by professors (and students) in the other universities. Another reason was the widespread engagement of academics from other universities as tutors or external examiners, who were often initially surprised by the quality of work produced by OU students.

Weinbren addresses particularly well the challenges the OU faced in terms of scale and the need for learner support for students working alone all over the country. (The OU started on day one in 1971 with 25,000 students and has grown since to 150,000 undergraduates and 30,000 postgraduates in 2014, with almost as many taking non-credit courses or modules.)

Weinbren, like the OU itself, struggles with integrating the competing pedagogical philosophies of behaviourist approaches to the design and development of high quality, mass-produced, course materials, and  learner-centred approaches based on face-to-face tutorials and summer schools. Overall, though, he emphasises that the fundamental pedagogical approach of the OU is focused on students developing personal meaning through interaction not only with course materials but also with faculty/tutors and other students, both face-to-face and later online. He describes with clarity how the often changing and complex learner support systems worked.

This chapter also explains why the Conservatives under Mrs. Thatcher continued to support the OU, despite concerns from some Ministers and the national press about ‘Marxist bias’ in its publicly available materials (especially the broadcasts). Thatcher had little time for traditional universities and saw in the OU a chance for reforming and galvanising the higher education system, especially with regard to improved cost-effectiveness.

This chapter also discusses the rapid development and changes in the use of media at the OU, with broadcasting over time playing a less direct role in teaching, greater use of recorded media such as cassettes, and the development of home experiment kits for science and technology courses.

Part III: The OU since the 1990s

This part is much influenced by the many changes since 1990 in the overall higher education system in Britain, and the consequent attempts by the OU at adaptation and accommodation to such changes. The key change has been the shift from direct government funding for teaching and learning to funding largely through student fees supported by loans (the OU now relies almost entirely on student fees), and the attempt by governments (both Labour and Conservative) to introduce more competition between universities and a more ‘market-oriented’ approach. This has resulted in the OU being treated as just any other university by government, rather than the special and separate treatment it received in earlier years.

Also over the past 45 years, the whole HE sector in the UK has expanded rapidly, making access at least theoretically more open to a much higher proportion of the population. Another important development has been the increased use of online learning by conventional universities. Together these have eroded some of the unique differences and advantages of the OU over the rest of the system.

The OU has responded to these changes in a number of ways, including:

  • expanding its international reach, especially but not exclusively in the rest of Europe;
  • the development of continuing education courses and modules;
  • more diversification regionally to respond to national political devolution;
  • contracts with non-commercial agencies, such as the National Health Service, as well as commercial organisations
  • leading the charge to quality assurance processes;
  • moving increasingly to online learning, and in the process, reducing dramatically the high-cost summer schools and face-to-face tutorial support; and integrating the role of counsellors with that of tutors;
  • increased use of learner-centred and project-based learning;
  • creation of open educational resources, such as FutureLearn and BBC/OU programs aimed at the general public.

Although Weinbren does a good job of covering the increasingly diverse and wide-ranging activities of the OU in the years from 1990 to the present, the OU’s unique role and place in the UK HE system becomes inevitably more fuzzy and its future direction less clear. However, the same criticism could apply to the whole of the UK HE sector, which seems to be increasingly forced back to a highly selective and tiered system, by government policies based on a more commercialised and employer-focused view of higher education. The OU’s place in such a system is by no means clear.

Part IV: Half a century of learning

This is a truly wonderful chapter about the student experience at the OU and lets students speak in their own words. This chapter helps explain why the OU is such an iconic component of British culture, and why it is so loved by students and staff alike (it consistently comes out top in student satisfaction in annual surveys of British universities). More importantly, this chapter clearly demonstrates how the OU has changed millions of students lives for the better.

Weinbren looks at several aspects of the student experience. While the OU has a very broad mix of incomes and occupations, it has opened up higher education particularly to working class families, students with disabilities, prisoners, those without high school qualifications, and above all to women. In this sense it is a truly open university, offering not just opportunities but also qualifications and realistic chances of success for everyone.

Weinbren illustrates how important the OU has been to women, particularly in the early year of the OUs, in terms of personal development and increased self-esteem. The importance of summer schools for engaging students and making them feel part of a university community is particularly well described. I also read with great interest how the OU enabled both Republican and Loyalist prisoners in Northern Ireland ‘to develop political confidence and an understanding of methods other than violence’, some going on to become politicians following the Good Friday Agreement of 1998.

Above all, this chapter provides an excellent antidote to the current market-oriented and functional philosophy of higher education now so prevalent in Britain and elsewhere. If you can read only one chapter, this is the one.

Personal reflections on the book

The OU was a very important part of my life for 20 years. I was one of the first staff appointed in 1969, and I ended up doing at one time or another research and evaluation into the educational effectiveness of different media (including the BBC television and radio programs), designing and writing course units, marking student assignments, directing summer schools, and attending endless meetings about policies, directions and the use of media. I left the OU in 1989, partly because I was frustrated that it wasn’t changing fast enough, particularly with regard to the use of online learning. It’s hard for me then to be objective in reviewing Weinbren’s book and even more so in assessing the contribution of the OU to higher education.

Nevertheless, the book captures wonderfully my lived experience of the OU, especially the student response. Weibren has blended together an impeccable range of resources, anecdotes, events and above all personal contributions from academics, staff and students at the OU into a well-written, captivating chronicle that reflects the spirit as well as the history of the OU.

There are criticisms, of course. At times, it becomes a hagiography of an institution (if that’s possible). Weinbren does describe the many criticisms of the OU, but always provides a contradictory positive contribution to offset each criticism. In particular, he could have been harsher about the OU’s increased bureaucracy and sclerosis as it has become older. True, there have been many innovations, for instance, in the use of technology, but changing its cumbersome and now outdated course development system has proved to be extremely difficult. Although it was one of the first institutions to adopt online learning, it has been a real struggle to make it a central rather than a peripheral part of the teaching system.

More importantly, Weinbren does not look into the future, yet there are surely lessons for the future from his book. The OU is facing an almost existential crisis, with many competitors, a very difficult financial situation, and massive changes and innovation going on elsewhere in the UK higher education system. What is the role of the OU in the 21st century? In what ways can it continue to provide a unique and valuable contribution? What teaching model will best meet the needs of its students in the 21st century? This is probably another book altogether, but Weinbren is particularly well placed to ask and address these issues. As I say to Ph.D. students, the conclusion is your chance to let rip and say what you really think now you have established your credentials. It’s a pity that Weinbren did not take this opportunity, but he has probably other means to let his views be known.

These though are minor caveats. Weinbren has undertaken an extremely challenging task and met the challenge superbly. I hope you will enjoy the book as much as I have. More importantly, there are very important lessons to be drawn from this book about the nature of university education, equity, and government policy toward higher education.

Online learning in 2012: a retrospective

© The Greening of Gavin, 2012

Well, 2012 was certainly the year of the MOOC. Audrey Watters provides a comprehensive overview of what happened with MOOCs in 2012, so I won’t repeat what she has done. Instead in this post I will focus mainly on trying to explain with regards to MOOCs what appears to me to be highly irrational organizational behaviour, more akin to lemmings than pillars of higher learning.

Why MOOCs?

For those of us who work mainly in universities and colleges, the hype around MOOCs is like living in two parallel universes: what we do every day in online learning, and what we read or hear about in the media. (I leave you to judge which is the true reality.) Even organizations that should know better think that online learning started at MIT in 2002 with OpenCourseWare. So why have MOOCs in particular got so much press?

This is an exercise in social anthropology.

To quote from Wikipedia:

It is unknown why lemming populations fluctuate with such variance roughly every four years, before plummeting to near extinction.

Now some evidence suggests their predators’ populations, particularly the stoat, may be more closely involved in changing the lemming population

Lemmings can swim and may choose to cross a body of water in search of a new habitat. In such cases, many may drown if the body of water is so wide as to stretch their physical capability to the limit.

 I believe there are several themes that have led to MOOC hysteria in 2012:

  • they appear to be free. The direct costs of higher education, especially but not only in the USA and the UK, have been systematically transferred from the tax payer to the individual student or parents through cuts in government funding and increases in tuition fees. In other words, the cost of higher education has become more transparent. It’s really expensive. Free of course is better than expensive. MOOCs have been promoted as being free. However, there are no free services. All services have a true cost. At least to date, MOOCs are the opposite of transparency on the true cost. We do know that over a hundred million dollars have been invested this year alone in MOOCs, but what are the costs of the professors’ time, the cost of managing large numbers of students, and above all, the cost of ensuring student learning (however it is measured)? We just don’t know. Until we do, it’s a shell game
  • it’s also a numbers game: input matters more than output. The focus of the media has been on the massive numbers enrolling. However, there has been little focus on what students are actually learning. All we know is that completion rates are pathetic (less than 10%), and many of those that do complete are already well educated. Nevertheless it is argued that on a global perspective, the completion numbers are still large. However, so are the numbers in traditional higher education, and also in credit-based online learning. Sloan and Babson have been tracking the online credit numbers for years. They have been growing at a steady rate of between 12-20% a year. Ontario alone has over 500,000 online course registrations in its public universities and colleges, with completion rates in the 75-85%, matching completion rates in face-to-face classes. Millions are taking online courses for credit in Asia. But does this get mass coverage in the media? No.
  • technology triumphs over teaching: MOOCs in general have been driven by computer scientists who believe that just ‘delivering’ content over the Internet equates to learning. It doesn’t, but broadcast content delivery is something that lazy reporters can easily understand.
  • it’s all about the elite institutions. The media love to focus on the ivy league universities to the almost total neglect of the rest of the system (the cult of the superstar). Here is an appalling irony. The top tier research universities have by and large ignored online learning for the last 15 years. Suddenly though when MIT, Stanford and Harvard jump in, all the rest follow like lemmings. MOOCs are seen as an easy, low risk way for these universities not only to catch up, but to jump into the front line. But they are hugely wrong. Moving from broadcasting to learning is not going to be easy. More importantly, MOOCs are a side issue, a distraction. The real change for universities is going to come from hybrid learning – a mix of on-campus and online learning. Those top tier research universities though are going to miss out on this, by sidelining their online learning to a peripheral, continuing education activity.
  • don’t forget the politics: There’s just been a presidential election in the USA. A number of corporate leaders and some in the Republican party want to privatize the US higher education system. Anything that will undermine it is heavily promoted. MOOCs to some extent have been a tool in the hands of the media for suggesting that education need not be expensive and could be ‘free’, or at least much lower cost, if left to business. This fits the agenda of the right.

Having said all this, I believe that there is a future for MOOCs, but that’s for another post, my outlook for 2013, which comes in January.

In the meantime, there were, believe it or not, several other interesting developments in online learning, but before exploring those as well, let’s see how right I was in my outlook for 2012.

What I predicted

  1. The year of the tablet: 99% probability
  2. Learning analytics: 90% probability
  3. Growth of open education: 70% probability (depending on definition of open education)
  4. Disruption of the LMS market: 60% probability
  5. Integration of social media into formal learning: 66% probability
  6. The digital university: 10% probability
  7. Watch India
  8. The great unknown: 10% probability

Well, not a great record at prediction. I suppose you could include MOOCs within ‘growth of open education’. But look at what I actually wrote:

open access to high quality (all right, highly qualified) instructors is likely to be limited to idealistic volunteers, or to limited events (e.g. a MOOC), mainly because of a mis-match between supply and demand. Too many people want access to what they may incorrectly assume to be high quality instructors at elite institutions, for instance. This is partly an institutional barrier, as institutions try to protect their ‘star’ faculty, which is why this form of openness depends largely on individual volunteers.

Not actually wrong, but it certainly didn’t capture the mania that would develop around MOOCs in 2012.

Although there have been lots of interesting individual uses of tablets, particularly in k-12, they certainly haven’t taken off to the extent to which I predicted, at least in post-secondary education. However, so much in prediction depends on timing – maybe it will happen this year. For instance, mobile learning, one of my predictions for 2011, certainly expanded in many institutions in 2012, and will certainly continue to grow in 2013. The use of data analytics definitely increased, but still in a minority of institutions, in 2012, but learning analytics are still being used by a very tiny minority. The technology isn’t quite ready yet. (Again, this depends on definition – I’m talking about the hope that learning analytics will help instructors to achieve better learning outcomes, or put another way, will help students to improve their learning.)

What you read

Another way at looking at 2012 is to see what you chose to read. There are just over 1,800 posts on the site. Here are the top 14 posts in 2012, with the number of hits. (If you missed one, just click on it.)

Recommended graduate programs in e-learning

15,685

What’s right and what’s wrong about Coursera-style MOOCs

7,089

e-learning outlook for 2012: will it be a rough ride?

6,827

New technologies for e-learning in 2012 (and a little beyond)

6,658

A short critique of the Khan Academy

5,026

Can you teach ‘real’ engineering at a distance?

4,988

What Is Distance Education?

4,083

Why learning management systems are not going away

3,624

E-learning quality assurance standards, organizations and research

3,221

A personal view of e-learning in Saudi Arabia

2,844

A student guide to studying online

2,513

10 types of plagiarism (and why I’m pleading guilty to at least one charge)

2,353

Daniel’s comprehensive review of MOOC developments

2,264

Designing online learning for the 21st century

1,929

The numbers of course are skewed by their date of  posting. Those posted early in the year have more chance of being accessed than those posted later. Timing also matters in terms of external events. Despite all the hype about MOOCs, only two of the top 14 posts were specifically on MOOCs (although there were several others posted). I am though surprised at the amount of interest in prediction, especially given how bad I am at it!

The inclusion of ‘Can you teach real engineering at a distance?’ at no. 6 is really interesting. This was posted originally on July 5, 2009, but it has sustained a long discussion that is still active today. I was also pleased to see that designing online learning for the 21st century squeezed in, as this was about design of online learning. I’m glad there’s still at least some interest in this issue. There is also evidence that the site is being used by  a lot of online students (or potential students), which is very gratifying. I need to do more posts targeted to students next year.

What I did

Since I’m not free and open (except here), this is some indication of what institutions were interested in this year (at least enough to pay me for it).

Site visits for consultancies or discussions with faculty/staff on strategies or designs for online learning

  • Mexico City: to develop a business plan for a national Mexican virtual university
  • Edmonton: Campus St-Jean, University of Alberta: informal review of online learning activities
  • Université de Sherbrooke, l’université Laval and Université de Québec en Abitibi-Témiscamingue, Québec
  • Vancouver Community College, Kwantlen Polytechnic University, and University of British Columbia, BC
  • University of Manitoba, Winnipeg
  • EFQUEL conference, Granada, Spain
  • COHERE conference, Calgary, Alberta

Online consultancies

MOOCs and Webinars

  • planning and managing online learning: participant in #Change 11 cMOOC
  • costs of online learning: guest instructor for University of Maryland University College/University of Oldenberg, Germany
  • Elections Canada: online course design

Institutional site visits and reports on gamechanging institutions

  • Western Governors University
  • Open University, UK
  • Open University of Catalonia, Spain
  • London Knowledge lab, Institute of Education, London, UK.

It can be seen there was a great deal of interest in:

  • strategies and management,
  • new course designs,
  • design and organization of online institutions,
  • the costs of online learning

during 2012. These issues are not likely to disappear next year, either.

Politics and economics

In 2012, there were major developments in both the politics and economics of online learning. Governments in the USA and Europe accelerated cost cutting in post-secondary education. Nearly one billion dollars has been cut from the community college system in California alone since 2008. Student tuition fees have risen dramatically over the last five years in both the USA and the U.K. Even in Canada, provincial governments are facing the need to constrain public funding.

In Ontario, Canada’s largest province, the government threw down a challenge to the post-secondary institutions. Enrollments will need to increase, quality must be obtained, but there will be no new money. What can the institutions do to increase productivity through innovation? It’s a good question. Business cannot go on as usual. There is surely room for improvement and change in our institutions.

This theme is likely to continue into 2013. Governments, parents and increasingly students will be looking to online learning to increase productivity: better learning outcomes for less money. Are we up to the challenge?

Goodbye, 2012

I asked the question last year: will it be a rough ride? It’s certainly been a fast ride and quite bumpy at the same time. I don’t know how you feel, but I feel I’m hanging on, but only just. It’s good though that it’s exciting, stimulating, infuriating, and frustrating. It means that online learning is alive and well, growing in both breadth and more importantly depth.

So to all my readers, thank you for coming along for the ride. Have a great break, merry Christmas, happy Hanukah, or just have a good time, whatever your religion or beliefs. And I look forward to sharing my outlook for 2013 in the new year.

Questions

1. What pleased, surprised or disappointed you in 2012 with regard to online learning?

2. What do you think was the most important development in 2012 for online learning? Obama’s re-election? MOOCs? New course designs? Or something else?

3. Are we up to the challenge of using online learning to increase productivity through innovation? If so, what would that look like?

European open universities feeling the stress of austerity

Walton Hall, which houses the office of the Vice-Chancellor, the UK Open University

A trip down Memory Lane

I’ve just returned from a very interesting week in Europe, accompanied by my colleague, Maxim Jean-Louis, the President of Contact North. My philosophy in life is never look back. So my visit to the Open University in England was the first time I had been there in 23 years, since I emigrated to Canada. Similarly, I had worked at the Open University of Catalonia (Universitat Oberta de Catalunya – or UOC) as a part-time research professor between 2003-2007, and again I hadn’t been back for five years. The third place I visited was the London Knowledge Lab at the Institute of Education, University of London, where I was a post-graduate student so long ago that I typed my thesis on an IBM golf-ball typewriter (which is about all I can remember of the experience).

First, amazingly, there were still very good friends and colleagues working at all these institutions, and I was somewhat discombobulated by the very warm welcome I received from them. MOOCs may come and go, but good friends are for life. Second, the dynamism that made all these places fun to work at is still there.

Why I was there

I will be writing detailed reports on each of these visits, but the purpose of my visit from my perspective was to look at the research being done on online learning in these three institutions, and there was great work being done in each place. Maxim’s interest was to identify the real ‘game-changers’ in online learning. More to come on this in later posts.

Money is the root of all evil

However, transcending both purposes is the current financial context in both the UK and Spain. These countries are undergoing profound economic stress and the two open universities are not able to avoid the consequences, although the challenges differ according to the context.

In particular, the UK Open University, which has been and continues to be immensely successful, with over 180,000 credit students, is suffering the law of unintended consequences. The UK government, in its wisdom, has completely changed the system of post-secondary funding. In effect, it has stopped giving universities funding to support teaching. Instead, universities now have to depend entirely on student tuition fees at around $15,000 per student per year. To cover the costs, students can take out low interest loans which are underwritten by the government. However, to qualify for such loans, students must take a full time ‘load’ of courses.

This financial strategy strikes at the very heart of the Open University. It has been forced to move from essentially a ‘free’ university to one where students are now paying around $7,500 a year in tuition fees, which is still less than half the going rate at most other universities in England and Wales. However, most of its students are working adults, and are not taking a full load of courses. Indeed, many of them are somewhat like MOOC students, taking individual courses or unique combination of courses that fit their particular needs as adults. These however do not lead to the ‘recognized qualifications’ that makes them eligible for a student loan. Thus the change in financing is devastating for many low-income learners wanting to study through the Open University, as well as for the university. As its Vice-Chancellor stated to us, if you want to look at a successful model for financing higher education, don’t look at Britain. Maybe Canada’s Mark Carney, the new Governor of the Bank of England, can help!

Meeting the challenge

Not surprisingly, the OU is looking at a range of strategies, including a strong move to internationalizing its offerings, to ensure its survival, and indeed it has seen a change in its student demographics, with younger students who under previous conditions would be applying to conventional institutions instead opting for the lower cost but still high quality and more flexible programs offered by the Open University. So you win some and you lose some. Nevertheless, I sincerely hope that the current UK government’s financial strategy for higher education is a temporary aberration – isn’t that what elections are for in a democracy?

Watch this space

The next posts on these institutions will focus on the positive, the very interesting work they are doing in research in online learning.

Book review: Quality assurance in distance education and e-learning

© Insights, 2012

Jung, I. and Latchem, C. (2012) Quality Assurance and Accreditation in Distance Education and e-Learning New York/London: Routledge

What the book is about

There is relatively little in the literature about QA and accreditation in distance education and online schooling, college or university education, non-formal adult and community education or workplace training. Drawing on international theory, research and the experience and expertise of the contributors, this book shows why and how these are applied across the globe, considers the lessons learned, and suggests frameworks and guidelines for their implementation.’

What’s in it

The book is essentially a collection of 23 chapters by invited contributors, with a foreword from Sir John Daniel, President of the Commonwealth of Learning. The chapters fall into roughly three groups:

  • three chapters that provide an overview of the main concepts and premises behind quality assurance and accreditation
  • seventeen chapters on quality assurance in distance education and e-learning in different regions or countries around the world, or in specific sectors, as follows:
    • Asia
    • sub-Saharan Africa
    • USA and Canada
    • United Kingdom
    • Europe
    • Australia and New Zealand (including a separate chapter on Open Universities Australia)
    • open universities
    • Indonesian Open University (Universitas Terbuka)
    • University of the South Pacific and the University of the West Indies
    • Palestinian Al-Quds University
    • Commonwealth of Learning
    • open and online schooling
    • workplace training
    • telecentres
  • three concluding chapters, one on competencies and quality assurance, one on learners’ perceptions, and a concluding chapter.

Review

Overall

This book provides comprehensive coverage of the practice and applications of quality assurance in distance education and some elements of e-learning around the world. The articles are in general well written and authoritative. The book is comprehensive in the sense that it covers the main issues and ways in which quality assurance has been applied, particularly in distance education. It will be of particular value to those working in particular areas of the world where distance education is still not accepted, and there are still many countries where this is the case.

What I liked

I very much liked the foreword by John Daniel, who provides an extremely succinct overview of the book and the issues arising.

This book really brings home the struggle that distance education and online learning still have to prove their legitimacy, despite the sometimes extraordinary lengths that they have gone to demonstrate their quality. Frequently, the issue of double standards arise, whereby distance and online programs have to meet standards or criteria that are never applied to campus-based programs.

The book also brings home how widespread the QA movement is in distance education, and how many countries and regions are struggling with similar issues.

I also enjoyed reading many of the individual chapters on what’s happening in countries and regions as diverse as Indonesia, Korea, the European Union, Australia, Asia and North America. It reinforced particularly the peculiar mess that the USA has got itself into with regard to accreditation and quality assurance. Although a better system of accreditation would surely help, I doubt that greater attention to QA processes is the answer to their growing problems in post-secondary education.

Lastly, I particularly liked Insung Jung and Colin Latchem’s concluding chapter, which pulled together a number of important QA issues which are worth summarizing:

  • focus on outcomes as the leading measure of quality
  • take a systemic approach to quality assurance
  • see QA as a process of continuous improvement
  • move the institution from external controls to an internal culture of quality
  • poor quality has very high costs so investment in quality is worthwhile.

What I didn’t like

I said earlier that the book covers ‘some elements of e-learning’ because the book does not touch on the greatest area of application of e-learning, which is in the traditional campus-based universities and two year colleges. It is a pity that this issue has not been addressed. First, besides being a huge area, the issues around quality assurance are somewhat different from those of open and distance learning institutions, or even the dual-mode institutions, because most of the campus-based institutions moving for the first time into e-learning are already accredited as campus-based institutions. Some traditional institutions develop very high quality e-learning, while others (particularly for example in the US college system) fail to follow best practices and as a result often deliver very poor quality e-learning. More than any other, it is the traditional campus-based institutions that have moved into e-learning that need to put in place policies and procedures that assure quality in online learning, yet there are no examples of what is being done is in this sector in this book. However, Insung Jung and Colin Latchem’s chapter on competencies and quality assurance should be required reading in such institutions.

It could of course be argued that these institutions anyway follow standard QA processes for all their programs, but the standard degree quality assurance process does not adequately cover the specifics of e-learning or online learning on campus-based institutions, although some quality assurance boards (such as the QAA in the UK and PEQAB in Ontario) have put in place specific ‘benchmarks’ for online courses being offered in certain sectors (but not all). As more and more already accredited (and ‘high quality’) campus-based institutions start moving into hybrid learning, the establishment of quality in the e-learning elements of programs will become even more important. However, this is probably another book, and should not diminish the value of this book.

Some general points

Despite this book, I still have very mixed feelings about the quality assurance movement in post-secondary education. Yes, there are enormous differences in the quality of online and distance learning providers, and some of the more elite institutions are the worst culprits in terms of a quality online or digital learning experience. But ensuring quality in online learning is not rocket science. There is plenty of evidence of what works and what doesn’t, such as regular and challenging interaction between instructor and students, efficient student administration, interactive and well structured materials, etc. We don’t need to build a bureaucracy around this, but there does need to be some mechanism, some way of calling institutions when they fail to meet these standards. However, we should also do the same for campus-based teaching.

Also, QA when applied rigidly can stifle innovation in online teaching and learning. ‘Best practice’ may need occasionally to be challenged, so we can experiment with and evaluate new approaches.

But my main reason for harbouring doubts about the QA movement is that QA doesn’t really address the need for more differentiation in our educational system. Distance education organizations are not the same as elite traditional universities and shouldn’t try to be. This means that different types of institution will and should evaluate quality differently. Of course, this also requires greater sophistication amongst potential learners and even more so, governments.

There is a famous put-down in an Oscar Wilde play when an aristocratic lady comments on a bourgeois woman’s attempt at ‘class’: ‘She tries so hard, doesn’t she?’ QA doesn’t really deal with the issues of elitism. Yes, Oxford University has QA procedures that meet the external QAA requirements, but does anyone believe that this is the reason why it has its cachet of ‘quality’? Sometimes I think that distance education institutions in particular try too hard to establish their quality credentials, when what they should be emphasising is their differences, and especially the different purposes they exist to meet. I really would look for different measures of quality in the Open University, for instance, than in Cambridge University. Neither one is necessarily better (depending on what you are looking for), but the learning experience will be different. In the meantime, let’s pay much more attention to what campus-based institutions are doing when they move to online learning. Are they following best practices, or even better, innovating and evaluating?

Having got that off my chest, let’s come back to the book. This is an important book, especially for distance educators, but will also be useful for administrators from conventional institutions that are moving into hybrid and fully online learning. It provides the current state of the art on quality assurance in this field and as such I highly recommend it.