November 20, 2017

A better ranking system for university teaching?

Who is top dog among UK universities?
Image: © Australian Dog Lover, 2017 http://www.australiandoglover.com/2017/04/dog-olympics-2017-newcastle-april-23.html

Redden, E. (2017) Britain Tries to Evaluate Teaching Quality Inside Higher Ed, June 22

This excellent article describes in detail a new three-tiered rating system of teaching quality at universities introduced by the U.K. government, as well as a thoughtful discussion. As I have a son and daughter-in-law teaching in a U.K. university and grandchildren either as students or potential students, I have more than an academic interest in this topic.

How are the rankings done?

Under the government’s Teaching Excellence Framework (TEF), universities in England and Wales will get one of three ‘awards’: gold, silver and bronze (apparently there are no other categories, such as tin, brass, iron or dross for those whose teaching really sucks). A total of 295 institutions opted to participate in the ratings.

Universities are compared on six quantitative metrics that cover:

  • retention rates
  • student satisfaction with teaching, assessment and academic support (from the National Student Survey)
  • rates of employment/post-graduate education six months after graduation.

However, awards are relative rather than absolute since they are matched against ‘benchmarks calculated to account for the demographic profile of their students and the mix of programs offered.’ 

This process generates a “hypothesis” of gold, silver or bronze, which a panel of assessors then tests against additional evidence submitted for consideration by the university (higher education institutions can make up to a 15-page submission to TEF assessors). Ultimately the decision of gold, silver or bronze is a human judgment, not the pure product of a mathematical formula.

What are the results?

Not what you might think. Although Oxford and Cambridge universities were awarded gold, so were some less prestigious universities such as the University of Loughborough, while some more prestigious universities received a bronze. So at least it provides an alternative ranking system to those that focus mainly on research and peer reputation.

What is the purpose of the rankings?

This is less clear. Ostensibly (i.e., according to the government) it is initially aimed at giving potential students a better way of knowing how universities stand with regard to teaching. However, knowing the Conservative government in the UK, it is much more likely to be used to link tuition fees to institutional performance, as part of the government’s free market approach to higher education. (The U.K. government allowed universities to set their own fees, on the assumption that the less prestigious universities would offer lower tuition fees, but guess what – they almost all opted for the highest level possible, and still were able to fill seats).

What are the pros and cons of this ranking?

For a more detailed discussion, see the article itself but here is my take on it.

Pros

First this is a more thoughtful approach to ranking than the other systems. It focuses on teaching (which will be many potential students’ initial interest in a university) and provides a useful counter-balance to the emphasis on research in other rankings.

Second it has a more sophisticated approach than just counting up scores on different criteria. It has an element of human judgement and an opportunity for universities to make their case about why they should be ranked highly. In other words it tries to tie institutional goals to teaching performance and tries to take into account the very large differences between universities in the U.K. in terms of student socio-economic background and curricula.

Third, it does provide a simple, understandable ‘award’ system of categorizing universities on their quality of teaching that students and their parents can at least understand.

Fourth, and most important of all, it sends a clear message to institutions that teaching matters. This may seem obvious, but for many universities – and especially faculty – the only thing that really matters is research. Whether though this form of ranking will be sufficient to get institutions to pay more than lip service to teaching remains to be seen.

Cons

However, there are a number of cons. First the national student union is against it, partly because it is heavily weighted by student satisfaction ratings based on the National Student Survey, which thousands of students have been boycotting (I’m not sure why). One would have thought that students in particular would value some accountability regarding the quality of teaching. But then, the NUS has bigger issues with the government, such as the appallingly high tuition fees (C$16,000 a year- the opposition party in parliament, Labour, has promised free tuition).

More importantly, there are the general arguments about university rankings that still apply to this one. They measure institutional performance not individual department or instructor performance, which can vary enormously within the same institution. If you want to study physics it doesn’t help if a university has an overall gold ranking but its physics department is crap or if you get the one instructor who shouldn’t be allowed in the building.

Also the actual quantitative measures are surrogates for actual teaching performance. No-one has observed the teaching to develop the rankings, except the students, and student rankings themselves, while one important measure, can also be highly misleading, based on instructor personality and the extent to which the instructor makes them work to get a good grade.

The real problem here is two-fold: first, the difficulty of assessing quality teaching in the first place: one man’s meat is another man’s poison. There is no general agreement, at least within an academic discipline, as to what counts as quality teaching (for instance, understanding, memory of facts, or skills of analysis – maybe all three are important but can how one teaches to develop these diverse attributes be assessed separately?).

The second problem is the lack of quality data on teaching performance – it just isn’t tracked directly. Since a student may take courses from up to 40 different instructors and from several different disciplines/departments in a bachelor’s program, it is no mean task to assess the collective effectiveness of their quality of teaching. So we are left with surrogates of quality, such as completion rates.

So is it a waste of time – or worse?

No, I don’t think so. People are going to be influenced by rankings, whatever. This particular ranking system may be flawed, but it is a lot better than the other rankings which are so much influenced by tradition and elitism. It could be used in ways that the data do not justify, such as justifying tuition fee increases or decreased government funding to institutions. It is though a first systematic attempt at a national level to assess quality in teaching, and with patience and care could be considerably improved. But most of all, it is an attempt to ensure accountability for the quality of teaching that takes account of the diversity of students and the different mandates of institutions. It may make both university administrations and individual faculty pay more attention to the importance of teaching well, and that is something we should all support.

So I give it a silver – a good try but there is definitely room for improvement. 

Thanks to Clayton Wright for drawing my attention to this.

Next up

I’m going to be travelling for the next three weeks so my opportunity to blog will be limited – but that has been the case for the last six months. My apologies – I promise to do better. However, a four hour layover at Pearson Airport does give me some time for blogging!

More webinars on ‘Teaching in a Digital Age’

Linda Harasim’s pedagogy of group discussion (from Harasim, 2012): a topic for discussion in webinar 1?

Linda Harasim’s pedagogy of group discussion (from Harasim, 2012): a topic for discussion in webinar 1?

Last year I did a series of five webinars on topics from my online, open textbook, ‘Teaching in a Digital Age.’ These proved to be very popular, with up to 200 requests for participation for each webinar. We limited registrants though to a maximum of 60 for each webinar, and there have been more than 20,000 downloads since the first webinars were offered, so I am grateful to Contact North for offering a second round of these webinars.

The topics I will be covering in these webinars, which as well as being live will also be available in recorded form, will be:

  1. Teaching with Technology – How to Use the Best Practice Models and Options (covers chapters 1, 2, 3, 4 and 5 of Teaching in a Digital  Age)
  2. Choosing Media – How They Differ and How to Make the Best Choices for My Teaching (covers chapters 6, 7 and 8 of Teaching in a Digital Age)
  3. Making the Choice – How to Choose between Online, Blended or Campus-Based Delivery for Effective Learning (covers chapters 9 of Teaching in a Digital Age)
  4. Ensuring Quality – How to Design and Deliver Quality Courses in a Supportive Learning Environment (covers chapter 11 and Appendix 1 of Teaching in a Digital Age)
  5. How Open Education will Revolutionize Higher Education: the Impact of Open Research, Open Textbooks, OERs and Open Data on Course Design and Delivery (covers Chapter 10 of Teaching in a Digital Age)

Register today for the first 60-minute webinar on Teaching with Technology – How to Use Best Practice Models and Options on Tuesday, October 18, 2016, at 1:00 p.m. Eastern. 

Webinar 1: Teaching with Technology – How to Use Best Practice Models and Options

In this webinar, we will discuss:

  • what kind of knowledge or skills students need in a digital age;
  • what kind of learning theories or pedagogy will best suit your subject area or preferred teaching style
  • what teaching approaches are most appropriate for a digital age.

The webinar features a short introduction to the topics, and I will be posing a series of questions for discussion amongst the webinar participants on the topic and an open Q&A. You are advised to read the first five chapters of the book in advance of the webinar, as I will not be able to do justice to each of the topics in a short introduction.

Registration is limited to keep the session interactive so register early to avoid disappointment.

The remaining four webinars will be held in November, December, January and February at the same times. Watch this space for more details nearer the dates.

French version of ‘Teaching in a Digital Age’ now available

French version 2

The French version of ‘Teaching in a Digital Age’, L’enseignement a l’ère numerique‘, is now available from here.

I am very grateful to Contact North|Contact Nord for providing this professional translation.

There is now also a version in Vietnamese, ‘Dạy học trong kỷ nguyên số‘, translated by Lê Trung Nghĩa of the Ministry of Education in Vietnam, available through Dropbox here.

Spanish version, translated by staff in the Faculty of Engineering, Universidad de Buenos Aires, is almost complete and will be available from the BCcampus open textbook site (as will all the translations). I will provide an announcement containing the url when it is available.

A Chinese version, translated by staff at the Beijing Open University, will be available in August, 2016.

A Portuguese version, being translated by ABED, the Brazilian Association of Distance Education, will be available in time for its Annual Congress in September, 2016.

Turkish version is currently under consideration. I am awaiting more details.

Please note: under the Creative Commons license of the book, anyone is free to translate all or any part of the book, provided it is not used for commercial purposes and I am acknowledged as the author. I am sure that without this license, the book would not have become available so quickly in so many languages. However, if you do decide to translate the book, please let me know, so I can track its use and provide updates.

 

Automation or empowerment: online learning at the crossroads

Image: Applift

Image: AppLift, 2015

You are probably, like me, getting tired of the different predictions for 2016. So I’m not going to do my usual look forward for the year for individual developments in online learning. Instead, I want to raise a fundamental question about which direction online learning should be heading in the future, because the next year could turn out to be very significant in determining the future of online learning.

The key question we face is whether online learning should aim to replace teachers and instructors through automation, or whether technology should be used to empower not only teachers but also learners. Of course, the answer will always be a mix of both, but getting the balance right is critical.

An old but increasingly important question

This question, automation or human empowerment, is not new. It was raised by B.F. Skinner (1968) when he developed teaching machines in the early 1960s. He thought teaching machines would eventually replace teachers. On the other hand, Seymour Papert (1980) wanted computing to empower learners, not to teach them directly. In the early 1980s Papert got children to write computer code to improve the way they think and to solve problems. Papert was strongly influenced by Jean Piaget’s theory of cognitive development, and in particular that children constructed rather than absorbed knowledge.

In the 1980s, as personal computers became more common, computer-assisted learning (CAL or CAD) became popular, using computer-marked tests and early forms of adaptive learning. Also in the 1980s the first developments in artificial intelligence were applied, in the form of intelligent math tutoring. Great predictions were made then, as now, about the potential of AI to replace teachers.

Then along came the Internet. Following my first introduction to the Internet in a friend’s basement in Vancouver, I published an article in the first edition of the Journal of Distance Education, entitled ‘Computer-assisted learning or communications: which way for IT in distance education?’ (1986). In this paper I argued that the real value of the Internet and computing was to enable asynchronous interaction and communication between teacher and learners, and between learners themselves, rather than as teaching machines. This push towards a more constructivist approach to the use of computing in education was encapsulated in Mason and Kaye’s book, Mindweave (1989). Linda Harasim has since argued that online collaborative learning is an important theory of learning in its own right (Harasim, 2012).

In the 1990s, David Noble of York University attacked online learning in particular for turning universities into ‘Digital Diploma Mills’:

‘universities are not only undergoing a technological transformation. Beneath that change, and camouflaged by it, lies another: the commercialization of higher education.’

Noble (1998) argued that

‘high technology, at these universities, is often used not to ……improve teaching and research, but to replace the visions and voices of less-prestigious faculty with the second-hand and reified product of academic “superstars”.

However, contrary to Noble’s warnings, for fifteen years most university online courses followed more the route of interaction and communication between teachers and students than computer-assisted learning or video lectures, and Noble’s arguments were easily dismissed or forgotten.

Then along came lecture capture and with it, in 2011, Massive Open Online Courses (xMOOCs) from Coursera, Udacity and edX, driven by elite, highly selective universities, with their claims of making the best professors in the world available to everyone for free. Noble’s nightmare suddenly became very real. At the same time, these MOOCs have resulted in much more interest in big data, learning analytics, a revival of adaptive learning, and claims that artificial intelligence will revolutionize education, since automation is essential for managing such massive courses.

Thus we are now seeing a big swing back to the automation of learning, driven by powerful computing developments, Silicon Valley start-up thinking, and a sustained political push from those that want to commercialize education (more on this later). Underlying these developments is a fundamental conflict of philosophies and pedagogies, with automation being driven by an objectivist/behaviourist view of the world, compared with the constructivist approaches of online collaborative learning.

In other words, there are increasingly stark choices to be made about the future of online learning. Indeed, it is almost too late – I fear the forces of automation are winning – which is why 2016 will be such a pivotal year in this debate.

Automation and the commercialization of education

These developments in technology are being accompanied by a big push in the United States, China, India and other countries towards the commercialization of online learning. In other words, education is being seen increasingly as a commodity that can be bought and sold. This is not through the previous and largely discredited digital diploma mills of the for-profit online universities such as the University of Phoenix that David Noble feared, but rather through the encouragement and support of commercial computer companies moving into the education field, companies such as Coursera, Lynda.com and Udacity.

Audrey Watters and EdSurge both produced lists of EdTech ‘deals’ in 2015 totalling between $1-$2 billion. Yes, that’s right, that’s $1-$2 billion in investment in private ed tech companies in the USA (and China) in one year alone. At the same time, entrepreneurs are struggling to develop sustainable business models for ed tech investment, because with education funded publicly, a ‘true’ market is restricted. Politicians, entrepreneurs and policy makers on the right in the USA increasingly see a move to automation as a way of reducing government expenditure on education, and one means by which to ‘free up the market’.

Another development that threatens the public education model is the move by very rich entrepreneurs such as the Gates, the Hewletts and the Zuckerbergs to move their massive personal wealth into ‘charitable’ foundations or corporations and use this money for their pet ‘educational’ initiatives that also have indirect benefits for their businesses. Ian McGugan (2015) in the Globe and Mail newspaper estimates that the Chan Zuckerberg Initiative is worth potentially $45 billion, and one of its purposes is to promote the personalization of learning (another name hi-jacked by computer scientists; it’s a more human way of describing adaptive learning). Since one way Facebook makes its money is by selling personal data, forgive my suspicions that the Zuckerberg initiative is a not-so-obvious way of collecting data on future high earners. At the same time, the Chang Zuckerberg initiative enables the Zuckerberg’s to avoid paying tax on their profits from Facebook. Instead then of paying taxes that could be used to support public education, these immensely rich foundations enable a few entrepreneurs to set the agenda for how computing will be used in education.

Why not?

Technology is disrupting nearly every other business and profession, so why not education? Higher education in particular requires a huge amount of money, mostly raised through taxes and tuition fees, and it is difficult to tie results directly to investment. Surely we should be looking at ways in which technology can change higher education so that it is more accessible, more affordable and more effective in developing the knowledge and skills required in today’s and tomorrow’s society?

Absolutely. It is not so much the need for change that I am challenging, but the means by which this change is being promoted. In essence, a move to automated learning, while saving costs, will not improve the learning that matters, and particularly the outcomes needed in a digital age, namely, the high level intellectual skills of critical thinking, innovation, entrepreneurship, problem-solving , high-level multimedia communication, and above all, effective knowledge management.

To understand why automated approaches to learning are inappropriate to the needs of the 21st century we need to look particularly at the tools and methods being proposed.

The problems with automating learning

The main challenge for computer-directed learning such as information transmission and management through Internet-distributed video lectures, computer-marked assessments, adaptive learning, learning analytics, and artificial intelligence is that they are based on a model of learning that has limited applications. Behaviourism works well in assisting rote memory and basic levels of comprehension, but does not enable or facilitate deep learning, critical thinking and the other skills that are essential for learners in a digital age.

R. and D. Susskind (2015) in particular argue that there is a new age in artificial intelligence and adaptive learning driven primarily by what they call the brute force of more powerful computing. Why AI failed so dramatically in the 1980s, they argue, was because computer scientists tried to mimic the way that humans think, and computers then did not have the capacity to handle information in the way they do now. When however we use the power of today’s computing, it can solve previously intractable problems through analysis of massive amounts of data in ways that humans had not considered.

There are several problems with this argument. The first is that the Susskinds are correct in that computers operate differently from humans. Computers are mechanical and work basically on a binary operating system. Humans are biological and operate in a far more sophisticated way, capable of language creation as well as language interpretation, and use intuition as well as deductive thinking. Emotion as well as memory drives human behaviour, including learning. Furthermore humans are social animals, and depend heavily on social contact with other humans for learning. In essence humans learn differently from the way machine automation operates.

Unfortunately, computer scientists frequently ignore or are unaware of the research into human learning. In particular they are unaware that learning is largely developmental and constructed, and instead impose an old and less appropriate method of teaching based on behaviourism and an objectivist epistemology. If though we want to develop the skills and knowledge needed in a digital age, we need a more constructivist approach to learning.

Supporters of automation also make another mistake in over-estimating or misunderstanding how AI and learning analytics operate in education. These tools reflect a highly objectivist approach to teaching, where procedures can be analysed and systematised in advance. However, although we know a great deal about learning in general, we still know very little about how thinking and decision-making operate biologically in individual cases. At the same time, although brain research is promising to unlock some of these secrets, most brain scientists argue that while we are beginning to understand the relationship between brain activity and very specific forms of behaviour, there is a huge distance to travel before we can explain how these mechanisms affect learning in general or how an individual learns in particular. There are too many variables (such as emotion, memory, perception, communication, as well as neural activity) at play to find an isomorphic fit between the firing of neurons and computer ‘intelligence’.

The danger then with automation is that we drive humans to learn in ways that best suit how machines operate, and thus deny humans the potential of developing the higher levels of thinking that make humans different from machines. For instance, humans are better than machines at dealing with volatile, uncertain, complex and ambiguous situations, which is where we find ourselves in today’s society.

Lastly, both AI and adaptive learning depend on algorithms that predict or direct human behaviour. These algorithms though are not transparent to the end users. To give an example, learning analytics are being used to identify students at high risk of failure, based on correlations of previous behaviour online by previous students. However, for an individual, should a software program be making the decision as to whether that person is suitable for higher education or a particular course? If so, should that person know the grounds on which they are considered unsuitable and be able to challenge the algorithm or at least the principles on which that algorithm is based? Who makes the decision about these algorithms – a computer scientist using correlated data, or an educator concerned with equitable access? The more we try to automate learning, the greater the danger of unintended consequences, and the more need for educators rather than computer scientists to control the decision-making.

The way forward

In the past, I used to think of computer scientists as colleagues and friends in designing and delivering online learning. I am now increasingly seeing at least some of them as the enemy. This is largely to do with the hubris of Silicon Valley, which believes that computer scientists can solve any problem without knowing anything about the problem itself. MOOCs based on recorded lectures are a perfect example of this, being developed primarily by a few computer scientists from Stanford (and unfortunately blindly copied by many people in universities who should have known better.)

We need to start with the problem, which is how do we prepare learners for the knowledge and skills they will need in today’s society. I have argued (Bates, 2015) that we need to develop, in very large numbers of people, high level intellectual and practical skills that require the construction and development of knowledge, and that enable learners to find, analyse, evaluate and apply knowledge appropriately.

This requires a constructivist approach to learning which cannot be appropriately automated, as it depends on high quality interaction between knowledge experts and learners. There are many ways to accomplish this, and technology can play a leading role, by enabling easy access to knowledge, providing opportunities for practice in experientially-based learning environments, linking communities of scholars and learners together, providing open access to unlimited learning resources, and above all by enabling students to use technology to access, organise and demonstrate their knowledge appropriately.

These activities and approaches do not easily lend themselves to massive economies of scale through automation, although they do enable more effective outcomes and possibly some smaller economies of scale. Automation can be helpful in developing some of the foundations of learning, such as basic comprehension or language acquisition. But at the heart of developing the knowledge and skills needed in today’s society, the role of a human teacher, instructor or guide will remain absolutely essential. Certainly, the roles of teachers and instructors will need to change quite dramatically, teacher training and faculty development will be critical for success, and we need to use technology to enable students to take more responsibility for their own learning, but it is a dangerous illusion to believe that automation is the solution to learning in the 21st century.

Protecting the future

There are several practical steps that need to be taken to prevent the automation of teaching.

  1. Educators – and in particular university presidents and senior civil servants with responsibility for education – need to speak out clearly about the dangers of automation, and the technology alternatives available that still exploit its potential and will lead to greater cost-effectiveness. This is not an argument against the use of technology in education, but the need to use it wisely so we get the kind of educated population we need in the 21st century.
  2. Computer scientists need to show more respect to educators and be less arrogant. This means working collaboratively with educators, and treating them as equals.
  3. We – teachers and educational technologists – need to apply in our own work and disseminate better to those outside education what we already know about effective learning and teaching.
  4. Faculty and teachers need to develop compelling technology alternatives to automation that focus on the skills and knowledge needed in a digital age, such as:
    • experiential learning through virtual reality (e.g. Loyalist College’s training of border service agents)
    • networking learners online with working professionals, to solve real world problems (e.g. by developing a program similar to McMaster’s integrated science program for online/blended delivery)
    • building strong communities of practice through connectivist MOOCs (e.g. on climate change or mental health) to solve global problems
    • empowering students to use social media to research and demonstrate their knowledge through multimedia e-portfolios (e.g. UBC’s ETEC 522)
    • designing openly accessible high quality, student-activated simulations and games but designed and monitored by experts in the subject area.
  5. Governments need to put as much money into research into learning and educational technology as they do into innovation in industry. Without better and more defensible theories of learning suitable for a digital age, we are open to any quack or opportunist who believes he or she has the best snake oil. More importantly, with better theory and knowledge of learning disseminated and applied appropriately, we can have a much more competitive workforce and a more just society.
  6. We need to educate our politicians about the dangers of commercialization in education through the automation of learning and fight for a more equal society where the financial returns on technology applications are more equally shared.
  7. Become edupunks and take back the web from powerful commercial interests by using open source, low cost, easy to use tools in education that protect our privacy and enable learners and teachers to control how they are used.

That should keep you busy in 2016.

Your views are of course welcome – unless you are a bot.

References

Bates, A. (1986) Computer assisted learning or communications: which way for information technology in distance education? Journal of Distance Education Vol. 1, No. 1

Bates, A. (2015) Teaching in a Digital Age Victoria BC: BCcampus

Harasim, L. (2012) Learning Theory and Online Technologies New York/London: Routledge

Mason, R. and Kaye, A (Eds).(1989)  Mindweave: communication, computers and distance education. Oxford: Pergamon

McGugan, I. (2015)Why the Zuckerberg donation is not a bundle of joy, Globe and Mail, December 2

Noble, D. (1998) Digital Diploma Mills, Monthly Review http://monthlyreview.org/product/digital_diploma_mills/

Papert, S. (1980) Mindstorms: Children, Computers and Powerful Ideas New York: Basic Books

Skinner, B. (1968)  The Technology of Teaching, 1968 New York: Appleton-Century-Crofts

Susskind, R. and Susskind, D. (2015) The Future of the Professions: How Technology will Change the Work of Human Experts Oxford UK: Oxford University Press

Watters, A. (2015) The Business of EdTech, Hack Edu, undated http://2015trends.hackeducation.com/business.html

Winters, M. (2015) Christmas Bonus! US Edtech Sets Record With $1.85 Billion Raised in 2015 EdSurge, December 21 https://www.edsurge.com/news/2015-12-21-christmas-bonus-us-edtech-sets-record-with-1-85-billion-raised-in-2015

That was the year that was: what you read on my blog in 2015

Working in my study

Blogging away

I find it a fun exercise to analyse the statistics for my blog at the end of the year, to see what were the most popular posts, as it gives some idea of the topics which have grabbed readers over the year. First let’s look at the figures for 2015 as a whole.

Overall visits

Blog views per month 2015 2

The total number of visits this year was way up from previous years. In 2013 the blog struggled to reach 20,000 visits a month most months. In 2014 the blog was averaging about 25,000 visits a month. In 2015 it was above 30,000 visits for nine of the twelve months and reached almost 40,000 visits in November, or an average of over 1,000 visits a day every day of the year.

It will be seen that the main reason for this large increase of visits in 2015 is indirectly related to the publication of Teaching in a Digital Age. I used my blog to ‘trial’ chapters and sections of the book, and it appears some of these blog posts are now being used as set readings on courses, or at least are being sought out by students studying, with students returning continually to these posts. I have no direct proof of this and if anyone using the blog can provide me with information on why they are using some of the blog posts on a regular basis, it will be appreciated.

Breakdown by sectors

Top posts in 2015

Top posts in 2015

To understand the picture better we need to break this down into categories.

Category 1: Students seeking advice about online courses/programs

I have several posts that attract mainly students or potential students (rather than faculty or instructors), looking for advice on online or distance learning courses or programs. These would include:

  • 1. The world’s largest supplier of free online learning? 31,886 visits in 2015No, interestingly, this isn’t about Coursera or Udacity, but Alison. Many students want to know about the status of the certificates as well as the courses offered by Alison, and the post has a regular and ongoing number of comments from people thinking about or who have taken Alison programs. The original post goes back to April, 2012 and has been one of the top posts every year since. However, the numbers this year almost tripled from 2014 (12,606).
  • 3. Recommended graduate programs in e-learning. 20,180 visits in 2015. Now number 3, this has been the perennial number 1 until this year, despite the number of hits increasing from 16,715 in 2014. This was one of my first posts, going back to July 2008. Although I revise it regularly, it is probably a little out of date as there have been many new programs in this area developed in the last two or three years and I list only those I am familiar with.
  • 7. Can you teach ‘real’ engineering at a distance? (posted 5 July, 2009). 6,388 visits in 2015. A great favourite of mine, especially since the answer has gradually changed over the six years since it was first posted (see my post on Trends in 2015), and again indicating the need for student guidance on choice of programs.
  • 8. A student guide to studying online. 5,974 visits in 2015. This jumped up from 15th position last year, with 2,321 hits, to 8th this year.

Although the blog is focused on faculty and instructors, it is clear from this data that there is high demand from students or potential students for independent, objective information about online courses and programs (also noted last year). The main value of the Alison post is the comments provided by actual users of the program.

I have hesitated to recommend online courses or programs in general, as I have no way of directly evaluating the vast majority, and there are so many of them now. However, it should be possible to create an independent, sponsor-free social media-based site where students can share information about different types of online courses and programs. I am sure such a web site would be very popular, but I’m leaving that to someone else to do (if it’s not already been done).

Category 2: Sections from ‘Teaching in a Digital Age.’

Graphic used for a short history of educational technology: Charlton Heston as Moses

Graphic used for a short history of educational technology: Charlton Heston as Moses

The big difference this year is that many of my posts were ‘first drafts’ of sections or chapters of my book (many published though in 2014 as well as 2015). These have proved extremely popular with readers in 2015, accounting for nine of the top 27 posts for 2015:

  • 2. A short history of educational technology (posted 10 December, 2014): 25,156 visits in 2015. It’s the second most popular post, and has always been among the top daily posts during 2015. This came as a complete surprise to me. I have no explanation why this is by far and away the most popular of the ‘book’ blog posts (three times more popular than the next book blog post). I’m assuming it’s a key reading for several courses, but why is it in constant demand almost every day? Am I being followed by a Charlton Heston fan club?
  • 5. Learning theories and online learning (posted 14 July, 2014): 8,528 visits in 2015. This one is less surprising. This is a core or foundational topic for any program of study on online learning.
  • 6. The strengths and weaknesses of competency-based learning in a digital age (posted 15 September, 2014) 6,822 visits in 2015. This reflects the strong interest (and possibly lack of other independent/’neutral’ accounts) in competency-based learning in 2015. I’m a little surprised though that it scored higher than the sections on ADDIE or MOOCs in my book.
  • 9. The role of communities of practice in a digital age (posted 1 October 2014). 5,300 visits in 2015. Same kind of comments as for competency-based learning.
  • 10. Deciding on appropriate media for teaching and learning (posted 28 January 2015). 4,932 visits in 2015. This one is gratifying as it covers the SECTIONS model and unlike the book drafts on different teaching methods, this post reflects my own, original work.
  • 12. Is the ADDIE model appropriate for teaching in a digital age? (posted 9 September 2014). 4,724 visits in 2015. I thought this might be more popular, seeing how central it has been to online course design in the past. However, there are plenty of other sources to go to about ADDIE.
  • 13. Comparing xMOOCs and cMOOCs: philosophy and practice (posted 13 October 2014). 4,645 visits in 2015. This was a chapter I didn’t want to write but had to. Glad I did now.
  • 20. What is a MOOC? (posted 12 October, 2014) 2,930 visits in 2015. Now we are getting quite a way down the list.
  • 22. Key characteristics of learners in a digital age….(posted 24 August, 2014) 2,819 visits in 2015. 

In the past, when I posted a new post, there was a flurry of visits in the first week or so, followed by a relatively small number of visits (less than 10 a day) in subsequent weeks. For the book posts though, the ‘tail’ is much thicker, with the above posts regularly getting between 30-50 visits each a day, and the order in terms of the number of visits doesn’t change a great deal from day to day.

Category 3: Single posts that won’t die

What's right and what's wrong with Coursera-style MOOCs: still popular

What’s right and what’s wrong with Coursera-style MOOCs: still popular

These are posts or topics that seem to have a life of their own. They are not tied in any direct way to the book.

Category 4: New in 2015

You will have noticed that almost all the above posts were posted before 2015. So what was new in 2015? In all I posted 114 posts in 2015, down considerably from previous years. Roughly a third of these posts were ‘drafts’ from the book. Here’s what did make it into the top 25 posts:

In 2014, five posts published in 2014 made the top 25.

Lessons

What do I draw from this analysis?

  • first, this is a good example why data on its own isn’t very helpful. You need to know the reasons or the logic that drives the data. What do visits actually mean? One manic person visiting the same post thousands of times? Or thousands quickly glancing at something and finding no interest?
  • there seem though to be some drivers to individual posts:
    • search engines picking up on key words or phrases in the title of the post (e.g. competency-based learning)
    • blog posts that are now required or recommended readings for courses
    • students looking for some guides or hints about online courses and programs or about online learning in general
    • faculty or instructors looking for resources
    • topical or current news items.
  • the book, ‘Teaching in a Digital Age’, has both benefited from and driven people back to the blog. This is somewhat surprising, since the posts were often first drafts and the authoritative version is in the book, which is available from a totally different web site. Perhaps the blog though is more generally accessible or known. Again, comments from readers of the blog or the book on this issue would be welcome.

Blogging a lot less this year but doubling the number of visits does seem though to go against the first law of blogging, which is to blog every day if possible to drive traffic to your site. However, I suppose that once you have two thousand posts that have value to at least someone, the blog has its own life force. Nevertheless I will continue to add new posts during 2016.

What is clear to me is that the site now has developed a life of its own, as a set of resources on online learning that people keep coming back to. This is immensely rewarding, as this was always the intention.

In the meantime, have a great holiday and don’t spend too much time on screen!

© Return Path Blog, 2012

© Return Path Blog, 2012