December 5, 2016

Learning analytics and learning design at the UK Open University

Listen with webReader
Maxim Jean-Louis (President, Contact North) and myself outside Walton Hall, the headquarters of the UK Open University, in 2012. It was my first visit since I left the OU in 1989.

Maxim Jean-Louis (President, Contact North) and myself outside Walton Hall, the headquarters of the UK Open University, in 2012. It was my first visit since I left the OU in 1989.

The Open University (2016) Developing learning design and analytics for student success, Connections, Vol. 21, No. 3

The latest edition of the Commonwealth of Learning’s magazine, Connections, has an interesting if brief article of the effective use of learning analytics. There are four key points that I noted:

  • the OU has scaled up its predictive use of learning analytics to cover over 45,000 students, and it works as well in traditional universities as in the OU
  • learning analytics is used in connection with learning design to identify not only students at risk but also to improve the design of the learning materials:

    the OU for the first time can empirically analyse the design of its modules. By linking learning designs with student satisfaction and success measures, it became possible to systematically identify, measure and improve critical aspects of students’ learning experience.

  • the OU is the first university in the world to develop and adopt a policy relating to the ethical uses of student data for
    learning analytics, involving students themselves in the development of the policy. This makes the adoption and use of learning analytics much easier
  • the OU has appointed a reader in learning analytics, Dr. Bart Rienties: that is treating learning analytics really seriously.

Unfortunately there were no links or ways to follow up the article.

Building a national survey of online learning in Canada

Listen with webReader
Image: Canada Explore

Image: Canada Explore

The players

Since April I have been leading a small team that has been trying to build from scratch a national survey of online learning in Canadian post-secondary institutions.

For many years the Babson Survey Research Group has been tracking the growth of online learning in higher education in the USA. With the U.S. Federal Department of Education now collecting this data through its annual IPEDS survey, Jeff Seaman of Babson has been working with Russ Poulin of WCET to help interpret the IPEDS data.

Through the intervention of Tricia Donovan, the director of eCampus Alberta, Jeff and Russ approached me to see if I would be willing to get a Canadian national survey off the ground. I guess I was chosen because through my blog I had been strongly critical of the lack of such data in Canada. (Warning to bloggers: be careful what you ask for as you may end up doing it yourself.)

As a Research Associate with Contact North, I approached its President, Maxim Jean-Louis, for his support. He immediately offered $10,000 towards the cost of the survey. This was a crucial contribution as it enabled me to sound out possible consultants for the project, because Babson had found that the most important contributor to success was ensuring close communication and co-operation with the institutions themselves before the survey was even designed.

The Contact North funding enabled me to approach Dr. Ross Paul, formerly President of two Canadian universities and more importantly, as the author of “Leadership Under Fire”, a book about the role of university presidents in Canada, he was extremely well connected with and knowledgeable about the whole Canadian university sector.

Maxim Jean-Louis also put me in touch with Brian Desbiens, a former college president and also a former chair of the Canadian College Presidents Network, another consultant with an immensely impressive network in the Canadian college sector.

Finally it was immediately clear to us that we needed someone with knowledge and expertise in the francophone sector, and through the assistance of REFAD, the francophone distance education network, Denis Mayer, a former Associate Vice President of Student Services at Laurentian University, also joined the team.

So we now had a steering group for the survey:

  • Tony Bates (lead researcher)
  • Ross Paul (universities)
  • Brian Desbiens (colleges)
  • Denis Mayer (francophone)
  • Tricia Donovan (provincial government agencies)
  • Jeff Seaman (survey design and implementation)
  • Russ Poulin (US liaison)

The process

Our first task was to ensure that we had support, or at least not opposition, from the institutions, about 80 universities and over 200 publicly funded colleges. Fortunately in Canada there are almost no private universities and there is a clear distinction between provincially funded and supported colleges and private career and language schools. Our survey is focused then solely on the public system of post-secondary education, consisting of just over 2 million students.

One challenge is that there is no overall federal responsibility for the delivery of post-secondary education in Canada. This means that there are 10 provinces with 10 slightly different systems of post-secondary education. In addition there are anglophone, francophone and bilingual institutions.

Nevertheless there are two key national organisations, Universities Canada (UC), and Colleges and Institutes Canada (CICAN), that between them cover most of the institutions, so one of our first tasks was to brief them and gain their support in communicating with the institutions. Also there are several francophone organisations that represent the interests of francophone universities and colleges, and the unique system in Québec of CEGEPs, publicly funded pre-university colleges that offer a pre-university qualification that is necessary for admission to Québec’s universities (except for mature students). Secondary school and undergraduate degrees are both one year shorter in Quebec as a result.

These initial contacts with the national or regional organisations enabled us to identify the population base for the survey: the list of institutions to be covered. This enabled the consultants to e-mail directly the provosts and VPs Academic of every institution for their support and participation in the study.

At the same time, the Steering Committee was engaged in a series of discussions around the design of the questionnaires. We had the advantage of the prior work of the Babson Survey Research Group in the USA, but the questionnaires had to be adapted to the unique Canadian post-secondary education system. At the same time we are anxious to ensure that we can make international comparisons. It became quickly clear that we will need several different versions of the questionnaire, as follows:

  •  anglophone universities
  • anglophone colleges
  • francophone universities
  • CEGEPS
  • francophone colleges (outside Québec).

Core questions would be the same across all versions, but others would reflect the unique nature of each institution (e.g. what qualifications were offered partly or wholly online).

To get early feedback on the questionnaire design, two consultants attended the CIRPA conference of Canadian institutional researchers and held a special session devoted to feedback on the initial questionnaire design and especially in the definitions of fully online and blended/hybrid learning.

The first full versions of the questionnaires have now been designed. We have identified 10 universities and eight colleges across all 10 provinces who have volunteered to give feedback on the pilot questionnaire, and they have been asked to reply by the end of December. We are planning one more round of piloting after that, and hope to have the final version of the questionnaire distributed to all the universities and colleges in March.

In order to keep the questionnaire as short as possible, we are collecting as much key data about the institutions, such as their size, from other sources. For instance, the Canadian Virtual University has provided data on distance education enrolments for its dozen or so member institutions that go back to 2001. In the end, we will have an extensive and comprehensive database of Canadian post-secondary educational institutions, and of their activities in online learning.

I am working with Jeff Seaman on the design of the questionnaire analysis, and we will use the Babson Survey Research Group’s data entry and analysis facilities to process the questionnaire data. We envisage one overall, national report in English and French and a number of smaller reports focused on specific sectors, including a specially written report on the francophone sector. These will be published in the summer of 2017, and the results will be presented at the ICDE’s World Congress on Online Learning in Toronto in October.

Lastly, we will not be identifying any individual institution, unless they expressly request to be identified, but we do aim to make the data open and accessible to other researchers. We hope to locate the data with one or more of the organizations representing the institutions.

Funding

The Babson surveys in the USA benefited from financial support from the Sloan Foundation and also from a number of private sponsors, such as publishers.  Funding frankly has been the biggest challenge so far for the Canadian survey.

We decided to divide the funding requirements into three stages. The first stage would be to acquire funds to develop the institutional support needed, build the database, and design and pilot the questionnaire. The second stage of funding would be to cover the costs of the data collection, data entry, data analysis, report writing and dissemination, as well as having sufficient funds to start the development of the following year’s survey. The third phase would be to cover long-term and regular funding for future annual surveys.

We have successfully completed the first phase of fund raising, thanks to the help of Contact North and the provincial eCampuses (BCcampus, eCampus Alberta, Campus Manitoba and eCampus Ontario). This has raised $45,000.

We are still seeking funding for the second phase. We estimate that we will need somewhere around $100,000 to complete the second phase, and for the third phase we will need to raise about $125,000 a year.

We have submitted requests for second stage funding to eCampus Ontario’s Research and Innovation Fund and to a Canadian foundation, and we are waiting to hear from them. The Canadian arm of a major publisher has also expressed an interest in supporting the survey. However, we are now at the point where we urgently need to secure firm funding for the second stage.

What we need

The project is now at a critical point in its development. We have secured the support of the institutions, we are ready to pilot the questionnaire, and we are building the institutional database. However, we still need the following:

  • money to cover the costs of the actual survey and report writing (in both English and French)
  • feedback on the definitions of online learning, whether we have the right questions, and whether institutions can actually provide the data requested; the piloting will provide this feedback
  • all institutions, large and small, whether they have strong or no online programs at all, to complete the questionnaire.

The benefits

If we are successful in completing the study, we hope that we will have achieved the following:

  • established a reliable snapshot of the state of online learning across Canada in post-secondary education
  • created a comprehensive, national database of Canadian post-secondary educational institutions that could be used for further research purposes
  • provided a baseline for future studies of online learning, so trends can be tracked
  • identified the areas where online learning is growing or declining
  • identified some of the key issues that institutions are facing regarding online learning
  • enabled institutions to see how they compare with other institutions in Canada in terms of their online learning development
  • enabled Canada to compare itself with developments in online learning in other countries.

Your help

Although we are still pursuing a number of possible sources of funding, if you have ideas of where or how to secure the the second and third stages of funding, please contact me at tony.bates@ubc.ca.

In particular, I urge Canadian readers of this blog to give their support within their institution to ensure that we get as good a response as possible to completing the questionnaire so that we have a reliable and comprehensive survey.

Any other comments about the value of the survey or the strategy we are following will also of course be welcome.

In the meantime, watch this space for further developments.

References

Paul, R. (2011) Leadership Under Fire: The Challenging Role of the Canadian University President Montreal & Kingston: McGill-Queen’s University Press, pp. 333

 

5 IDEAS for a pedagogy of online learning

Listen with webReader

ideas-2

Guardia, L. (ed.) (2016) Next Generation Pedagogy: IDEAS for Online and Blended Higher Education. Barcelona Spain: UOC eLearn Centre.

This report aims

to provide a “roadmap” to inform strategic planning for the future of online and blended higher education, through an exploratory search and identification of trends and innovations in online, blended and lifelong learning provision globally, with a focus on pedagogy and the analysis of related institutional examples in the higher education sector.

‘Next generation’ pedagogies

The report’s executive summary provides a succinct description of emerging developments in online pedagogy, summarised in the acronym IDEAS: Intelligent, Distributed, Engaging, Agile and Situated.

The IDEAS framework presents five “signposts” on the roadmap of innovative approaches to teaching, which point to next-generation pedagogy:

  • Intelligent pedagogy is an approach to teaching in which technology is used to enhance the learning experience. Examples include using learning analytics to support course leaders in curriculum design decisions as well as to help students manage their learning, ensuring that both learners and teachers learn digital competences, creating a learning environment that is not restricted to an institutional learning management system, and the creative use of technologies such as virtual and augmented reality for learning and teaching.
  • Distributed pedagogy refers to shared or distributed ownership of different elements of the learning journey by different stakeholders in the process. It includes, at the one end of the spectrum, collaborative partnerships between institutions, and at the other, a deliberate separation of services to allow learners to select different aspects of their learning experience from a marketplace of potentially competing providers. It is possible for a single institution to have offerings at both ends of this spectrum.
  • Engaging pedagogy is an approach to curriculum design and delivery in which learners are encouraged to actively participate in the learning process. Related practices include supporting students to develop portfolios that have relevance for them outside of the classroom, involving the learners in producing content both for peers and for the wider public, creating conditions in which learners can construct knowledge for themselves, and including an element of gamification in the learning process. There is a related increased emphasis on teaching enhancement programmes for teaching staff to support them in making the learning experience engaging for learners.
  • Agile pedagogy refers to flexibility and customisation of the curriculum and the student experience. It includes personalised learning pathways and individualised support for learners, recognition of prior, non-formal learning achievements in order to widen participation and fast-track learners through programmes, responsiveness of institutions and systems to learners’ needs, and support for virtual mobility of students and internationalisation of the curriculum. All these developments also support the widening of participation in higher education, facilitating access for learners who might previously have been excluded.
  •  Situated pedagogy encompasses the idea of contextualisation of learning and emphasises the need for curricula with real-world relevance. It expands work-related learning opportunities for students, and supports.

It also provides real-world examples, drawn internationally, that illustrate each of the five developments.

This report is intended to be used as a launchpad for wide-ranging dialogue amongst stakeholders at distance teaching universities, contributing to the development of a bold vision of the impact that these institutions can have on global higher education and on society as a whole.

Emerging online practices

The five IDEAS are a useful organizational framework for summarising what in fact is a wide range of emerging online practices identified in this study, including:

  • Active learning (Arizona State University, USA)
  • Beyond the LMS: augmented reality: (University de Huelva, Spain)
  • Inter-institutional collaboration (BCcampus, Canada)
  • Digital literacy (DIGCOMP, EU)
  • Collaboration between HE institutions and employers (X-Culture, global)
  • Flexibility and personalisation (Capella University, USA)
  • Innovation as a teachable topic (MIT, USA)
  • Internationalisation (Massey University, New Zealand)
  • Learning analytics (Georgia State University, USA)
  • Non-formal and open learning (OpenClassrooms, France)
  • Recognition of prior learning (Athabasca University, Canada)
  • Faculty development (Penn State University, USA)

Comment

This report provides some intriguing suggestions for designers of blended and online learning. As the report states:

The examples of initiatives discussed ….. may be used as inspiration for course teams, departments or institutions to explore innovative practices.

It is clear that universities are going to change, not just because technology is at last beginning to radically shake up how we design courses, but also because the needs of learners are changing. In the end, the value of any new online pedagogy will be judged by how well it helps meets these needs. This report provides many useful ideas and examples that should help stimulate such developments.

Thanks to Richard Elliott’s excellent eLearning Watch for directing me to this publication

A post-graduate certificate in neuroscience and online learning

Listen with webReader
Image: Roche, 2016

Image: Roche, 2016

One of the interesting things I discovered during my visit to Drexel University a couple of weeks ago is that they offer a post-graduate certificate program in neuroscience, learning and online instruction.  Drexel University is an international leader in neuroscience research, and its nursing program is very innovative in its online teaching.  This program looks at the cognitive aspects of neuroscience and how it can be applied in the field of online learning.

The program:

provides the knowledge and skills to apply concepts of neuroscience, cognition, and learning theory to online instruction based on evidenced-based research and real-world practice.

After completing this six-course certificate you will be able to:

  • Address regulations related to online learning
  • Apply neurobiology and learning theory approaches to online instruction to support transfer of learning and self-efficacy
  • Apply procedural and metacognitive scaffolding strategies to online instruction
  • Compare and contrast neuroimaging techniques for investigating cognitive function following injury resulting in impairment
  • Describe neuroanatomy, cognitive function, and multi-sensory learning
  • Discuss cognitive, social, and emotional learning as it relates to adult learners and the memory process
  • Elucidate the relationship between neuroplasticity, neural networks, and rehabilitation
  • Explain the anatomical and physical bases of learning and memory.

The program is aimed at:

  • Post-baccalaureate professionals who instruct or plan to instruct courses in online and/or blended environments building upon the facets of neuroscience, cognitive function, and online learning.
  • Graduate students wishing to continue their graduate study to include a post-baccalaureate certificate in Neuroscience, Learning and Online Instruction.

The program is of course available entirely online.

I have to say that if I was starting my career today in online learning, this is exactly the kind of program I would need to take.

For further information go to: https://online.drexel.edu/online-degrees/nursing-degrees/pbc-neuro-learning/index.aspx

 

Towards an open pedagogy for online learning

Listen with webReader
Image: © University of Victoria, BC

Image: © University of Victoria, BC

The problems with OER

I was interviewed recently by a reporter doing an article on OER (open educational resources) and I found myself being much more negative than I expected, since I very much support the principle of open-ness in education. In particular, I pointed out that OER, while slowly growing in acceptance, are still used for a tiny minority of teaching in North American universities and colleges. For instance, open textbooks are a no brainer, given the enormous savings they can bring to students, but even in the very few state or provincial jurisdictions that have an open textbook program, the take-up is still very slow.

I have written elsewhere in more detail about why this is so, but here is a summary of the reasons:

  • lack of suitable OER: finding the right OER for the right context. This is a problem that is slowly disappearing, as more OER become available, but it is still difficult to find exactly the right kind of OER to fit a particular teaching context in too many instances. It is though a limitation that I believe will not last for much longer (for the reasons for this, read on).
  • the poor quality of what does exist. This is not so much the quality of content, but the quality of production. Most OER are created by an individual instructor working alone, or at best with an instructional designer. This is the cottage industry approach to design. I have been on funding review committees where institutions throughout a province are bidding for funds for course development or OER production. In one case I reviewed requests from about eight different institutions for funds to produce OER for statistics. Each institution (or rather faculty member) made its proposal in isolation of the others. I strongly recommended that the eight faculty members got together and designed a set of OER together that would benefit from a larger input of expertise and resources. That way all eight institutions were likely to use the combined OER, and the OER would likely be of a much higher quality as a result.
  • the benefits are less for instructors than students. Faculty for instance set the textbook requirement. They don’t have to pay for the book themselves in most cases. With the textbook often comes a whole package of support materials from the publisher, such as tests, supplementary materials, and model answers (which is why the textbook is so expensive). This makes life easier for instructors but it is the students who have to pay the cost.
  • OER take away the ‘ownership’ of knowledge from the instructor. Instructors do not see themselves as merely distributors of information, a conveyor belt along which ‘knowledge’ passes, but as constructors of knowledge. They see their lecture as unique and individual, something the student cannot get from someone else. And often it is unique, with an instructor’s personal spin on a topic. OER’s take away from instructors that which they see as being most important about their teaching: their unique perspective on a topic.
  • and now we come to what I think is the main problem with OER: OER do not make much sense out of context. Too often the approach is to create an OER then hope that others will find applications for it. But this assumes that knowledge is like a set of bricks. All you have to do is to collect bricks of knowledge together, add a little  mortar, and lo, you have a course. The instructor chooses the bricks and the students apply the mortar. Or you have a course but you need to fill some holes in it with OER. I suggest these are false metaphors for teaching, or at least for how people learn. You need a context, a pedagogy, where it makes sense to use open resources.

Towards an open pedagogy

I am making three separate but inter-linked arguments here:

  • OER are too narrowly defined and conceptualized
  • we need to design teaching in such a way that it is not just sensible to use OER but unavoidable
  • we should start by defining what we are trying to achieve, then identify how OER will enable this.

So I will start with the last argument first.

Developing the knowledge and skills needed in the 21st century

Again I have written extensively about this (see Chapter 1 of Teaching in a Digital Age), but in essence we need to focus specifically on developing core ‘soft’ or ‘intellectual’ skills in our students, and especially the core skills of independent learning and knowledge management. Put in terms of learning outcomes, in a world where the content component of knowledge is constantly developing and growing, students need to learn independently so they can continue to learn after graduation, and students also need to know how to find, analyse, evaluate, and apply knowledge.

If we want students to develop these and other ‘soft’ skills such as problem-solving, critical thinking, evidence-based argumentation, what teaching methods or pedagogy should we adopt and how would it differ from what we do now?

The need for teaching methods that are open rather than closed

The first thing we should recognise is that in a lecture based methodology, it is the instructor doing the knowledge management, not the student. The instructor (or his or her colleagues) decide the curriculum, the required reading, what should be covered in each lecture, how it should be structured, and what should be assessed. There is little independence for the learner – either do what you are instructed to do, or fail. That is a closed approach to teaching.

I am suggesting that we need to flip this model on its head. It should ultimately be the students learning and deciding what content is important, how it should be structured, how it can be applied. The role of the instructor then would not be to choose, organise and deliver content, but to structure the teaching to enable students to do this effectively themselves.

This also should not be a sudden process, where students suddenly switch from a lecture-based format as an undergraduate to a more open structure as a post-graduate, but a process that is slowly and increasingly developed throughout the undergraduate program or a two-year college program where soft skills are considered important. One way – although there are many others – of doing this is through project- or problem-based learning, where students start with real challenges then develop the knowledge and skills needed to address such challenges.

This does not mean we no longer need subject specialists or content experts. Indeed, a deep understanding of a subject domain is essential if students are to be steered and guided and properly assessed. However, the role of the subject specialist is fundamentally changed. He or she is now required to set their specialist knowledge in a context that enables student discovery and exploration, and student responsibility for learning. The specialist’s role now is to support learning, by providing appropriate learning contexts, guidance to students, criteria for assessing the quality of information, and quality standards for problem-solving, knowledge management and critical thinking, etc.

A new definition of open resources

Here I will be arguing for a radical change: the dropping of the term ‘educational’ from OER.

If students are to develop the skills identified earlier, they will need access to resources: research papers, reports from commissions, case-study material, books, first-hand reports, YouTube video, a wide range of opinions or arguments about particular topics, as well as the increasing amount of specifically named open educational resources, such as recorded lectures from MIT and other leading research universities.

Indeed, increasingly all knowledge is becoming open and easily accessible online. All publicly funded research in many countries must now be made available through open access journals, increasingly government and even some commercial data (think government commission reports, environmental assessments, public statistics, meteorological models) are now openly accessible online, and this will become more and more the norm. In other words, all content is becoming more free and more accessible, especially online.

With that comes of course more unreliable information, more false truths, and more deliberate propaganda. What better preparation for our students’ future is there than equipping them with the knowledge and skills to sift through this mass of contradictory information?  What better than to make them really good at identifying the true from the false, to evaluate the strength of an argument, to assess the evidence used to support an argument, whatever the subject domain? To do this though means exposing them to a wide range of openly accessible content, and providing the guidance and criteria, and the necessary prior knowledge, that they will need to make these decisions.

But we cannot do this if we restrict our students to already ‘approved’ OER. All content eventually becomes an educational resource, a means to help students to differentiate, evaluate and decide. By naming content as ‘educational’ we are already validating its ‘truth’ – we are in fact closing the mind to challenge. What we want is access to open resources – full stop. Let’s get rid of the term OER and instead fight for an open pedagogy.