July 20, 2018

Building a national survey of online learning in Canada

Image: Canada Explore

Image: Canada Explore

The players

Since April I have been leading a small team that has been trying to build from scratch a national survey of online learning in Canadian post-secondary institutions.

For many years the Babson Survey Research Group has been tracking the growth of online learning in higher education in the USA. With the U.S. Federal Department of Education now collecting this data through its annual IPEDS survey, Jeff Seaman of Babson has been working with Russ Poulin of WCET to help interpret the IPEDS data.

Through the intervention of Tricia Donovan, the director of eCampus Alberta, Jeff and Russ approached me to see if I would be willing to get a Canadian national survey off the ground. I guess I was chosen because through my blog I had been strongly critical of the lack of such data in Canada. (Warning to bloggers: be careful what you ask for as you may end up doing it yourself.)

As a Research Associate with Contact North, I approached its President, Maxim Jean-Louis, for his support. He immediately offered $10,000 towards the cost of the survey. This was a crucial contribution as it enabled me to sound out possible consultants for the project, because Babson had found that the most important contributor to success was ensuring close communication and co-operation with the institutions themselves before the survey was even designed.

The Contact North funding enabled me to approach Dr. Ross Paul, formerly President of two Canadian universities and more importantly, as the author of “Leadership Under Fire”, a book about the role of university presidents in Canada, he was extremely well connected with and knowledgeable about the whole Canadian university sector.

Maxim Jean-Louis also put me in touch with Brian Desbiens, a former college president and also a former chair of the Canadian College Presidents Network, another consultant with an immensely impressive network in the Canadian college sector.

Finally it was immediately clear to us that we needed someone with knowledge and expertise in the francophone sector, and through the assistance of REFAD, the francophone distance education network, Denis Mayer, a former Associate Vice President of Student Services at Laurentian University, also joined the team.

So we now had a steering group for the survey:

  • Tony Bates (lead researcher)
  • Ross Paul (universities)
  • Brian Desbiens (colleges)
  • Denis Mayer (francophone)
  • Tricia Donovan (provincial government agencies)
  • Jeff Seaman (survey design and implementation)
  • Russ Poulin (US liaison)

The process

Our first task was to ensure that we had support, or at least not opposition, from the institutions, about 80 universities and over 200 publicly funded colleges. Fortunately in Canada there are almost no private universities and there is a clear distinction between provincially funded and supported colleges and private career and language schools. Our survey is focused then solely on the public system of post-secondary education, consisting of just over 2 million students.

One challenge is that there is no overall federal responsibility for the delivery of post-secondary education in Canada. This means that there are 10 provinces with 10 slightly different systems of post-secondary education. In addition there are anglophone, francophone and bilingual institutions.

Nevertheless there are two key national organisations, Universities Canada (UC), and Colleges and Institutes Canada (CICAN), that between them cover most of the institutions, so one of our first tasks was to brief them and gain their support in communicating with the institutions. Also there are several francophone organisations that represent the interests of francophone universities and colleges, and the unique system in Québec of CEGEPs, publicly funded pre-university colleges that offer a pre-university qualification that is necessary for admission to Québec’s universities (except for mature students). Secondary school and undergraduate degrees are both one year shorter in Quebec as a result.

These initial contacts with the national or regional organisations enabled us to identify the population base for the survey: the list of institutions to be covered. This enabled the consultants to e-mail directly the provosts and VPs Academic of every institution for their support and participation in the study.

At the same time, the Steering Committee was engaged in a series of discussions around the design of the questionnaires. We had the advantage of the prior work of the Babson Survey Research Group in the USA, but the questionnaires had to be adapted to the unique Canadian post-secondary education system. At the same time we are anxious to ensure that we can make international comparisons. It became quickly clear that we will need several different versions of the questionnaire, as follows:

  •  anglophone universities
  • anglophone colleges
  • francophone universities
  • CEGEPS
  • francophone colleges (outside Québec).

Core questions would be the same across all versions, but others would reflect the unique nature of each institution (e.g. what qualifications were offered partly or wholly online).

To get early feedback on the questionnaire design, two consultants attended the CIRPA conference of Canadian institutional researchers and held a special session devoted to feedback on the initial questionnaire design and especially in the definitions of fully online and blended/hybrid learning.

The first full versions of the questionnaires have now been designed. We have identified 10 universities and eight colleges across all 10 provinces who have volunteered to give feedback on the pilot questionnaire, and they have been asked to reply by the end of December. We are planning one more round of piloting after that, and hope to have the final version of the questionnaire distributed to all the universities and colleges in March.

In order to keep the questionnaire as short as possible, we are collecting as much key data about the institutions, such as their size, from other sources. For instance, the Canadian Virtual University has provided data on distance education enrolments for its dozen or so member institutions that go back to 2001. In the end, we will have an extensive and comprehensive database of Canadian post-secondary educational institutions, and of their activities in online learning.

I am working with Jeff Seaman on the design of the questionnaire analysis, and we will use the Babson Survey Research Group’s data entry and analysis facilities to process the questionnaire data. We envisage one overall, national report in English and French and a number of smaller reports focused on specific sectors, including a specially written report on the francophone sector. These will be published in the summer of 2017, and the results will be presented at the ICDE’s World Congress on Online Learning in Toronto in October.

Lastly, we will not be identifying any individual institution, unless they expressly request to be identified, but we do aim to make the data open and accessible to other researchers. We hope to locate the data with one or more of the organizations representing the institutions.

Funding

The Babson surveys in the USA benefited from financial support from the Sloan Foundation and also from a number of private sponsors, such as publishers.  Funding frankly has been the biggest challenge so far for the Canadian survey.

We decided to divide the funding requirements into three stages. The first stage would be to acquire funds to develop the institutional support needed, build the database, and design and pilot the questionnaire. The second stage of funding would be to cover the costs of the data collection, data entry, data analysis, report writing and dissemination, as well as having sufficient funds to start the development of the following year’s survey. The third phase would be to cover long-term and regular funding for future annual surveys.

We have successfully completed the first phase of fund raising, thanks to the help of Contact North and the provincial eCampuses (BCcampus, eCampus Alberta, Campus Manitoba and eCampus Ontario). This has raised $45,000.

We are still seeking funding for the second phase. We estimate that we will need somewhere around $100,000 to complete the second phase, and for the third phase we will need to raise about $125,000 a year.

We have submitted requests for second stage funding to eCampus Ontario’s Research and Innovation Fund and to a Canadian foundation, and we are waiting to hear from them. The Canadian arm of a major publisher has also expressed an interest in supporting the survey. However, we are now at the point where we urgently need to secure firm funding for the second stage.

What we need

The project is now at a critical point in its development. We have secured the support of the institutions, we are ready to pilot the questionnaire, and we are building the institutional database. However, we still need the following:

  • money to cover the costs of the actual survey and report writing (in both English and French)
  • feedback on the definitions of online learning, whether we have the right questions, and whether institutions can actually provide the data requested; the piloting will provide this feedback
  • all institutions, large and small, whether they have strong or no online programs at all, to complete the questionnaire.

The benefits

If we are successful in completing the study, we hope that we will have achieved the following:

  • established a reliable snapshot of the state of online learning across Canada in post-secondary education
  • created a comprehensive, national database of Canadian post-secondary educational institutions that could be used for further research purposes
  • provided a baseline for future studies of online learning, so trends can be tracked
  • identified the areas where online learning is growing or declining
  • identified some of the key issues that institutions are facing regarding online learning
  • enabled institutions to see how they compare with other institutions in Canada in terms of their online learning development
  • enabled Canada to compare itself with developments in online learning in other countries.

Your help

Although we are still pursuing a number of possible sources of funding, if you have ideas of where or how to secure the the second and third stages of funding, please contact me at tony.bates@ubc.ca.

In particular, I urge Canadian readers of this blog to give their support within their institution to ensure that we get as good a response as possible to completing the questionnaire so that we have a reliable and comprehensive survey.

Any other comments about the value of the survey or the strategy we are following will also of course be welcome.

In the meantime, watch this space for further developments.

References

Paul, R. (2011) Leadership Under Fire: The Challenging Role of the Canadian University President Montreal & Kingston: McGill-Queen’s University Press, pp. 333

 

Is there too much innovation in education?

© Boloreport.com, 2012

Maxim Jean-Louis in a recent comment to my post on the e-learning outlook for 2012 referred to observations by Ben Levin who in an interesting interview with Cheryl Jackson argues that what the education system needs is not more innovation, but tested and evaluated improvements. In particular in the interview he singled out technology as an area that had failed to lead to improvements in the educational system. Ben Levin is a professor at the Ontario Institute for Studies in Education at the University of Toronto and formerly a top administrator (Deputy Minister) in the Ontario provincial government’s Ministry of Education. I strongly recommend you view the 12 minute video before reading this post.

In a similar vein, I heard recently of a dean in a Canadian university who complained bitterly at the continual cost of LMS upgrades (the event which prompted his comments), the push for the use of new technologies such as social media, and the pressure on faculty to constantly keep up with technological change. His comment to the Director of the Centre for Learning Technologies was: ‘All you guys want to do is to keep finding things to keep you employed. We don’t need innovation; we need stability in our teaching.’

Many of us working in the field of learning technologies take for granted the need for innovation, but it is incumbent on us that we do not push innovation for innovation’s sake. Nevertheless, my view is that at least for post-secondary education, we are in desperate need of innovation, and that e-learning and online learning needs to be a major component of changes to the system. In this post I want to discuss why I think that innovation is essential, and why learning technologies need to be a central part of such innovation, but also to discuss where there are areas of agreement as well as disagreement between Professor Levin and myself, because he raises some important points about the value or otherwise of innovation in education.

First it should be pointed out that Professor Levin was talking about the school or k-12 system, not post-secondary education. It should be noted that Ontario k-12 students perform really well in the PISA standardized tests in reading, mathematics and science, that there has been universal access to public schools in Canada for over 100 years and that over 90% of students successfully complete Grade 12 in Ontario. Where technology has been applied in the school system it has been used to supplement rather than replace the standard classroom. Lastly Ontario has one of the lowest teacher:student ratios in the world in its public education system (about 1:20). Professor Levin argues that all systems need some innovation, but the bulk of investment should be in maintaining well-tried processes and systems, and trying to improve them rather than change them.

However, I would argue that this is not the situation regarding post-secondary education in Canada. Yes, we do have excellent universities and two year community colleges, and we do have a high participation rate at over 50% of a cohort nationally going on to some form of public post-secondary education. However, it is only in the last 50 years that we have had mass post-secondary education, and the system continues to grow. (Ontario, which already has a post-secondary participation rate of 63%, is pushing for 70%, or another 60,000 places). This growth has not been matched by a similar growth in per-student funding anywhere in the Western world, resulting in much larger classes (average class size in many large Canadian research universities is 1:200 at undergraduate level). Furthermore, as in other countries, the post-secondary institutions have absorbed this increased growth by maintaining methods of teaching that worked well for an elite, low teacher:student ratio, but which are demonstrably inadequate in terms of developing high quality interaction with top research faculty (especially at the undergraduate level), and more pertinently, in developing the skills and competencies needed in today’s knowledge-based society.

Above all, Ontario in particular in Canada is facing three competing challenges:

  • the need to increase access to post-secondary education, to ensure an adequate supply of knowledge workers and to develop additional sources of economic growth to the current economy, which is heavily dependent on manufacturing and is struggling to get out of the 2008 recession;
  • the need to maintain and if possible increase the quality of post-secondary education if it is to remain economically competitive in a knowledge-based economy, by focusing more on the learning needs of 21st century students and workers;
  • lastly, and of particular significance for Ontario, to find ways to reduce or eliminate a current annual budgetary deficit of $14 billion, within five years.

Even if by some miracle the government is able to avoid cuts to the post-secondary budget, there is not going to be extra money for the other two challenges of additional growth and improved or even maintained quality. Basically, the status quo is untenable, not just in Ontario but in many other jurisdictions. There are several different options, such as increased taxation, or cuts in spending, but doing things differently to try to balance these three challenges, i.e. innovation, should also be at least part of the solution.

I would argue that Professor Levin’s distinction between improvement and innovation is a little contrived. One could argue that improvement is basically successful innovation. However, where I do agree with Professor Levin is that such innovation should be carefully piloted, evaluated and assessed before it is rolled out across a system as a whole.

Nevertheless, I think that the distinction is more than one of degree. I think there is a qualitative difference in the goals of innovation and improvement. Improvement means striving to do better something that is already considered successful and needs tweaking or continuous degrees of improved performance. In educational terms, this might mean higher scores on standardized tests. Thus the goals or learning outcomes remain the same – you just want to do the same thing a bit better. I think this is not a bad description of Ontario’s k-12 system.

Innovation though often means not just improvement but doing something radically different – Christensen’s disruptive innovation. This may mean seeking to reach new or different goals (or at least changing the emphasis within existing goals). And this is where I believe that post-secondary institutions universally are struggling. The outside world is changing faster than the post-secondary institutions can keep up with, given the challenges they are facing (more students and lower funding per student). Some of us also believe some of the goals of post-secondary education need to change, with more emphasis on skills, in particular knowledge management, and less on transmitting and remembering content. In particular, we need to look at what technology is doing outside the university or college, and the implications of this for teaching and learning. The focus on technology is not because it enables us to do the same thing slightly better, but because it requires us to do things differently in a world where technology changes so quickly.

Thus for me learning technologies, and particularly hybrid and online learning, provide opportunities for innovations in post-secondary education that are needed to meet the real challenges being faced by governments and post-secondary institutions. The trick is not to add on the technology to existing methods (which as Professor Levin has pointed out, leads to no significant learning differences) but to use technology to meet new goals that better reflect the needs of a knowledge-based society. This means redesigning not just our teaching methods but also our institutions to fully exploit the opportunities that technology provides.

Where I do agree with Professor Levin is that technology cannot bring about these changes on its own. It needs to be related to changes also in pedagogy (how we teach) and to identifying accurately the different learning outcomes that technology best supports. It also needs leadership and administrative changes that encourage innovation, and rewards successful and well-tested innovations. Above all, though, it depends on the recognition that the status quo is no longer possible and that change and innovation is necessary in our post-secondary institutions.