June 21, 2018

Follow Tony on Twitter

You are here: Home / Archives for Strategies, planning and management / quality and quality assurance

Learning analytics, student satisfaction, and student performance at the UK Open University

May 11, 2018 by Leave a Comment

There is very little correlation between student satisfaction and student performance. Image: Bart Rienties. Click on image to see the video.

Rienties, B. and Toetenel, L. (2016) The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules, Computers in Human Behaviour, Vol. 60, pp.333-341

Li, N. et al. (2017) Online learning experiences of new versus continuing learners: a large-scale replication study, Assessment and Evaluation in Higher Education, Vol. 42, No. 4, pp.657-672

It’s never too late to learn

It’s been a hectic month with two trips from Vancouver to Ontario and back and one to the UK and back, a total of four keynotes, two panel sessions and two one day consultancies. By the time I got to the end of the month’s travels, I had learned so much that at a conference in Toronto I had to go to my room and lie down  – I just couldn’t take any more!

At my age, it takes time to process all this new information, but I will try to summarise the main points of what I learned in the next three posts.

Learning analytics at the Open University

The Open University, with over 100,000 students and more than 1,000 courses (modules), and most of its teaching online in one form or another, is an ideal context for the application of learning analytics. Fortunately the OU has some of the world leaders in this field. 

At the conference on STEM teaching at the Open University that I attended as the opening keynote, the closing keynote was given by Bart Rienties, Professor of Learning Analytics at the Institute of Educational Technology at the UK Open University. Rienties and his team linked 151 modules (courses) and 111,256 students with students’ behaviour, satisfaction and performance at the Open University UK, using multiple regression models. 

His whole presentation (40 minutes, including questions) can be accessed online, and is well worth viewing, as it provides a clear summary of the results published in the two detailed papers listed above. As always, if you find my summary of results below of interest or challenging, I strongly recommend you view Bart’s video first, then read the two articles in more detail. Here’s what I took away.

There is little correlation between student course evaluations and student performance

This result is a bit of a zinger. The core dependent variable used was academic retention (the number of learners who completed and passed the module relative to the number of learners who registered for each module). As Rientes and Toetenel (p.340) comment, almost as an aside, 

it is remarkable that learner satisfaction and academic retention were not even mildly related to each other….Our findings seem to indicate that students may not always be the best judge of their own learning experience and what helps them in achieving the best outcome.’

The design of the course matters

One of the big challenges in online and blended learning is getting subject matter experts to recognise the importance of what the Open University calls ‘learning design.’ 

Conole (2012, p121) describes learning design as:

a methodology for enabling teachers/designers to make more informed decisions in how they go about designing learning activities and interventions, which is pedagogically informed and makes effective use of appropriate resources and technologies. LD is focussed on ‘what students do’ as part of their learning, rather than the ‘teaching’ which is focussed on the content that will be delivered.

Thus learning design is more than just instructional design.

However, Rienties at al. comment that ‘only a few studies have investigated how educators in practice are actually planning and designing their courses and whether this is then implemented as intended in the design phase.’ 

The OU has done a good job in breaking down some of the elements of learning design. The OU has mapped the elements of learning design in nearly 200 different courses. The elements of this mapping can be seen below (Rientes and Toetenal, 2016, p.335):

Rientes and Toetenel then analysed the correlations between each of these learning design elements against both learner satisfaction and learner performance. What they found is that what OU students liked did not match with learner performance. For instance, students were most satisfied with ‘assimilative’ activities, which are primarily content focused, and disliked communication activities, which are primarily social activities. However, better student retention was most strongly associated with communication activities, and overall, with the quality of the learning design.

Rientes and Toetenel conclude:

although more than 80% of learners were satisfied with their learning experience, learning does not always need to be a nice, pleasant experience. Learning can be hard and difficult at times, and making mistakes, persistence, receiving good feedback and support are important factors for continued learning….

An exclusive focus on learner satisfaction might distract institutions from understanding the impact of LD on learning experiences and academic retention. If our findings are replicated in other contexts, a crucial debate with academics, students and managers needs to develop whether universities should focus on happy students and customers, or whether universities should design learning activities that stretch learners to their maximum abilities and ensuring that they eventually pass the module. Where possible, appropriate communication tasks that align with the learning objectives of the course may seem to be a way forward to enhance academic retention.

Be careful what you measure

As Rientes and Toetenel put it:

Simple LA metrics (e.g., number of clicks, number of downloads) may actually hamper the advancement of LA research. For example, using a longitudinal data analysis of over 120 variables from three different VLE/LMS systems and a range of motivational, emotions and learning styles indicators, Tempelaar et al. (2015) found that most of the 40 proxies of “simple” VLE LA metrics provided limited insights into the complexity of learning dynamics over time. On average, these clicking behaviour proxies were only able to explain around 10% of variation in academic performance.

In contrast, learning motivations, emotions (attitudes), and learners’ activities during continuous assessments (behaviour) significantly improved explained variance (up to 50%) and could provide an opportunity for teachers to help at-risk learners at a relatively early stage of their university studies.

My conclusions

Student feedback on the quality of a course is really important but it is more useful as a conversation between students and instructors/designers than as a quantitative ranking of the quality of a course.  In fact using learner satisfaction as a way to rank teaching is highly misleading. Learner satisfaction encompasses a very wide range of factors as well as the teaching of a particular course. It is possible to imagine a highly effective course where teaching in a transmissive or assimilative manner is minimal, but student activities are wide, varied and relevant to the development of significant learning outcomes. Students, at least initially, may not like this because this may be a new experience for them, and because they must take more responsibility for their learning. Thus good communication and explanation of why particular approaches to teaching have been chosen is essential (see my comment to a question on the video).

Perhaps though the biggest limitation of student satisfaction for assessing the quality of the teaching is the often very low response rates from students, limited evaluation questions due to standardization (the same questions irrespective of the nature of the course), and the poor quality of the student responses. This is no way to assess the quality of an individual teacher or a whole institution, yet far too many institutions and governments are building this into their evaluation of teachers/instructors and institutions.

I have been fairly skeptical of learning analytics up to now, because of the tendency to focus more on what is easily measurable (simple metrics) than on what students actually do qualitatively when they are learning. The focus on learning design variables in these studies is refreshing and important but so will be analysis of student learning habits.

Finally, this research provides quantitative evidence of the importance of learning design in online and distance teaching. Good design leads to better learning outcomes. Why then are we not applying this knowledge to the design of all university and college courses, and not just online courses? We need a shift in the power balance between university and college subject experts and learning designers resulting in the latter being treated as at least equals in the teaching process.

References

Conole, G. (2012). Designing for learning in an open world. Dordrecht: Springer

Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: learning analytics in a data-rich context. Computers in Human Behavior, 47, 157e167. http://dx.doi.org/10.1016/j.chb.2014.05.038.

 

Filed Under: analytics, Bibliography, completion rates, course design, evaluating courses and programs, knowledge management, learning analytics, measuring institutional performance, quality and quality assurance, Strategies, planning and management, Teaching and learning, Technology Tagged With: , , , , ,

Online learning and disruptive change at the UK Open University

May 2, 2018 by 11 Comments

The old Walton Hall on the OU campus in Milton keynes

Sturm und Strang

I’ve was in England last week,  attending the 7th eSTEeM conference at the Open University as the opening keynote speaker, only my second visit to the OU since I left nearly 30 years ago.

The Open University, described by several commentators as one of the most successful innovations in Britain since the Second World War, is currently going through an existential crisis, which culminated two weeks ago with the resignation of its Vice-Chancellor, Peter Horrocks, following a devastating vote of no confidence by faculty and staff.

The OU is facing enormous pressure, due mainly to the policies of the recent Conservative governments. Over the last six years, the government has treated the OU just the same as other, more traditional, universities in England and Wales. The government severely cut the OU’s operating budget requiring it to dramatically increase fees, and also made all part-time students (i.e. students not taking a full annual course load) ineligible for government-guaranteed, low interest loans. It also has required students at the OU, like all other students in England and Wales, to complete their bachelor studies within three years, compressing their time for study. It is expected to have a £20 million (CS$36 million) operating deficit this year and was proposing to save £100m from its £420m annual budget by cutting courses and staff.

Since the vast majority of its 200,000 students in 2012 were part-time, working adults without a first degree and who required the maximum flexibility in their studies, it’s hardly surprising that its student numbers have dropped by more than a third since 2012. At the same time it has invested heavily in FutureLearn, a MOOC-type platform which is still struggling to find a viable business model. The recent changes mean that the whole concept of open-ness and accessibility for OU students, and its unique position in the British higher education system, are under existential threat. 

To cap all this, the university itself recognises that it needs to fundamentally change its operational model. Like many other Open Universities, it has not changed fast enough to accommodate to the digital revolution in post-secondary teaching. It is burdened with a heavy legacy of a print-based design model and an expensive regional tutoring system, despite the recent elimination of all local face-to-face operations.

“We want to transform the University of the Air envisaged by Harold Wilson in the 1960s to a University of the Cloud, a world-leading institution which is digital by design and has a unique ability to teach and support our students in a way that is responsive both to their needs and those of the economy,” according to Horrocks. As a result the (now leaderless) executive team is working on a ‘transformational model’ for the university, which is still a work in progress.

This is the battlefield into which I parachuted this week.

The eSTEeM conference

The Open University has offered science and technology programs since its inauguration in 1971. It initially used a combination of print, home experiment kits mailed to students’ homes, and one week residential schools in the summer. The residential schools have long since gone (too expensive) although in general students loved them and at least in the early days the residential schools provided such a morale boost for students that many who would have dropped out then went on to continue successfully.

For the last seven years, the STEM Faculty/academic department at the OU has been holding an annual conference to demonstrate the scholarship of its faculty and staff. I was the opening speaker for this year’s conference, on the topic: ‘Digital learning in an era of change: challenges and opportunities for STEM teaching and the OU.’

However, as well as the very interesting STEM components of the conference, on which I will write two separate posts, there was an almost full day, well-organised workshop called ‘Digital by Design’, which focused on what the future as a whole should be at the OU. The workshop enabled a quick and close, if incomplete, ‘parachute’ view of some of the challenges the OU is facing and how academic and regional staff are responding. In this post I will focus on these general, internal challenges that the OU still has to resolve that emerged from this and other discussions in which I participated.

Online but not digital

It is clear that many of the teaching staff have not really ‘got it’ with regard to digital learning. In many cases, print still remains the core teaching technology, and where online is heavily used, it is often just a print model moved online, with a heavy emphasis on content transmission. Many in the OU are still arguing for a ‘blended’ learning model, which in this case refers to a mix of print and online, with print having at least an equal contribution.

In particular, the OU is really weak in its exploitation of the networking and student collaboration that the web offers and in its integration of social media within the design of courses. In this it is not unlike many conventional universities, but nevertheless this realisation came as a real shock to me. This was the original open, distance university, not a conventional one.

Why I am so shocked is that one of the many reasons I emigrated to Canada in 1989 was that I got frustrated at the inability of myself and others at the OU such as Robin Mason and Tony Kaye to get the OU to take online learning seriously. We had contributed to a course, DT200, in 1988 that had an online discussion forum component that had merely been bolted on to the standard 36 week print and broadcast design. The next logical step would have been to have pioneered a fully online course, but neither the university management nor the faculty were interested.

It is important to understand that the OU has a relatively small core of permanent faculty based at its headquarters in Milton Keynes who are primarily engaged in the design of courses, in particular the choice and structuring of content, and a legion of regional staff tutors who provide most of the student learning support. There is a long-established Institute of Educational Technology, where the staff have full academic status, and conduct research as well as advise the OU’s course teams on best practices in the design of distance education.

Here I am 30 years later, and there are still arguments going on about the wisdom of going fully online. This despite the fact that Gilly Salmon, who wrote a standard text on teaching online (2011), worked at the OU for several years, and despite the fact that the OU has an Institute of Educational Technology that has excellent design models developed for online learning that it struggles to get faculty to adopt. This is so reminiscent of Athabasca University and its failure to exploit the expertise of Terry Anderson and its other distance education specialists. 

The fixation on print as the ‘core’ medium/technology

I participated in several discussions where I challenged the focus on printed material as the core teaching technology. First though I would like to set out some of the arguments OU staff put forward in support of print.

Arguments for print

These were made mainly by OU staff to me.

  1. The OU made its reputation in its early days in the 1970s by the very high quality of its printed materials. As well as being beautifully produced and illustrated (full colour), they were and still are extremely well structured. This was recognised immediately by many faculty in more traditional universities, and the quality of its printed materials is still much appreciated by the students. If it was effective then, it must be effective now.
  2. Access: there are still students in Britain who do not have access to the Internet or cannot afford a computer.
  3. Most OU students are working and many spend all day at work looking at screens; the OU printed material provides an essential break from being on-screen all day.
  4. Students prefer to read printed material; it’s easier for study purposes and revision than searching online.
  5. If the textual material was delivered online rather than printed, the OU would be transferring the cost of print to the students, as they would want to print out the textual material.

Arguments against print

These were made mainly by me to OU staff.

  1. Online learning provides students with the opportunity of ‘any time, any place’ discussion and interaction with each other and teaching staff.
  2. Student activities and interaction with online text is more integrated and immediate than with printed text. In particular immediate feedback can be provided through online tests or automated feedback, etc.
  3. Students are not limited by the boundaries of the printed course material once they go online. Everything on the Internet is potential study material. In particular students can access open educational resources from many different sources.
  4. In order to develop the skills students need in the 21st century, we need to focus more on skills development than on the transmission of content. Online learning can focus better on the development of these soft skills, such as communication and knowledge management.
  5. Access has always been a limitation for any technology. For instance students with visual impairment or dyslexia have difficulties with print. When the OU first started, many students did not have access to the broadcasts. Most students in Britain now have access to the Internet, although in more remote areas there are still bandwidth limitations. The OU’s policy in general has been that when access exceeds 80% of the target audience, alternatives are found for the remaining students. It is wrong to deny the benefits to the vast majority of students because of the needs of a small minority which could be met in other ways.
  6. Students need to learn digitally if they are to earn digitally. Digital literacy is now a core skill required by everyone.
  7. The costs for a print-based system are very high, not just in the actual costs of full colour printing, but in the editing, and above all, the lengthy time it takes faculty and instructors to prepare, check and revise the printed materials (many OU courses take at least two years to design). Savings by going digital could be used to reduce substantially tuition fees.

The need to think digitally when designing online learning

The issue is not whether print has educational value; it does, and there may be specific situations where students may prefer to have hard copy. However, it should not be the default medium. It’s really important when designing online learning to be open to all the media the Internet enables: text, audio, video, computing, augmented reality, simulations, social media, and so on. This requires thinking digitally when designing courses, which is difficult if your first and preferred option is always print.

Of course, this is identical to the challenge that on-campus instructors face about digital learning, but instead of print, their default option is face-to-face teaching.

This is why moving to online learning requires a major cultural change and why it takes so long. However, in the OU’s current existential crisis, it does not have the time for gradual change (that should have started back in 1989). The need for change must be embraced now, ironically, not for financial reasons but for pedagogical reasons: only this way will it better prepare its students for the future. The financial pressures merely make this devastatingly urgent.

Necessary but not sufficient

Forcing change for financial reasons is unlikely to work. Making changes that are not accepted or resisted by staff is more likely to lead to failure or collapse in an organization. Even if by some miracle the (remaining) OU staff manage to pull it off, moving to the University of the Cloud (whatever that means – some kind of heaven for students?) will not meet the needs of the nation that the former OU met.

Lifelong learning is not a luxury but a necessity in a digital age, where the knowledge base expands exponentially and citizens need to continuously learn new content and new skills. Traditional universities do not do lifelong learning well; they are not really designed for it. The OU was, but government policies of starving financial support for part-time learners and reducing the flexibility of study to fit some 1950s view of elite higher education is going to be disastrous for the future British economy. At no time has the OU been more important to Britain. Without a radical change of government policy though its future is indeed dismal, whatever else it does.

Up next

Your intrepid online learning war correspondent will do two more posts from my visit to the OU:

  • the OU’s use of learning analytics for analysing student course evaluations
  • the OU’s use of online labs

Also I will be reporting on a conference on active learning I attended this week at the Royal Military College, Kingston, Ontario. Buy, busy, busy. (Don’t even ask about retirement).

Reference

Salmon, G. (2011). E-moderating: The key to teaching and learning online (3rd ed.). New York: Routledge.

Filed Under: access, blended learning, Blogs, change management, conferences, costs and benefits, course design, design of virtual learning organizations, developing intellectual skills, Events, innovation, leadership, Monthly Feature, planning, policies and management - institutional, quality and quality assurance, selection of media and technologies, Strategies, planning and management, Teaching and learning, Technology, Tony's Blog, Tweets Tagged With: , , , , ,

‘Making Digital Learning Work’: why faculty and program directors must change their approach

April 15, 2018 by 4 Comments

Completion rates for different modes of delivery at Houston Community College

Bailey, A. et al (2018) Making Digital Learning Work Boston MA:The Boston Consulting Group/Arizona State University

Getting blended learning wrong

I’ve been to several universities recently where faculty are beginning to develop blended or ‘hybrid’ courses which reduce but do not eliminate time on campus. I must confess I have mixed feelings about this. While I welcome such moves in principle, I have been alarmed by some of the approaches being taken.

The main strategy appears to be to move some of the face-to-face lectures online, without changing either the face-to-face or the online lecture format. In particular there is often a resistance to asynchronous approaches to online learning.  In one or two cases I have seen, faculty have insisted that students watch the Internet lectures live so that there can be synchronous online discussion, thus severely limiting the flexibility of ‘any time, any place’ for students.

Even more alarming, academic departments seem to be approaching the development of new blended learning programs the same way as their on-campus programs – identify faculty to teach the courses and then let them loose without any significant faculty development or learning design support. Even worse, there is no project management to ensure that courses are ready on time. Why discuss the design of the online lectures when you don’t do that for your classroom lectures? 

Trying to move classroom lectures online without adaptation is bound to fail, as we saw from the early days of fully online learning (and MOOCs). I recognise that blended or hybrid learning is different from fully online learning, but it is also different from face-to-face teaching. The challenge is to identify what the added value is of the face-to-face component, when most teaching can be done as well or better, and much more conveniently for students, online, and how to combine the two modes of delivery to deliver better learning outcomes more cost-effectively.  In particular, faculty are missing the opportunity to change their teaching method in order to get better learning outcomes, such as the development of high-level intellectual skills.

The real danger here is that poorly designed blended courses or programs will ‘fail’ and it is ‘blended learning’ that is blamed, when really it’s ignorance of best teaching practices on the part of faculty, and program directors especially. The problem is that faculty, and particularly senior faculty such as Deans and program directors, don’t know what they don’t know, which is why the report, ‘Making Digital Learning Work’ is so important. The report provides evidence that digital learning needs a complete change in culture and approaches to course and program development and delivery for most academic departments. Here’s why.

The report

The Arizona State University Foundation and Boston Consulting, funded by the Melinda and Bill Gates Foundation, conducted a study of the return on investment (ROI) of digital learning in six different institutions. The methodology focused on six case studies of institutions that have been pioneers in post-secondary digital education:

  • Arizona State University
  • University of Central Florida
  • Georgia State University
  • Houston Community College
  • The Kentucky Community and Technical College System
  • Rio Salado Community College.

These are all large institutions (over 30,000 students each) and relatively early adopters of online learning. 

The study had three aims:

  • define what ROI means in terms of digital education, and identify appropriate metrics for measuring ROI
  • assess the impact of digital learning formats on institutions’ enrolments, student learning outcomes, and cost structures
  • examine how these institutions implemented digital learning, and identify lessons and promising practices for the field.

The study compared results from three different modes of delivery:

  • face-to-face courses
  • mixed-modality courses, offering a mix of online and face-to-face components, with the online component typically replacing some tradition face-to-face teaching (what I would call ‘hybrid learning)
  • fully online courses.

The ROI framework

The study identified three components of ROI for digital learning:

  • impact on student access to higher education
  • impact on learning and completion outcomes
  • impact on economics (the costs of teaching, administration and infrastructure, and the cost to students).

The report is particularly valuable in the way it has addressed the economic issues. Several factors were involved:

  • differences in class size between face-to-face and digital teaching and learning
  • differences in the mix of instructors (tenured and adjunct, full-time and part-time)
  • allocation of additional expenses such as faculty development and learning design support
  • impact of digital learning on classroom and other physical capacity 
  • IT costs specifically associated with digital learning.

The report summarised this framework in the following graphic:

While there are some limitations which I will discuss later, this is a sophisticated approach to looking at the return on investment in digital learning and gives me a great deal of confidence in the findings.

Results

Evidence from the six case studies resulted in the following findings, comparing digital learning with face-to-face teaching.

Digital learning resulted in:

  • equivalent or improved student learning outcomes
  • faster time to degree completion
  • improved access, particularly for disadvantaged students
  • a better return on investment (at four of the institutions): savings for online courses ranged from $12 to $66 per credit hour.

If you have problems believing or accepting these results then I recommend you read the report in full. I think you will find the results justified.

Conditions for success

This is perhaps the most valuable part of the report, because although most faculty may not be aware of this, those of us working in online learning have been aware for some time of the benefits of digital learning identified above. What this report makes clear though are the conditions that are needed for digital learning to succeed:

  • take a strategic portfolio approach to digital learning. This needs a bit of unpacking because of the terminology. The report argues that the greatest potential to improve access and outcomes while reducing costs lies in increasing the integration of digital learning into the undergraduate experience through mixed-modality (i.e. hybrid learning). This involves not just one single approach to course design but a mix, dependent on the demands of the subject and the needs of students. However, there should be somewhat standard course design templates to ensure efficiency in course design and to reduce risk.
  • build the necessary capabilities and expertise to design for quality in the digital realm. The experience of the six institutions emphasises that significant investment needs to be made in instructional design, learning sciences and digital tools and capacity (and – my sidebar – faculty need to listen to what instructional designers tell them)
  • provide adequate student support that takes account of the fact that students will often require that support away from the campus (and 24/7)
  • fully engage faculty and provide adequate faculty development and training by fostering a culture of innovation in teaching
  • tap outside vendors strategically: determine the strategic goals first for digital learning then decide where outside vendors can add value to in-house capacity
  • strengthen analytics and monitoring: the technology provides better ways to track student progress and difficulties

My comments on the report

This report should be essential reading for anyone concerned with teaching and learning in post-secondary education, but it will be particularly important for program directors. 

It emphasises that blended learning is not so much about delivery but about achieving better learning outcomes and increased access through the re-design of teaching that incorporates the best of face-to-face and online teaching. However this requires a major cultural change in the way faculty and instructors approach teaching as indicated by the following:

  • holistic program planning involving all instructors, instructional designers and probably students as well
  • careful advanced planning, and following best practices, including project management and learning design
  • focusing as much on the development of skills as delivering content
  • identifying the unique ‘affordances’ of face-to-face teaching and online learning: there is no general formula for this but it will require discussion and input from both content experts and learning designers on a course by course basis
  • systematic evaluation and monitoring of hybrid learning course designs, so best (and poor) practices can be identified

I have a few reservations about the report:

  • The case study institutions were carefully selected. They are institutions with a long history of and/or considerable experience in online learning. I would like to see more cases built on more traditional universities or colleges that have been able successfully to move into online and especially blended learning
  • the report did not really deal with the unique context of mixed-modularity. Many of the results were swamped by the much more established fully online courses. However, hybrid learning is still new so this presents a challenge in comparing results.

However, these are minor quibbles. Please print out the report and leave it on the desk of your Dean, the Provost, the AVP Teaching and Learning and your program director – after you’ve read it. You could also give them:

Bates, A. and Sangra, A. (2011) Managing Technology in Higher Education San Francisco: Jossey-Bass/John Wiley

But that may be too much reading for the poor souls, who now have a major crisis to deal with.

Filed Under: Bibliography, blended learning, change management, completion rates, costs and benefits, course design, design of virtual learning organizations, developing intellectual skills, e-learning and distant education, evaluating courses and programs, faculty development and training, General, measuring institutional performance, Monthly Feature, physical space, planning, policies and management - institutional, quality and quality assurance, selection of media and technologies, Strategies, planning and management, Teaching and learning Tagged With: , , , , , , , , , ,

Meeting the challenge of online degrees for the professions

March 22, 2018 by 1 Comment

 

Tina the Avatar from Drexel University’s nursing program. Tina not only responds to questions asked by students but can also be physically examined and will respond according to how she is being treated.

Chatlani, S. (2018) Navigating online professional degrees – potential and caution, Education Dive, March 21

In previous posts, I have pointed out the challenges of getting online qualifications recognised by professional associations, for instance:

The Chatlani article though shows how some institutions have worked with professional associations to obtain recognition.

Which institutions have received recognition from professional associations?

Chatlani looks at several institutions who have succeeded in getting their online professional degrees recognised. These include:

  • Syracuse University College of Law
  • Western Governors University College of Health Professions
  • Faulkner University’s Masters of Science in Counseling
  • The Santa Barbara & Ventura Colleges of Law

To these institutions I would add a couple of Canadian examples:

Some of these programs are not fully online, but are hybrid, with a good deal of online learning however.

How to get online professional degrees recognized

First, it ain’t easy. It’s no good just trying to convert your on-campus content into an online version. You have to do much more to satisfy professional associations – and quite rightly, in most cases.

The biggest challenge is providing a satisfactory online context for experiential learning: enabling students to apply what they have learned in an online environment that is ‘real’ enough to transfer to an actual workplace. Typical examples would be:

  • use of video, computer simulations, and augmented or virtual reality to teach procedures and/or motor (hands-on) skills
  • use of remote labs/equipment that students can manipulate online
  • ‘virtual’ offices, companies or workplace situations that mirror real companies and their work
  • online development of inter-personal skills through one-on-one online monitoring
  • use of synchronous as well as asynchronous delivery: Syracuse designed their law program so that 50% of each online course will be in real time with students and professors interacting just as they would in a residential program, with intense Socratic dialogue in real time
  • on-campus evaluation of specific skills, such as counselling, even if they are taught online.

In addition to providing appropriate experiential learning, there are general quality issues to be addressed:

  • secure validation of student identity and online assessment;
  • investment in ‘best practice’ online course design, which will involve using learning design and learning technology specialists;
  • opportunity for substantive interaction between faculty and students; 
  • close monitoring of student activities;
  • extensive training of faculty in online teaching.

This is rather a daunting list, even if not all of these requirements apply to all professional training.

Will it be enough?

One has to look to motive here for moving online. One motive is a scarcity of professionals (or more likely, a coming scarcity). This is one major reason for Queen’s University’s Bachelor in Mining Engineering. A shortage of professionals pushes up the costs of professionals and  a shortage of professionals may mean that there are unacceptable delays in court cases (as in Canada), for instance. Offering programs partly or wholly online enables those working or with families to study more flexibly and in the end results in a larger pool of professionals.

Another motive is cost: the cost of traditional, on-campus professional degrees is often so high that many who could benefit from such programs are just unable to afford it. The hope is that online programs can bring down the cost without losing quality.

Chatlani interviewed Christopher Chapman​, CEO of AccessLex Institute, a legal education advocacy group, who argued the hybrid degree option is necessary to make becoming a lawyer more accessible and possibly less expensive:

Truly experimentation in legal education is critical to the long-term future of the field and lawyers. This could allow for the development of better pedagogy and allow for scaling where schools may be able to eventually lower their price point.

However, often professional education does not necessarily scale easily as it may require fairly small class sizes if quality is to be maintained. This is not to say there are no economies of scale: once a simulation or a virtual reality environment is created, it can be used many times with many students, but this often means not only a heavy up-front investment, but also a sophisticated business model that allows for return on investment over several or even many years.

It is worth noting that none of the example institutions above are what might be called elite institutions, who have dominated education for professionals in law, medicine and engineering for many years, and whose alumni are often the ones who set accreditation requirements for the professional associations.

And this is the problem. It benefits existing professionals to limit the number of new professionals by making existing labour scarce. If the people who are responsible for accrediting educational programs for professional recognition benefit by keeping the market restricted and themselves come from elite institutions with no experience of online learning, then online professional programs become a huge risk for the departments planning to offer them and for the students who sign up for them.

The best approach is to ensure the support of the relevant professional associations before investing heavily in such programs. The worst case scenario is to spend lots of money on developing such programs only for students to find that they still cannot get a well paid professional job with their qualification.

Filed Under: accreditation, Bibliography, course design, experiential learning, experiential learning, faculty development and training, identity management, planning, policies and management - institutional, quality and quality assurance, remote labs, Strategies, planning and management, Teaching and learning Tagged With: , , , , , , ,

Some thoughts on scaling online and digital learning

February 14, 2018 by 1 Comment

Image: Fortune.com

Chatlani, S. (2018) How to effectively scale a digital learning model, Education Dive, accessed February 13

Bates, A. and Sangra, A. (2011) Managing Technology in Higher Education: Strategies for Transforming Teaching and Learning San Francisco: Jossey Bass, Chapter 7

The sources

The Chatlani article is interesting if a little frustrating, as it is a report on a presentation at a conference of unpublished research (or at least unreferenced in the article) that looks at several case studies of successful scaling of digital learning. (If this research has been published, I would really appreciate access to the report or at least a reference.)

Nevertheless the results reported by two of the researchers, Lou Pugliese, director for the Technology Innovation Action Lab at Arizona State University, and Kate Smith, vice president of academic affairs at Rio Salado College, are really interesting and worth examining.

I have referenced also the research published in Managing Technology in Higher Education on the scaling of the University of British Columbia’s very successful Master of Educational Technology, which is still running today, although originally designed in 2001. The program has undergone a number of major changes over those 16 years but the scaling model has remained largely intact.

It is interesting then to compare the results of the two studies.

The institutional cases

The research reported by Chatlani, funded by the Boston Group and the Gates Foundation, examined the characteristics of digital learning programs from six diverse institutions:

  • Kentucky Community & Technical College System,
  • the University of Central Florida,
  • Georgia State University,
  • Houston Community College,
  • Rio Salado College and
  • Arizona State University. 

These institutions include some of the largest public post-secondary online providers in the USA. UCF was also one of the cases examined in Bates and Sangra.

Results

Pugliese and Smith reported the following four key findings from the study:

  • take a strategic portfolio approach to digital learning. This reflects the UBC MET program, which was developed as part of an institutional strategy to move towards program-based online learning. In fact several other graduate programs using the same model were developed at UBC around the same time;
  • build capabilities and expertise to design for quality in the digital realm. This also reflects the UBC strategy. The MET program was originally developed as a partnership between the Faculty of Education and the Distance Education Unit of UBC’s Continuing Studies department, which provided project management, instructional design, and media production support. Several years later, UBC moved the technical and educational support from Continuing Studies into a central Centre for Teaching, Learning and Technology, which also incorporated faculty development, to serve the whole university. The Faculty of Education also developed its own learning technology support team.
  • provide the differential student support to succeed in fully online learning. This was a critical component of the MET model. The key here is the difference between fixed and variable costs. Course development costs are mainly fixed; course delivery costs are mainly variable, as they are driven primarily by student numbers. The key savings in scaling comes from the distribution of fixed costs across increasing numbers of students and the lower costs of using adjunct faculty. In the MET model, tenured faculty were responsible for the content and design of the courses and in some cases for online teaching of at least one section. However, most of the delivery was supported by a team of adjunct faculty supervised by tenured faculty members. Student fees (at the same rate as for on-campus courses) more than covered both the costs of development – including the hiring of the necessary extra tenured research faculty – and delivery. Scaling was possible because of the lower cost of adjunct faculty but working to a quality model of delivery that kept student-instructor ratios at 30 or less.
  • engage faculty as true partners, equipping them for success. This was also an essential element of the UBC MET model which engaged faculty from Education from the start. Everything went through normal faculty quality assurance processes, involving a total of 27 faculty consultation meetings over a period of two years before the program even started. Perhaps more importantly, the business model ensured that the bulk of the revenues went directly to the Faculty of Education, which then paid overheads for the program to the central administration. Any profits from the program were ploughed back into faculty hirings. Thus the academic department was rewarded for innovation as well as for its efforts.

Discussion

As the article points out, online learning has been around now for 20 years or more and it is timely to look at what models have been successful in scaling quality online learning – and those that have not. 

The research suggests that scaling with quality requires a delicate balance between:

  • team work involving tenured faculty, specialist online experts such as instructional designers and media producers, and adjunct instructors, with full involvement of faculty in all aspects of the design and development of the programs,
  • using adjunct faculty as instructors to support program delivery as the enrolments grow,
  • managing student-instructor ratios so that adjuncts are not overloaded,
  • ensuring the adjunct instructors are adequately trained or experienced in teaching online.

Other important factors in scaling with quality are:

  • being sure there is an adequate market demand to justify the scale of online/digital programs you are proposing: good market research is essential,
  • being confident that new entrants into the market will not have the scale or quality to capture your market,
  • being sure that there is a sufficient pool of available qualified adjunct instructors,
  • developing a multi-year business plan that will accommodate losses in the first two years in return for later economies of scale and scope,
  • a sympathetic and creative administration that will consider and encourage new funding models.

I look forward to the publication of the report and hope it will be widely disseminated.

Filed Under: Bibliography, costs and benefits, course design, learner support, Monthly Feature, planning, policies and management - institutional, quality and quality assurance, Strategies, planning and management, Teaching and learning Tagged With: , , , , ,
Next Page »