Carey, T., & Trick, D. (2013). How Online Learning Affects Productivity, Cost and Quality in Higher Education: An Environmental Scan and Review of the Literature. Toronto: Higher Education Quality Council of Ontario.
Tom Carey is one of the authors of the above study, and as an example of the best of reflective practice, he has kindly provided his thoughts about the report, now that it is finished. His first thoughts were published yesterday. This is the second part.
Tom Carey: Part II of Reflections on Researching and Writing on Emerging Developments in Online Learning
In yesterday’s guest post, I provided some reflections on the process and product of a research project for the Higher Education Quality Council of Ontario (HEQCO): How Online Learning Affects Productivity, Cost and Quality in Higher Education: An Environmental Scan and Review of the Literature. That post described the Results that Surprised in the report, from my perspective as an author. Today’s post provides some insights about what we chose to not include in the report, following the old advice of expert film editors that the most interesting scenes in a movie may be those “left behind on the cutting room floor”.
Developments We Didn’t Include
There were some emerging developments on our original target list for which we could not find compelling examples at scale: Semantic Web, Mobile Learning, Ubiquitous Connective, etc. I am sure these are going to be important, but in the interests of preserving the Teachable Moment aspects we focused only on developments with convincing data for impacts on learning outcomes and productivity: convincing in the context of Ontario higher education institutions. For example, the Ithaka study of the Open Learning Initiative software allowed us to highlight Adaptive Learning Systems at scale (and the recent follow-up book by William Bowen contains several other insights we could cite if we were starting now).
Similarly, the report only deals with Open Educational Resources as a sideline in the discussion of Open and Affordable Textbooks: the rationale was that the textbook developments were a hot topic in “peer” higher education systems ‒ British Columbia, California, New York, etc. ‒ represented low hanging fruit in terms of potential for building collaborations amongst students, faculty and academic leaders within institutions. And we didn’t do any justice to the Canadian developments in connectivist MOOCs, mostly because we had our hands full trying to help our target audience make sense of the instructionist MOOCs that were hogging the headlines and couldn’t work out how to get beyond that without losing their attention. (I have already apologized to George for this: Stephen, Dave et al can consider themselves included in the apology.) These choices about which innovations to highlight may have bypassed the disruptive in favour of the radical, but helping decision makers to make sense of – and act on – opportunities for radical change was more than enough for us to bite off.
Issues We Couldn’t Include
Some of the content we wrote but could not include in the report was just not clear enough or complete enough to be included in the public document. For a few topics, we were keenly aware that more work had to be done but that we had not made sense of what that work might be. We didn’t want to go on at length in the report about these points for several reasons: calling for further research sounds like too familiar an ending to a Research Report, including more than a quick mention for what is not yet clear seemed to detract from our Teachable Moment goal, and some of the further exploration needed would be an outcome of our proposed Call for Action through collaborative sense-making across institutions. For those interested in ‘where to next’ in terms of understanding the impact of emerging developments, here is my personal list of high priority issues that need more clarity.
The different roles of various online learning interactions in various contexts: I would like to have referenced the work by Terry Anderson and others on how increases in one form of learning interaction can result in a decreased need for another type of interaction. This was implicit in our Call to Action around understanding and leveraging scalability: use more scalable interactions where appropriate in order to redirect resources – especially time – into other interactions which are less readily scaled.
Here is my current woefully incomplete attempt to reframe our analysis of emerging developments in online learning using different types of learning interactions – whether online or not:
- Learner-content interactions can be used effectively to advance Quality and Productivity for technical mastery outcomes, e.g., performance tasks with single solutions and predictable pathways to completion (allowing adaptive systems to provide task guidance)
- Learner-learner interactions can be used effectively to advance Quality and Productivity for (some) of the question-and-answer and formative feedback roles traditionally carried out with learner-instructor interactions, and seem to be essential (at the moment?) for outcomes involving complex challenges with diverse ways of knowing.
- Learner-instructor interactions appear to be essential for outcomes involving deep personal change related to learning itself: grappling with threshold concepts, enhancing learning mindsets and strategies, and ‘getting better at getting better’ for knowledge-intensive work
- Learner-expert interactions are required for formation of learners’ identity and practice as members of knowledge-building communities, whether in professional/career contexts or in their roles as community members and global citizens.
Much more work to be done in this area, including ensuring that the outcomes listed above that are not readily scaled don’t get left out in the quest for greater productivity: if we neglect such outcomes, where would the ‘higher’ be in higher education?
Institutional productivity gains may be possible @ scale in traditional institutions: you may have noticed that the list of interactions above has unbundled the role of “instructor” (who can apply expertise in pedagogical content knowledge) from the role of “practice expert” who can help learners transition into full engagement with knowledge-practice networks. Traditional institutions may struggle to unbundle such roles, or even to respect their differing contributions.
Traditional institutions – and in Ontario higher education, that is all we have at the moment ‒ may also struggle to reinvest the results of productivity gains from online learning beyond a course context. As long as we think of ‘workload’ in terms of ‘courses taught’, any savings in effort may disappear into other localized activities. How can we reframe the work, and workload, of teaching to optimize educator and learner time, without resorting to an alien ‘managerial’ language? (Mention “activity-based-costing” in a budget meeting and the challenges to reinvesting productivity gains become all too clear J.)
I tried adapting the idea of Constructive Alignment from instructional design, with an expansion into “Productive Alignment” where educators also include in their designs the goal of optimizing resource usage. If certain students can achieve certain learning outcomes with reduced learner-instructor interactions, e.g., with MOOC resources used in a hybrid course format, then effective instructional design requires that we achieve this Productive Alignment to optimize time and resources. I couldn’t explain this notion effectively enough to include it in the report, but I am convinced that some such changes in the ways we talk about educator roles and responsibilities are going to be needed if the full potential of online learning is to be achieved. And this is going to be both more necessary and more difficult with the ‘higher’ learning experiences and outcomes listed above, which develop slowly over the course of a program and are not readily described as discrete competencies to be tested in a short-term performance task.
Dealing with Quality and Productivity in tandem is a fiscal, political and pedagogical necessity: finally, I wish we had been able to make a better case for the pedagogical rationale for dealing with Scalability, Quality and Productivity issues in parallel. We did include some rationale for determined action now on systematic collaborations across institutions to understand and leverage the emerging potential of online learning.
However, that argument was framed mostly in terms of fiscal and political realities. The fiscal reality across higher education systems requires that we get more focused on deploying the least resources to achieve the highest level of outcomes, and the political reality requires that we in public higher education either ‘do or be done to’ in our dealings with Quality and Productivity.
But there is another rationale that is more closely linked to our purposes and ideals in higher education. Even if we did not have our current fiscal constraints and the expectations of stricter constraints in the future…even if public higher education had the full confidence of political leaders as to our ability to change and adapt to our changing circumstances…if our students see us clinging to traditional practices and structures rather than taking on our challenges with boldness and confidence, what model are we presenting to them about how to deal with challenges in their workplaces, their families and communities, in the earth’s environment and the global knowledge economy? Will our plea for engagement with the knowledge and wisdom of the past, present and future fall on deaf ears if we don’t practice what we preach?
Making that case was beyond our reach in this project, but it remains on my personal to-do list. I keep thinking of Parker Palmer’s concise formulation in The Courage to Teach: “How we teach is a critical part of what we teach”. That is the pedagogical rationale for our taking charge of higher education’s fate by applying emerging knowledge and wisdom about online learning…with care, compassion and courage.
One link I left out of this post: the work by Terry Anderson et al on the potential tradeoffs between the various interactions in online learning (or any other formats) is summarized at http://equivalencytheorem.info/.
Thanks for these two posts. Great stuff and I couldn’t agree more with the need to develop learning tools and systems with higher productivity – AND quality. I think part of the reason that MOOC producers are maiNly American, is due to American higher education pricing itself “out of the market”. The reason that most industries adopt higher productivity tools and techniques is because they have to – if they don’t their competitors do.
I apprecIate the call out to my work and the http://equivalencytheorem.info/ site.
And as you imply, it is difficult to say what types of interaction are most effective for particular types of learning (much less other variables like culture, level, discipline ETC. But you tried as below:
Learner-content interactions can be used effectively to advance Quality and Productivity for technical mastery outcomes, e.g., performance tasks with single solutions and predictable pathways to completion (allowing adaptive systems to provide task guidance)
I THINK WE ARE JUST BEGINNING (DESPITE YEARS OF EFFORT) IN UNDERSTANDING THE WAYS IN WHICH CONTENT CAN BE USED TO INTERACT. CERTAINLY, LEARNING TO SUCCEED IN ONLINE GAMES, INDUSTRIAL SIMULATIONS, PUZZLES, LEARNING TO PLAY A PIANO AND OTHER TASKS ARE AND CAN BE DONE THROUGH HIGH QUALITY LEARNER-CONTENT INTERACTION – AND WHO HASN’T LEARNED ANYTHING FROM READING A BOOK. SO, I WOULD EXPAND THE POSSIBILITIES FAR BEYOND “PREDICTABLE OUTCOMES”
Learner-learner interactions can be used effectively to advance Quality and Productivity for (some) of the question-and-answer and formative feedback roles traditionally carried out with learner-instructor interactions, and seem to be essential (at the moment?) for outcomes involving complex challenges with diverse ways of knowing.
I AGREE WITH THESE OUTCOMES, BUT YEARS OF WORK WITH BOTH COLLABORATIVE AND COOPERATIVE LEARNING PROVIDES PRETTY SOLID EVIDENCE OF THE MOTIVATIONAL, SOCIAL AND (AS YOU STATE) COGNITIVE AFFORDANCES OF PEER INTERACTION
Learner-instructor interactions appear to be essential for outcomes involving deep personal change related to learning itself: grappling with threshold concepts, enhancing learning mindsets and strategies, and ‘getting better at getting better’ for knowledge-intensive work
CERTAINLY THE STUDENT-TEACHER INTERACTIONS YOU LIST ARE IMPORTANT AND NOT EASILY MET WITH OTHER FORMS OF INTERACTION. I WOULD ADD HUMAN CAPACITY TO DIAGNOSE BARRIERS AND SOCIAL BARRIERS – EVOLUTIONARY STUFF WE HAVE LEARNED THROUGH GENERATIONS OF TRIBAL LIVING. I WISH THERE WAS CLEARER KNOWLEDGE HOW MUCH OF TEACHER-STUDENT INTERACTION CAN BE CAPTURED AND PLAYED BACK – THUS CREATING STUDENT-CONTENT INTERACTION. IS THE STAGE PLAY REALLY THE SAME AUDIENCE INTERACTION AS THE MOVIE?? IS IT JUST AS POWERFUL?
THANKS AGAIN TOM (AND TONY)
Many thanks, Terry (and Tom) for these comments.
I will be discussing in more detail areas where I think there are possibilities for scaling – and where there are not or where we should avoid scaling.