April 25, 2014

How online learning is going to affect classroom design

Listen with webReader
Steelcase Node Classroom

Steelcase Node Classroom Design (© Steelcase, 2013)

I have an unusual occupation. A couple of weeks ago I attended a meeting in an office furniture warehouse in a small industrial park in Vancouver. The purpose of the meeting? To explore how online learning is going to change the design of campus classrooms.

The meeting was called by Steelcase, a leading American manufacturer of office and educational furniture. It came as a bit of a shock to me to realise that this large American manufacturer was not only conducting impressive research into learning environments, but is way ahead of many of our post-secondary institutions in thinking through the implications of online learning for classroom design. Don’t take my word for it: go to their educational research website, and in particular look at two of their reports: Active Learning Spaces and 360°: Rethinking Higher Education Spaces.

Designing active learning spaces

In Active Learning Spaces, Steelcase reports:

Formal learning spaces have remained the same for centuries: a rectangular box filled with rows of desks facing the instructor and writing board….As a result, today’s students and teachers suffer because these outmoded spaces inadequately support the integration of the three key elements of a successful learning environment: pedagogy, technology and space.

Change begins with pedagogy. Teachers and teaching methods are diverse and evolving.  From one class to the next, sometimes during the same class period, classrooms need change. Thus, they should fluidly adapt to different teaching and learning preferences. Instructors should be supported to develop new teaching strategies that support these new needs.

Technology needs careful integration. Students today are digital natives, comfortable using technology to display, share and present information. Vertical surfaces to display content, multiple projection surfaces and whiteboards in various configurations are all important classroom considerations. 

Space impacts learning. More than three-quarters of classes include class discussions and nearly 60 percent of all classes include small group learning, and those percentages are continuing to grow. Interactive pedagogies require learning spaces where everyone can see the content and can see and interact with others. Every seat can and should be the best seat in the room. As more schools adopt constructivist pedagogies, the “sage on the stage” is giving way to the “guide on the side.” These spaces need to support the pedagogies and technology in the room to allow instructors who move among teams to provide real-time feedback, assessment, direction and support students in peer-to-peer learning. Pedagogy, technology and space, when carefully considered and integrated, define the new active learning ecosystem. 

For some of us, this is nothing new. I was teaching this way in a primary/elementary school in Britain in the 1960s. What is new is the addition of (modern) technology to the mix. With students now doing an increasing amount of their work online (and often outside the classroom), the classroom needs to take into account this fact. This means opportunities for accessing, working on, sharing and demonstrating knowledge both within and outside the classroom. Thus if the classroom is organized into ‘clusters’ of furniture and equipment to support small group work, these clusters will also require power so students can plug in their devices,wireless Internet access, and the ability to transmit work to shared screens around the room (i.e. a class Intranet). Students also need quiet places or breakout spaces where they can work individually as well as in groups.

This still assumes that students are learning in relatively small classes. However, we are seeing an increased move to redesign large lecture classes by moving them towards hybrid designs such as flipped classrooms. Given the current financial context, I don’t think we should assume that the classroom time for these redesigned large lectures classes will be spent in small groups in individual classrooms (there are probably not enough small classrooms to accommodate these classes which often have several thousand students). We may need larger spaces which can be organized into smaller working groups then easily reconvened into a large, single group. What the space for these large classes certainly should NOT be is the raked rows of benches that now are now the norm in most large lecture theatres.

What about faculty space?

Steelcase is also doing research on appropriate spaces for faculty. For instance, if a university or department is planning a learning commons or common area for students, why not locate faculty offices in the same general area rather than in a separate building? Indeed, a case could be made for integrating faculty office space with more open teaching areas.

The cost of change

You can see why a company such as Steelcase is interested in these developments. There is a tremendous commercial opportunity for selling new and better forms of classroom furniture that meets these needs.

However, that is the problem. Universities and colleges simply do not have the money to move quickly towards new classroom designs, and even if they did, they should do some careful thinking first about what kind of campus will be needed over the next 20 years, given the rapid moves to hybrid and online learning, and how much they need to invest in physical infrastructure when students can do much of their studies online.

Nevertheless, there are several opportunities for at least setting priorities for innovation in classroom design:

  • where new campuses or major buildings are being built or renewed
  • where large first and second year classes are being redesigned: maybe a prototype classroom design could be tried for one of these large lecture redesigns and tested; if successful the model could be added slowly to other large lecture classes
  • where a department or program is being redesigned to integrate online learning and classroom teaching in a major way; they would receive priority for funding a new classroom design
  • all major new purchases of classroom furniture to replace old or worn out equipment should first be subject to a review of classroom designs.

Conclusions

The important point here is that investment in new or adapted physical classroom space should be driven by decisions to change pedagogy/teaching methods. This will mean bringing together academics, IT support staff, instructional designers and staff from facilities, as well as architects and furniture suppliers.

Second, I strongly believe in the statement that we shape our environments, and our environments shape us. Providing instructors with a flexible, well-designed learning environment is likely to encourage major changes in their teaching; stuffing them into rectangular boxes with rows of desks will do the opposite.

What is clear is that institutions now need to do some some hard thinking about online learning, its likely impact on campus teaching, and above all what kind of campus experience we want students to have when they can do much of their studying online. It is this thinking that should shape their investment in desks and chairs.

See also: The beginning of the end of the lecture hall?

Questions

If you have been involved in any new designs for classrooms or learning spaces, please share your experiences. In particular:

  • to what extent was online learning a factor in the (re)design?
  • to what extent were instructors or instructional designers involved in the (re)design? Did this make a difference or were their views ignored?
  • are instructors making full or best use of the (re)design? If not, why not? If so, how?

A review of the HEQCO report on productivity and quality in online learning in higher education

Listen with webReader

The view from HEQCO, Toronto

The view from HEQCO, Toronto

Carey, T., & Trick, D. (2013). How Online Learning Affects Productivity, Cost and Quality in Higher Education: An Environmental Scan and Review of the Literature. Toronto: Higher Education Quality Council of Ontario

Why this paper is important

In July, the Higher Education Quality Council of Ontario published the above report. This is a very important development for online learning in post-secondary education as it takes a very hard look at quality, cost and productivity and comes forward with recommendations to government. This is a paper that is likely to be read (and should be read) by legislators, state and government policy makers, university and college boards and senior university and college administrators.

I am also exploring through a series of blogs the issue of productivity and online learning, partly because of dissatisfaction with the current state of thinking about this issue, which became apparent working with this project.

For this reason, I am setting aside my hat as an Advisory Board member who commented on the penultimate draft, and and am here providing a full analytic review of the paper. To do this, I have had to reproduce key parts of the document, but I strongly recommend that the HEQCO document is read in full. Quotes from the actual paper are in italics, although I have edited and abbreviated in part.

The paper focuses on the following questions:

  • What are the cost implications of a shift to online learning? Specifically, does a greater use of online instruction save institutions or systems money and, if so, under what circumstances?
  • What do we know about the relationship between online learning and important variables that are often considered when discussing the “quality” of an institution or of a system?

Main findings

  • The evidence reviewed suggests that, for a range of students and learning outcomes, fully online instruction produces learning that is on par with face-to-face instruction.
  • the students most likely to benefit are those who are academically well prepared and highly motivated to learn independently. Students who are not well prepared to learn at the postsecondary level or do not devote the necessary time to learning are less likely to benefit from online learning and may in fact do better in a face-to-face setting.
  • the provincial government… should have an interest in making sure [well-prepared and motivated students] have online learning opportunities available to them. These opportunities should serve students’ learning needs, and – if carried out at large scale – should produce cost efficiencies for higher education institutions, the student or both.
  • there is no evidence that all of the learning outcomes expected of postsecondary students in Ontario can be achieved solely by online learning. 

Main recommendations to the Ontario provincial government and Ontario universities and colleges

  • set a target that, within three years, a specified list of high-demand university and college programs that are primarily or entirely online will be available to Ontario students.
  • set a target that, within three years, a specified list of high-demand courses will be available online and will be accepted for credit at all Ontario universities and colleges that offer a program in that discipline.
  • a set of high-quality degree programs that qualify the student for admission to any Ontario graduate school, and a set of high-quality courses that are accepted for credit by every Ontario institution, will be preferable to a multiplicity of courses and programs that operate on a small scale.
  • By working with other institutions in Ontario and elsewhere, Ontario colleges and universities can leverage and help shape emerging developments in online learning.
  • Coordination will be required to ensure that economies of scale are achieved in an environment of rapid technological change. 
  • Ontario colleges and universities should be encouraged to work with peer institutions to ensure that engagement with advances in online learning fully supports the province’s strategic goals for quality and access in a time of constrained funding. 
  • An effective government strategy will begin by adapting existing regulatory infrastructure to remove unnecessary barriers to high-quality online education. 
  • Hybrid courses that blend online learning with face-to-face instruction should also be encouraged where they improve learning outcomes. Hybrid courses fit well within the government’s existing regulatory structure and so present fewer policy challenges. 

Methodology

We have looked especially for meta-analyses which compare traditional versus online education at a system, course or activity level. We have made only secondary use of studies and reports from individual instances or instructors where institutionalization and sustained use have not been addressed.

Costs:

There is remarkably little empirical literature that documents the costs of online education relative to face-to-face education. So very little evidence on costs is available in this report

The authors though do provide an extensive list of barriers to cost reduction.

The authors conclude this section as follows:

To the extent that online education reduces costs, there is no consensus about who should or would benefit from the reduction. Students seek lower tuition fees; governments seek reduced subsidies for higher education; university employees seek better compensation. This situation presents a principal-agent problem: it is difficult to motivate change when those affected by change will not receive the contemplated financial benefit.

Emerging developments

The following emerging developments are discussed:

  • Affordable and open textbooks
  • Adaptive interactions with learning resources
  • Optimizing student-instructor interaction time
  • Targeting instructional effort based on student program data
  • Minimizing marginal costs via Massive Open Online Courses

The authors also identify several common themes across the individual developments:

  • Aligning Support to the Student’s Individual Learning Needs
  • “Thinking Globally, Acting Locally” to Achieve Benefits at Scale
  • Transparency and Knowledge Intensity in Instructional Design
  • Reputational Capital From and For Online Learning
  • The Challenge of Investment at/for Scale

Observations (for recommendations, see above)

Fully online education presents opportunities for major economies of scale. By definition, these economies can only be achieved if a large scale is reached.

Fully online education has the potential to provide a high-quality education – for some students, in some fields of study – at significantly lower unit costs than traditional forms of instruction. The cost savings have the potential to help fund the cost of improving traditional learning, including the costs of introducing hybrid models that lead to better learning outcomes. The challenge is to make it happen.

What is striking to us about these viewpoints is the agreement that what is least likely to be done effectively at scale and with technological mediation is precisely what matters most in higher education.

My critique

Overall, this is an excellent report that will be valuable to policy makers, if they read it in full. The danger is that they will jump to the recommendations, which are not really the strength of this report. Its value lies in exploring assumptions and beliefs about online learning and productivity and providing data and evidence that sometimes supports such beliefs, and other times challenges them. The section on emerging developments is particularly strong, especially the analysis of common themes across the individual developments.

Comparative quality of online learning

Although the authors focused their literature review on ‘meta-analyses of rigorous experimental studies’, the result is a master lesson on why such studies are usually a waste of time, particularly with regard to ‘quality’ defined in this report as to whether online learning achieves equal learning outcomes to face-to-face teaching. Such studies on using different media and technologies to deliver education date back until the early 1970s, and results are consistent: mode of delivery is less important than method of teaching and multiple other factors. In statistical terms, variance within experimental groups is larger than variations between experimental groups. In plain language, the pedagogy matters, a point recognized by the authors later in the document when they acknowledge the importance of instructional design.

This is one reason why I am cautious about the research on ‘non-traditional’ students that suggests that online learning works less well for them. While I do not disagree with this in general, it can work well for some in this group when designed to meet their specific needs. The problem is that the Jaggars and Di Xu research quoted to support the conclusion in the HEQCO report is based on data from U.S. community colleges, many of which have a very poor record of using instructional design and best practices in online learning. You have to look at the quality of the teaching (in both modes), not just the delivery method.The HEQCO authors also correctly note that while many of Jaggars/Di Xu findings point to performance differences between online and face-to-face learning that are statistically significant, the differences are fairly small.

Costs

This is by far the most disappointing part of the study. The report draws on only two actual studies of the costs of online learning (both from the USA), neither of which are very helpful.

For reasons of time pressure and consistency, the authors decided to limit their research review to studies published in the last five years. As a result, studies such as my own on the cost of the University of British Columbia’s fully online Master in Educational Technology (which was originally published in 2003) are not included, even though the study provides a comprehensive analysis of the costs and more importantly the cost structures of a program that is still running on much the same cost basis as in 2003. This program has been remarkably successful with the following features:

  • fully cost-recoverable (including overheads and planning) from tuition fees alone
  • tuition fees the same as for on-campus graduate programs (fee level regulated by government)
  • over 300 students in the program each year with over 900 course enrolments
  • courses can be taken and paid for individually
  • 70-80 admissions a year, and 70-80 graduates a year, thus with a degree completion rate (for those enrolling in the full degree program) of over 90%.

This program alone has more than doubled the number of graduate students in the whole of the Faculty of Education and UBC has adopted this cost model for a number of its other professionally based masters programs, such as rehab science and creative writing. Not to include this because the study was done 10 years ago is almost perverse, because it shows that for certain kinds of courses, and certain kinds of students, online learning can be far more productive than face-to-face teaching. It is perverse, because real productivity gains only become apparent over time – a five year window is often too small to see the full benefits.

Emerging developments

For me, this was by far the strongest part of the paper, particularly the analysis of common themes across the developments. The paper is worth reading for this section alone.

Recommendations

Although I would support all the recommendations, they are very cautious.Partly because of the weakness or lack of research into online learning, costs and productivity, the recommendations necessarily have to be cautious.

However, since HEQCO itself is a government-funded policy research organization, perhaps an obvious recommendation would have been for more research on the costs of online learning, given the paucity of studies. Another area for research would be on institutional barriers and government policies that prevent greater scalability or adoption of online learning in Ontario universities.

It is still shocking to me that Ontario has such a poor system of credit transfer even between universities that make it almost impossible to set up consortium programs or enable student students to select combinations of courses/programs from different universities, given that a main advantage of online learning is that students could take courses from any university in Ontario. Maybe government regulation is necessary in this area, since the universities and colleges were given $65 million I believe over a year ago to solve this problem and haven’t done so yet.

None of the recommendations really addresses the issue of scale. I’m not sure I agree with the statement on p. 43:  What is striking to us about these viewpoints is the agreement that what is least likely to be done effectively at scale and with technological mediation is precisely what matters most in higher education, i.e. modelling, coaching, enabling students to construct knowledge, etc. Certainly, most MOOCs don’t do this, but it doesn’t necessarily mean that with a focused effort on instructional design, we could not design more cost-effective, high quality learning experiences through online programs on a larger scale than at present but not necessarily at massive level. This would combine lower cost per student with higher quality learning: the Nirvana of educational productivity

Thus I would like to have seen a recommendation to government and the institutions to put in the same level of investment as for MOOCs, but to develop a model that combines best practices in online learning combined with new technologies such as social media, to build partly self-supporting student learning communities on a larger scale than current campus-based programs, with high quality learning outcomes and completion rates. I think it could be done, but it needs substantial investment beyond the risk level of most individual universities, which is why government should be a partner.

Conclusion

Despite my criticisms this is an excellent report on a difficult topic and completed within a tight timeframe. It provides grist for productive discussions on costs and quality and really advances our understanding of the challenges of increasing productivity without losing quality in higher education.

Reference

Bates, A. and Sangra, A. (2013) Managing Technology in Higher Education San Francisco: Jossey-Bass (especially Chapter 7: Resources, Money and Decision-Making).

 

Is there a link between flexible access and ‘productivity’ in higher education?

Listen with webReader

© Dr. Scott Simmerman, 1993

© Dr. Scott Simmerman, 1993

This is the third in a series of posts about ways in which learning technologies and online learning could improve educational productivity in higher education. The first two posts were:

In response to my first post on this topic, Stephen Downes commented:

Minimally, what we need is a model of educational productivity, not a theory, so we can get a sense of some of the complexities involved. Ideally, we’d get a model suited to the 21st century, not something cooked up by some TQM economists, a model that weighs multiple competing interests, variable outcomes, and competing theories of value.

Well, Stephen, I completely agree – I’ll do my best.

The challenge

In this post I begin by asking a difficult but essential question: how should we measure productivity in higher education? The reason for the question is that we need to be clear about what we are trying to achieve through the use of learning technologies, particularly in terms of outcomes, since productivity is an attempt to improve outcomes at the same or less cost, in this particular instance through the use of technology.

Above all, we need to be careful not to focus on what is easily measurable at the cost of excluding more important outcomes that are more difficult to measure. Also, I am asking you to make a value judgement about what is considered to be the role and function of higher education, and technology’s tole, and there will inevitably be major differences of opinion here, hence the conditional ‘should.’

Lastly, this discussion could easily become very abstract. It’s important not to lose sight of the pragmatics of how learning technologies can be used to bring about changes in university and college teaching, so this should be our starting point. As a result, I’ll start by looking at the possible link between flexible access and productivity.

What are the goals of e-learning?

In my view, unless you are someone whose job depends on the use of learning technologies, technologies and tools are means to an end, not goals in themselves. My hammer in the tool box has no inherent purpose. I may use it to bang in some nails, but even banging nails into wood isn’t a goal in itself.

So I worry when I see institutional plans that set out goals such as 25% of all classes will be hybrid by 2015, or we will have ten fully online masters programs by 2020. It’s like saying I’m going to knock in 10,000 nails over the next five years (or will use a power hammer instead). These may indeed be useful steps in the process of getting somewhere, but we need to be clear about what we are trying to achieve by moving more classes online. One goal of course may be to improve productivity (although it rarely is stated this clearly) – but that still requires a definition of what we are trying to be more productive about.

We shall see that goals in education are like Russian dolls: inside each goal is a set of sub-goals, and within each sub-goal is another set of sub-goals, except that often in education we often start with the smallest sub-goal, then build out to other goals as a post-hoc rationalization for what we are already doing.

So let’s look at some of the educational goals or outputs commonly associated with the use of technology, and then see how these might be measured in some way. In this post, I will focus on just one goal, flexible learning, but will address other goals in later posts.

Increase flexible access to learning

This has been the clearly stated goal of several governments. This goal is in two parts.

1. Using online learning to increase post-secondary participation rates

Thus in Ontario, the government has set the following goals:

  • ensuring space for every qualified Ontario student by creating an additional 60,000 spaces in the system
  • creating the conditions to reach a 70% [post-secondary] attainment rate among Ontario’s adult population.

There are two ‘higher order’ goals behind these more specific goals: equitable access (every qualified Ontario student), and economic development (ensuring that Ontario has a highly qualified work-force that can compete in an increasingly knowledge-based economy). The government could of course attempt to achieve these goals without online learning, merely by building more campus-based institutions. However, the rationale for investing in online learning is that to get to the 70% participation rate, the government has to reach potential students for whom campus-based education is difficult, such as working adults. Indeed, the Ontario government is urging all post-secondary institutions to increase their use of online learning, as a way to increase access.

Thus one simple measurement of this goal is: does the investment in online learning result in a higher participation rate than would have been achieved by an increase in campus-based teaching? Stating the measurement is one thing, though; actually working out how to do the measuring is another. How do we know that if more campuses were built, they wouldn’t reach these students? We are left with more indirect ways of measuring:

  • do we get more older students in online courses? There is fairly strong evidence to date that suggests we do, which suggests online learning is more attractive to lifelong learners. However, we need to also track the demographics of on-campus students, as it seems that these are getting older as well.
  • do we get more students in online courses who are working full-time or even part-time?  Again, there is fairly strong anecdotal evidence that this is the case, but reliable comparative data is in short supply.
  • do we get more students in online courses from other groups previously under-served by campus-based post-secondary education, such as aboriginal students, students with disabilities, new immigrants, etc.? If there is evidence of this kind, I have not seen it, and indeed for some of these groups, online learning may actually be a disadvantage. However, we don’t know, and need to measure this in some way.

My point is that using online learning to increase access to post-secondary education is a feasible, measurable goal. In absolute terms, it would be nice of course to link increased participation rates to increased economic productivity, but a more measurable goal is an actual increase in participation rates that can be directly linked to an increase in online learning activities. Since there is now a good deal of evidence that outcomes are similar between online and campus-based courses, where best practice is followed, then online learning increases productivity by bringing in additional students/learners who would not be able to take campus-based courses. Whether or not this can be done at the same or less cost online is not a factor for this goal, which is a higher participation rate.

I have suggested that the productivity of this approach has been partly measured, but more could be done by institutions through better tracking of student demographics and linking them to mode of delivery (and even more importantly, making such studies publicly available). What we can’t do yet, because no-one has done the study, is to say that online learning has led to an increase of, for instance, 5% per annum in the number of student enrollments over what would have been achieved by increasing campus places, or that a specific sub-group, e.g. new immigrants, that had been mainly excluded is now participating in greater numbers because of online learning. If this is a goal, though (which I think it is), we should be measuring its success.

2. To shorten time to completion of a qualification

The second aspect of measuring the productivity of flexible access is more problematic. This is the rationale that blended/hybrid learning and/or fully online learning is more convenient for existing students, and thus will result in faster completion of qualifications or better grades (I will cover the goal of better grades in another post). This rationale is based on the notion that most so-called full-time students are actually working at least part-time, because of the high cost of tuition. This results in students missing classes, or taking longer to graduate. By providing more flexible access, students can better fit their studies into their busy lives.

Again, there is some evidence to suggest that increased time spent working as well as studying is associated with longer time to completion. We know that in many U.S. universities, students are now taking up to seven years to graduate from a four-year baccalaureate program (put another way, only 35% of students in the USA are completing within four years). This  has been correlated with increases in tuition fees over time. For many years the majority of UBC’s online students (around 85%) were fourth year on-campus undergraduate students taking one or two online courses, either because the face-to-face classes for courses they needed for graduation were full, or because they had dropped one or two courses in previous years, and were trying to make these up to avoid coming back for another year. However, is there any research that analyses the effect of online learning on speed to degree completion of campus-based students? I don’t know of any systematic studies, but this would be a useful indicator of increased productivity due to online learning if such evidence exists.

In these cases we are talking about the impact of fully online courses. Will the same be true for hybrid courses? Will a reduction in class time – but still the need to come on campus at least some of the time – be sufficient to speed up completion rates? Will a more flexible schedule of attendance on campus bring in additional students who are currently unable to attend full-time? Hybrid learning is so new that we just don’t know yet. From a government or institutional perspective, we should be looking at the impact of hybrid learning both in terms of the impact on student enrollments and on time to completion of qualifications.

In terms of both these goals, we should also be examining the relative productivity of alternative approaches to online learning. For instance, would building more campuses be a more effective way to increase enrollments? To do this, though, we would need to know the relative costs (and without doing the study, it appears that online learning would have far lower ‘fixed’ or capital costs per student.) Perhaps more feasibly, how would changes to admission policies such as accepting work experience or prior learning assessment compare with increasing online learning in increasing access without loss of performance?

Lastly, are there other ways to increase access to learning, such as greater use of OERs, MOOCs, remote labs or other technology-based applications? If so, how would we measure their impact on productivity?

Conclusions

It can be seen that there are methodological challenges in trying to assess the impact of online learning on productivity. Nevertheless we don’t have to have perfect answers – we need to know where the balance of evidence is leading. However, at the moment, we don’t have even this – or rather the evidence is scattered across a large range of sources and is not collected together with a view to looking at the issue of productivity.

Indeed, institutions are sitting on much of the data needed to answer many of the questions posed in this post. What is needed is more analysis and communication of the findings that would flow from this analysis. Big data could provide a relatively low cost method of extracting the needed data, but also needed are researchers with the funding and time to do this. A small investment by government in this area could indeed be highly productive and provide useful guidelines for policy.

I believe we can provide reasonably good answers on the impact of online learning on access, at least, and this could show that it has increased the productivity of the system by bringing in students who would otherwise have been excluded.

Coming up next

Not sure – this series of posts is as much an exploration as a map! However, here are some of the topics I’m considering:

1. Can online learning or learning technologies lead to better learning outcomes at the same or less cost than classroom teaching? I will try to define what this might look like, and how this might be measured.

2. Can massive economies of scale be achieved through the use of technology without loss of quality? (One of the issues touched on by Sir John Daniel in his review of Higher Education in the Digital Age – I will try to address this in more detail.) This of course may well overlap or be pre-empted by (1) above, but I’ll probably need at least two posts on this topic anyway.

3. Could technology increase the productivity of faculty – in other words, can technology enable faculty to use their time more productively?

4. Is the quest for improved productivity an appropriate way to look at the use of technology for teaching in higher education? Or does it lead us away from the core benefits of higher education?

5. How could increased collaboration between institutions contribute to increases in productivity? How would this work?

As I said, this is an exploration. I really welcome your suggestions for topics or comments  - even ‘Stop it – it’s boring!’

 

 

MOOCs, MIT and Magic

Listen with webReader

MOOC panel: Dan Hastings, Anant Agarwal, Tony Bates, Sanjay Sarma, John Daniel

In my previous post, there were two sessions at the LINC 2013 conference that referred specifically to MIT’s own strategies for technology-enabled learning within MIT. These resulted in my asking the following question towards the end of the conference:

Why is MIT ignoring 25 years of research into online learning and 100 years research into how students learn in its design of online courses?

This post then will discuss both why I think this is the case, based on MIT’s own presentations at the conference, and the broader implications for educational research and instructional design.

MOOCs, MIT and Magic

The first session at the LINC conference was on four perspectives on MOOCs. There were four speakers before the coffee break, then the four speakers formed a panel to respond to questions from the audience after the coffee break. All four presentations are available in full from here, so I will provide a very brief summary of the main points made by each presenter. The session presenters were introduced by Richard Larson, Director of LINC.

First though, MIT’s Chancellor, Eric Grimson, laid out the reasons why MIT is making such a large commitment to OpenCourseWare, MOOCs and edX, and these reasons were reinforced by other MIT speakers:

  • to rethink the campus experience in the light of developments in online learning
  • increase access to learning worldwide by making MIT resources and courses available to anyone, anywhere
  • to conduct research on learning, especially by mining and analyzing the large amount of data generated by MOOCs
  • Anant Agarwal, the Director of edX, also later added: to develop an open source platform for (massive) online learning.

Sanjay Sarma, Director of MIT’s Office of Digital Learning, opened the session. He made the distinction between MOOCs as open courses available to anyone, reflecting the highest level of knowledge in particular subject areas, and the ‘magic’ of the on-campus experience, which is distinctly different from the online experience. He argued that it is difficult to define or pin down the magic that takes place on-campus, but referred to ‘in-the-corridor’ conversations between faculty and staff, hands-on engineering with other students outside of lectures and scheduled labs, and the informal learning that takes place between students in close proximity to one another. Not mentioned but implicit was also the very high standard of students admitted to MIT, and the impact of continuous contact on campus between student and professor, none of which of course is available to MOOC students. MOOCs however as well as providing a route to high quality learning for self-directed learners can be also be re-used and incorporated by other instructors in other institutions for credit.

Sir John Daniel took a much more critical view of MOOCs, suggesting, using the Gartner ‘hype cycle’, that MOOCs would soon enter the ‘trough of disillusionment’ and reflected on whether or how MOOCs will reach the “plateau of productivity”. He also pointed out that open and virtual universities in both developed and developing countries have been providing open and distance learning on a massive scale for over 40 years, and these initiatives have provided high quality and recognized qualifications.

Professor Anant Agarwal, the President of edX, provided some facts and figures about edX MOOCs, and mentioned that MIT had awarded a scholarship to the 15 year old Mongolian student who scored 100% on the final exam of an MIT MOOC course (although he will not receive credit for it). He pointed out that although over 150,000 learners enrolled in edXs first MOOC, 26,000 did the first activity, and 7,000 went on to complete successfully the certificate based on an online exam. (This woud provide a completion rate of approximately 28%, which is probably the most valid way to calculate completion rates for MOOCs.) More importantly, Agarwal defined the pedagogical ‘innovations’ in MOOCs as follows:

  • active learning: short video lectures interspersed with student tests/activities
  • self-paced learning
  • instant feedback
  • simulations/online labs to teach design of experiments
  • peer-to-peer learning.

Some of this has been made possible by MIT engineers building original software for automatic grading or feedback, including enabling students to write formulae as answers.

Me, MOOCs and pedagogy

I was the last speaker in this session and focused on the pedagogy of MOOCs, and suggested some ways in which they could be improved, based on 25 years of research in online learning. In summary the basic points I made are as follows:

  • MOOCs face several challenges, in particular low completion rates, problems with student  assessment, especially for assessment that requires qualitative or essay-type answers, and poor Internet access in developing countries
  • there is 25 years of experience and research into what works and what doesn’t in online learning
  • by and large, this knowledge is not being applied to the design of edX or Coursera MOOCs, which are based mainly on video recordings of classroom lectures
  • paying more attention to pedagogical issues and instructional design could help mitigate some of the challenges
  • in particular more attention needs to be paid to skills development, knowledge construction/deep learning and learner support
  • research should focus on course designs that focus on skills development rather than the transmission of information, on how to scale up learner support and oncosting models that provide resources for improved learner support
  • MOOCs should not be ‘second best’ for developing countries, replacing more locally based provision
  • for all this to happen, computer specialists and educators/instructional designers need to work together as equals

A copy of my presentation can be obtained by sending me an e-mail (tony.bates@ubc.ca) and I will send you an invitation via Dropbox to download the slides.

Technology-enabled learning: what’s going on at MIT?

This was the title of another session that described in more detail MIT’s other technology-enabled activities besides MOOCs. First I need to describe how MIT organizes its technology-enabled teaching and learning, based on the Executive Director of OpenCourseware, Cecilia d’Oliveira’s, clear presentation about 10 years history and the organizational structure of educational technology initiatives at MIT.

The Office of Digital Learning

Most of the better known MIT activities in this area come under the umbrella of the Office of Digital Learning, whose Director is Dr. Sanjay Sarma, a Professor of Mechanical Engineering.

Within this division is MIT OpenCour.seWare, which collects and provides a portal for video recordings of lectures and support materials that faculty have agreed can be shared openly. Currently MIT OCW offers materials from 2,150 MIT courses, plus courses from more than 300 universities worldwide. However, these are open educational materials (OERs), not full courses. Cecilia d’Oliveira, whose background is mainly in IT, is the Exective Director of MIT OpenCourseWare.

Also within the Division of Digital Learning is MITx, which works with faculty and academic departments to develop MOOCs (massive online courses, including currently 16 available at the moment through edX), and is responsible for the platform used not only for its own online courses but also for other edX courses. Some of these courses are available to MIT students for credit, as well as being open to other learners (but without credit).

While edX uses the MITx platform (which is open source and open to other developers) for its courses, edX is a ‘portal’ or stage for bringing together the MOOCs from MIT, Harvard and other partners in edX, such as UC Berkeley. There are currently 26 universities contributing MOOCs to edX, which is a non-profit organization supported mainly by a grant of $60 million from MIT and Harvard. Professor Anant Agarwal is the President of edX, and is Professor of Electrical Engineering and Computer Science and was formerly Director of the Computer Science and Artificial Intelligence Laboratory.

Media Production Services has approximately 25 staff who help with the video capturing and production for online courses, OCW, and other technical services.

Lastly, the Office of Educational Innovation and Technology (OEIT) is also within the Office of Digital Learning. OEIT works with faculty, staff and students to enable and promote the development and dissemination of innovative uses of technology in teaching and learning. Its focus is on innovative software and hardware to support learning and teaching. For instance, the ARTEMiS project is developing high-quality visualizations by applying the principles of visual communication and using the tools of modern computer graphics to create visualizations that accurately portray scientific and technological concepts. OEIT also maintains four physical Experimental Learning Environments (ELE) and a small pool of laptops for flexible deployment for innovative curricula.  These spaces are intended as incubators for testing new or different technologically enhanced pedagogical paradigms.  These physical spaces host a suite of technologies, applications and tools. The Director of OEIT is Dr. M.S. Vijay Kumar, who has a doctorate in academic computing in education.

Office of the Dean for Undergraduate Education

Also, within the Office of the Dean for Undergraduate Education, the Teaching and Learning Laboratory (TLL) collaborates with faculty, teaching assistants, and students to promote excellence in teaching and learning throughout the Institute, assisting with MIT-wide innovations in pedagogy, curriculum, and educational technology in STEM teaching and learning.  It also conducts research in teaching at MIT. Dr. Leslie Breslow, Senior Lecturer, Sloan School of Management, is the Director.

The Office of Faculty Support, provides support to help the faculty develop and coordinate the undergraduate curriculum and educational programming and provides grants and opportunities for faculty development . Professor Diana Henderson, Professor of Literature, is the Director.

Ten years of learning technology innovation at MIT

Research into teaching and learning at OEIT

There has been over 10 years of research and experimentation in teaching with technology at MIT. Brandon Murumatsu of OEIT described two current research projects. The first was the development of an online version of an MIT ‘hard’ course in mechanics and materials, traditionally delivered in class mainly by one hour 25 minute lectures and supported by problem sets that are done as homework. In the distance version, the lectures are recorded with a TA taking notes on the different topics or ‘chunks’ addressed during the lecture. This enables  the online video to be embedded in an interactive web page that is indexed and linked to the different ‘chunks’ of the lecture (see diagram below). Students can therefore search quickly for different parts of the lecture, slow down, speed up or repeat each ‘chunk, etc. No information was given on how successful this was.

The second experiment was a take a ‘flipped’ lecture class and embed assessment with immediate feedback into the lecture, through the use of simple multiple choice questions. This enabled a large set of data to be collected and anlyzed about how students responded to the different parts of the lecture. It was reported  that students love or even dream of getting the green checkmark when they get the answers right.

MIT student responses to online learning

The last session was in some ways the best of the conference. Three MIT students presented on their experiences of online learning. Sam Shames reported how he used online learning for his project work, exploring the ‘universe’ of online, open resources (in particular OCW) to help with problem-solving and project work, and in particular the opportunity it provides for students to find individual pathways through online OERs.

Ethan Solomon reported on his experience of taking four MOOCs. The good:

  • the ability to go over materials again and again
  • ability to go at one’s own pace
  • immediate feedback

The bad:

  • the limitation of multiple choice questions
  • MOOCs are mainly just lectures
  • difficulty of organizing massive numbers of students, especially in discussions.

Comments

I found the conference fascinating, for many reasons, but here are the main points I came away with.

1. MIT is making genuine efforts to open up its teaching, its materials and opportunities for learning across the world. It has invested very heavily in this, and many institutions, instructors and learners outside MIT are taking advantage. The quality of the content is often outstanding.

2. MIT is still tied though to the lecture as the main means of delivery for online learning. In fact, the MIT students on the panel showed that they understand the need to adopt a different approach to online learning better than the faculty.

3. MOOCs are the consequence of lecture capture technology. This technology makes it easy to move teaching online, but without changing the design of the teaching. This usually results in information transmission becoming the primary pedagogy, without addressing the many limitations of lectures, except the ability for asynchronous access, which is an important improvement on the ‘live’ lecture.

4. MIT  is using a behaviourist approach to its online learning, based mainly on Skinnerian thinking and research. Long lectures are still a core part of its campus pedagogy as well, but there is additional ‘magic’ provided on campus (informal and experiential learning and close contact with faculty) which is not available to its online learners. In my view, it is a mistake to believe that such ‘magic’ cannot be created online. It can, but it needs good course design based on sound educational principles.

5. If instructional designers exist at MIT, they play a minor role or have little power. This shows in both the design of its MOOCs and in the research being conducted.

6. In my view, MIT will struggle to make an impact on educational research if it continues to ignore the potential contribution of educators. It is as if researchers such as Piaget, Bruner, Vigotsky, Carl Rogers, Gagné, and many later researchers had never existed. Can you imagine anyone trying to develop a new form of transportation while deliberately ignoring  Newtonian mechanics? Yet this is what MIT is doing in its educational research. In fact, as the research described above shows, they are re-inventing the wheel. It was admiited that many of the results they are getting are not new but have been known for many years.

For me this is a tragedy. MIT’s engineers have so much to offer in helping to improve educational technology but it needs to be informed and embedded in theories of learning, and must take account of prior research, for it to gain traction and be of value. This means working in a team with educators who have the design and research knowledge and experience, and working with them as equal partners.

Of course, MIT does not need this advice. It is immensely successful and will continue to produce great engineers. But it could also do so much more.

Having said all that, I learned a great deal from the conference, was treated with immense courtesy, and I am very grateful for the invitation to attend.

 

A report on e-learning from the University of Ottawa

Listen with webReader

At the beginning of May I was in Ottawa, first to present to the University of Ottawa’s Board of Governors about the current state of online learning in Canada, in support of their excellent plan for e-learning that was before the Board of Governors.

A steady move to increased hybrid learning

U of Ottawa already has five programs and 76 courses that are available online, and is planning to have at least 20% of all sections in a hybrid mode (50% face-to-face and 50% online) by 2020. Although the Board felt this target was too timid, a feature of the plan is to ensure all faculty are properly trained and supported before they start developing hybrid courses.

One advantage that the U of Ottawa has is that it has already in place a strong Centre for Mediated Teaching and Learning which includes a Centre for e-Learning, which means it can move more quickly to greater hybrid and online learning.

The impact on classroom spaces

Another feature of the plan is an extremely detailed analysis of the likely impact of this move to hybrid learning on classroom spaces. U of Ottawa is facing increasing pressure for more classroom spaces and their detailed analysis in an appendix to the report suggests that a 20% target of hybrid courses could lead to a 10% reduction in the number of extra classroom spaces needed as it expands its student numbers over the next few years. This is the first time I have seen such a careful estimate of the impact of hybrid learning on campus classroom space.

To MOOC or not?

Another feature of the report is a detailed analysis of the pros and cons of the U of Ottawa developing MOOCs. As part of the analysis, U of Ottawa has estimated the costs of developing and delivering a Coursera MOOC. I was surprise to see their estimated development cost for one MOOC as being $110,000. This is between two to three times the cost of delivering an LMS-based credit course of similar length at most institutions in Canada. Then annual maintenance runs at another $29,000 a year. I’d be interested to hear from other institutions who are already offering MOOCs if these costs are typical. If they are, then the benefits to the institution need to be that much greater, compared with spending the money on credit-based students.

The plan recommends not developing MOOCs in the short term without first doing a detailed market analysis and a feasibility study, but in the mid-term to offer flag-ship MOOCs in French (U of Ottawa is a bilingual university), where the market is more open.

More information

While at the U of Ottawa I also gave a presentation to faculty on designing university teaching to meet the needs of 21st century students.

If you want copies of my slides for presentations either to the Board or faculty, send an e-mail to tony.bates@ubc.ca.

The U of Ottawa e-learning plan is available for downloading from the U of Ottawa web site. If your institution is contemplating a major increase in hybrid and /or online learning, this report is well worth reading in full.