April 24, 2014

Producing ‘innovative’ graduates and how online learning can help

Listen with webReader
© Heath Robinson 1939

© Heath Robinson 1939 – it’s not just engineers needed

Avvisati, F., Jacotin, G., and Vincent-Lacrin, S. (2013) Educating Higher Education Students for Innovative Economies: What International Data Tells Us, Tuning Journal for Higher Education, No. 1

Hoidn, S. and Kärkkäinen, K. (2014) Promoting Skills for Innovation in Higher Education A Literature Review on the Effectiveness of Problem-based Learning and of Teaching Behaviours Paris: OECD Education Working Papers, No. 100, OECD Publishing.

Innovation, higher education and online learning

OECD’s Centre for Educational Research and Innovation (CERI) has recently published two interesting papers which while not specifically about online learning, address issues that are very relevant for teaching in higher education. These papers give some useful directions for the design of online learning for developing the skills and knowledge that lead to innovation.

In this post, I will discuss the paper by Avvisati et al, and in another post the paper by Hoidn and Kärkkäinen.

The paper’s goal and methodology

Avvisati et. set out to address the following question:

What is the broad mix of skills needed in innovative societies and sectors, and how can higher education institutions and innovation policies contribute to fostering this mix?

Avvisati et al. analysed the responses to two OECD surveys of tertiary graduates five years after their graduation, namely the twin surveys Reflex and Hegesco.

Avvisati et el. define “highly innovative” professionals as those working in innovative organisations and involved in the introduction of innovations; they represents on average 56% of tertiary-educated professionals in the 24 or so mainly European countries that were surveyed.

Main results

  1. Innovation requires a broad mix of academic subject domains. For instance:
    • in manufacturing industries, 50% of ‘highly innovative professionals’ have an engineering/science degree
    • in business and finance industries, the bulk of the highly innovative workforce is formed by business graduates, social sciences graduates, and law graduates
    • a significant proportion from all fields [of study] work in a highly innovative job: 60% of engineering/science graduates; 58% of arts/agriculture graduates; 50% of education graduates

This conclusion has important policy implications, as innovation policies concerned with human resources tend to have a narrow focus on scientists and engineers (and sometimes entrepreneurship). An overly exclusive focus on the training of scientists and engineers to promote innovation is largely misplaced, given that other graduates do also contribute significantly to innovation and that the relative importance of the manufacturing sector, where STEM graduates predominate, [is] decreasing in most OECD economies.

2. The critical skills that distinguish innovators from non-innovators the most are:

    • creativity (“come up with new ideas and solutions” and the “willingness to question ideas”), followed by
    • the “ability to present ideas in audience”,
    • “alertness to opportunities”,
    • “analytical thinking”,
    • “ability to coordinate activities”, and the
    • “ability to acquire new knowledge”

3. ‘Highly innovative professionals’ tend to agree that universities developed mostly their thinking and learning skills (analytical thinking and the ability to rapidly acquire new knowledge) as well as their domain-specific expertise (mastery of their own field or discipline).

4. At the same time, respondents were dissatisfied with the level of social and behavioural skills acquired through their university programme; ’presenting ideas’ and ‘coming up with new ideas and solutions’ were not considered to be a particularly strong point of university education.

5. Respondents also reported that their progress as students was consistently and significantly associated with the quality of teaching and learning inputs:

    • graduates are more likely to participate in innovation processes after having attended …programmes stressing practical knowledge, such as student-led projects and problem-based learning
    • theory-based forms of instruction, such as lectures and the learning of theories and paradigms, have a positive, but weaker association with innovation
    • the emphasis on theoretical knowledge and conceptual understanding are particularly associated with … analytical thinking, in acquiring new knowledge, and in writing
    • the emphasis of programmes on practical knowledge, on student-led projects and on problem-based learning are reflected in the level of creative skills, of oral communication skills and of teamwork and leadership skills of students
    • thus a diverse offer of pedagogies is the most effective way to foster all skills for innovation in the working population.

6. The mastery of one’s own field is not among the very top skills that differentiate the most highly innovative from less innovative professionals. Many of the critical skills for innovation can be fostered in all domains, even though it could take a different shape from one subject to the other.

7. There is overall no strong evidence of a shortage (or coming shortage) of tertiary education graduates in STEM disciplines in the OECD area.

Comment and discussion

Some care is needed in interpreting these results. It should be noted that they reflect the views of ‘innovative workers’ five years into post-degree employment, not employers or more senior executives or managers, and most of the responders would have been European. Innovation itself is not clearly defined other than it’s what people in the survey say it is. For most of us, these results will not appear surprising, and will reinforce beliefs that are held by many – but not all – academics. However, the results of this study do challenge conventional wisdom sometimes found among policy-makers and the general public.

I draw the following conclusions from this study:

  • we need to continue to support a wide variety of disciplines and subject domains in our universities if we really want innovation across our society and economy; STEM subjects are important for innovation in many but by no means all areas of innovation in work and society
  • as always in pedagogy, it is not a question of either theory or practice, of information transmission or knowledge management. We need diverse approaches to pedagogy/teaching methods, and these need to be fine tuned within different subject domains
  • more empirical work needs to be done on the link between productive innovation, intellectual skills development, content, and teaching methods
  • nevertheless, it seems clear to me that knowledge management, independent learning and lifelong learning will become increasingly important skills for the promotion and development of innovation in work and society
  • learning technologies and in particular online learning can contribute significantly to developing skills that will foster innovation, but the technology must also be wedded to appropriate teaching methods
  • teaching for innovation is still more art than science, but it is not totally a shot in the dark.

Next

I will review the other OECD paper that is a literature review of the effectiveness of problem-based learning for promoting skills for innovation in higher education, and what that might mean for online learning.

 

 

2020 Vision: Outlook for online learning in 2014 and way beyond

Listen with webReader

 2020 visionTaking the long view

Doug Saunders in the Globe and Mail on  January 4 wrote an interesting piece on prediction, entitled: “Gadgets alone don’t make the future.” Having shown how amazingly accurate technologists in 1961 were in predicting what technologies would roll out in the future, he also showed how poorly they predicted how these gadgets would impact on our lives. In summary:

‘We are very good at guessing where our inventions might lead. We are very poor in understanding how humans might change their lives….the decision of what kind of life to live between the screens remains a political one, shaped not by our inventions but by our own decisions.’

Last year I spent some time discussing the value of predictions. One point I didn’t mention is the limitation of predicting just one year ahead, because you can’t identify the long term directions, and so often you’re driven by what happened in the very recent past, i.e. last year, because that’s the latest and often only data you have. More importantly, though, looking one year ahead assumes that there is no choice in what technologies we will use and how we will use them, because they are already entering our society. Also, this is likely to be the last year in which I make predictions for the future. I will be 75 in April, and I plan to stop all paid professional activities at that point (although I will keep my blog, but more as a journalist than as a practitioner).

So this seems to be a good point to look not just at 2014, but where we might be going five to ten years from now, and in doing this, I want to include choice or human decision-making as well as technological determinism. In other words, what kind of online learning do I expect in the future, given what I know so far?

The disappearance of online learning as a separate construct

In 2020, people won’t be talking about online learning as such. It will be so integrated with teaching and learning that it will be like talking today about whether we should use classrooms. In fact, we may be talking much more about classrooms or the campus experience in 2020, because of online learning, and how it is changing the whole way that students are learning. There is likely to be heated discussions about the role and purpose of campuses and school buildings, the design of classrooms, and who needs to be there (teachers and students) and more importantly what for, when students can do so much of their learning online – and generally prefer to, because of the flexibility, and of their control over their own learning. The big changes then are likely to be on-campus, rather than on-line.

Steelcase Node Classroom

Steelcase Node Classroom

Multi-mode delivery concentrated in fewer institutions – but more diversity

Quite a few public and smaller private post-secondary institutions will be gone or radically transformed by 2020. Particularly at risk are smaller, low status state or provincial universities and colleges or their campuses in metropolitan areas, where there is local and regional competition for students. They will have lost students to more prestigious universities and high status vocationally oriented institutions using online and flexible learning to boost their numbers. Government will be increasingly reluctant to build new campuses, looking to more flexible and more cost effective online delivery options to accommodate increasing demand. Nevertheless, politics will occasionally trump economics, with small new universities and colleges still being created in smaller towns away from the larger urban areas. Even these though will have much smaller campuses than today and probably as much as 50% of all course enrollments online, often in partnership with more established and prestigious universities through course sharing and credit transfer.

Those institutions that have survived will be offering students a range of choices of how they can access learning. Courses or programs will be deliberately designed to accommodate flexibility of access. Thus students will be able to decide whether to do all their studying on campus, all of it online, or a mix of both, although courses or programs are likely to have a common assessment strategy (see below). This will not be driven so much by academic or even political decisions, but by students voting with their feet (or mouses) to study at those institutions that provide such flexibility.

Multi-purpose, open delivery, with multiple levels of service and fees

Content will be multi-purposed, depending on a learner’s goals. Thus the same content can be part of a credit-based degree-level course, program or competency, part of a non-credit certificate or diploma, or available as open access. Learners will also be able to choose from a range of different course or program components, dependent on their needs and interests. Because most content will be open and modular, in the form of open textbooks, open multimedia resources, and open research, institutions will offer a variety of templates for courses and programs built around open content. For example, for a degree in physics, certain topics must be covered, with a strong recommendation for the sequence of study, but within those core levels of competency, there will be a variety of routes or electives towards a final degree, where broadly based learning outcomes are set, but multiple routes are offered for progress to these outcomes. Those content components can be accessed from a wide range of approved sources. It is the competency and academic performance of the learner that the institution will accredit.

Most institutions will have an open education portal, that contains not only a wide range of open educational resources, but also a range of open services, such as program templates or free academic guidance for specific target groups, as part of their enrollment strategy. Although such portals are likely to include materials from a wide range of sources from around the world, special emphasis will be given to open content developed by their own faculty, based on their latest research or scholarship, as a way of branding their institution. iTunesU, MIT’s Opencourseware, OpenLearn, and MOOCs are early prototypes, but content quality in the future will be greatly improved in terms of pedagogical and media design to accommodate online learners. Also states and provinces will also establish system-wide portals of open educational resources, particularly at the k-12 and two year college level (see eLearnPunjab and open.bccampus.ca as prototype models).

Because academic content is almost all open, free and easily accessible over the Internet, students will not pay tuition fees for content delivery, but for services such as academic guidance and learning support, and these fees will vary depending on the level of service required. Thus students who want a traditional course that covers guidance on and access to content, tutorial help, access to campus facilities, feedback and assessment will pay full fee (some of which may still be government subsidized in the public system). Students who want just open access will pay nothing, but will get few if any support services, and if they need a formal assessment, they will need to pay for this (although again this may be subsidized in a public system). Other students may want feedback and some form of continuous assessment, but will not want to pay for full tutorial support.

There are several consequences of this increased flexibility. Some institutions will specialize in small-class, on-campus education at high cost. Others will focus on high quality delivery through a variety of delivery modes, with a particular emphasis on course design and learner support. Some institutions will focus on low cost, competency-based open access programs, supported by businesses requiring specific skilled labour, and a few institutions will be specialists in fully online distance delivery operating on a national or international basis, at a lower cost but equally high quality as campus-based institutions. The majority of institutions though will become multi-purpose, multiple delivery institutions because of the economies of scale and scope possible.

Goodbye to the lecture-based course

In most institutions, courses based on three lectures a week over 13 weeks will have disappeared. There are several reasons for this. The first is that all content can be easily digitalized and made available on demand at very low cost. Second, institutions will be making greater use of dynamic video (not talking heads) for demonstration, simulations, animations, etc. Thus most content modules will be multi-media. Third, open textbooks incorporating multi media components and student activities will provide the content, organization and interpretation that are the rationale for most lectures. Lastly, and most significantly, the priority for teaching will have changed from information transmission and organization to knowledge management, where students have the responsibility for finding, analyzing, evaluating, sharing and applying knowledge, under the direction of a skilled subject expert. Project-based learning, collaborative learning and situated or experiential learning will become much more widely prevalent. Also many instructors will prefer to use the time they would have spent on a series of  lectures in providing more direct, individual and group learner support, thus bringing them into closer contact with learners.

This does not mean that lectures will disappear altogether, but they will be special events, and probably multi-media, synchronously and asynchronously delivered. Special events might include a professor’s summary of his latest research, the introduction to a course, a point mid-way through a course for taking stock and dealing with common difficulties, or the wrap-up to a course. It will provide a chance for an instructor to makes themselves known, to impart their interests and enthusiasm, and to motivate learners, but this will be just one, relatively small, but important component of a much broader learning experience for students.

61730023

Goodbye to the written exam – and welcome to the final implementation of lifelong learning

For most post-secondary qualifications, written exams will have been replaced by assessment through multimedia portfolios of student work. These will show not only students’ current knowledge and competencies, but also their progression over time, and a range of equally important skills, such as their ability to work collaboratively, self-management of learning, and general communication skills. Assessment will be mainly on a continuous, on-going basis.

As well as change in the method of assessing learning there will be greater variety in the range of accredited qualifications. Degrees, certificates and diplomas will still be important, but these will be complemented with a wide range of assessments of informal or non-formal learning, such as badges, some offered by post-secondary institutions, others offered by employers’ organizations or co-operatives of professionals. University and college diplomas and degrees will increasingly be seen as milestones on the journey to lifelong learning, and for demographic and economic reasons, the lifelong learning market will become a much larger market than the high school leaver market.

This means academic departments will need to develop programs and courses that range from introductory or foundational through undergraduate degrees to professional masters to lifelong learning, again using similar content modules adapted to different markets, as well as creating or adapting new content, based on the latest research in a field, for these newer markets. Much of the lifelong market will lend itself to online and hybrid learning, but in different structures (short modules, for instance) than the undergraduate and higher degree market. Universities and colleges will increasingly compete with the corporate training industry for these post-postgraduate learners, who will be able and willing to afford top dollar for top-level lifelong learning opportunities, based on the latest research coming out of universities, government and businesses.

However, a large part of the lifelong learning market will become occupied by communities of practice and self-learning, through collaborative learning, sharing of knowledge and experience, and crowd-sourcing new ideas and development, particularly assisted by an evolution of what are now known as cMOOCs. Such informal learning provision will be particularly valuable for non-governmental or charitable organizations, such as the Red Cross, Greenpeace or UNICEF, or local government, looking for ways to engage communities in their areas of operation. These communities of learners will be open and free, and hence will provide a competitive alternative to the high priced lifelong learning programs being offered by research universities. This will put pressure on universities and colleges to provide more flexible arrangements for recognition of informal learning, in order to hold on to their current monopoly of post-secondary accreditation.

Image: © Etienne Wenger, 2010

Image: © Etienne Wenger, 2010

New financial models

Because most content will be freely accessible, and because students will pay incrementally for a wide variety of services, new financial models will need to be developed, to support the flexibility and range of services that students will increasingly demand and require. The biggest move is likely to be away from block funding or enrollment-driven funding by government towards pay-for-service through student fees for teaching. There will be further separation of the funding for research and teaching (this has already happened in some countries, such as in England and Wales.) As a result government financing may well change, so that students are given a post-secondary grant at the age of 17, and have the right to decide how to spend that grant on post-secondary education, rather than funding institutions directly for teaching.

This may have some unexpected benefits for academic departments. Under this model it makes much more sense to fund programs directly from fees for the program, than to pool grants and fees centrally then break out money for teaching and filter it down through the departments. Thus program fees or service fees  would come to academic departments (or more accurately the program areas) directly, then the programs would pay for university services such as registration and financial services on a direct cost basis, plus a percentage for general overheads. This is already happening in some public universities at post-graduate levels, where tuition fees for online professional masters more than cover all the costs, direct and indirect, of a program, including the cost of full-time research professors who teach on the program.

This model would also have two other benefits. It would put pressure on service departments, such as HR, financial services, the Registry, etc., to become more cost-efficient, because direct costs to programs become more transparent. Second, since online students do not need a range of campus services such as campus building maintenance, lighting, and heating, it would lead to the different costs of online vs campus-teaching becoming more transparent and comparable, with an economic incentive to move more towards the most cost-efficient delivery model.

There are also disadvantages. Some model would be needed to support more expensive programs to deliver, or programs that are specialized but important in a university community. However, a program-based financial model may help save small departments who are struggling for minimal enrolments from their local market. Online courses can open the market to regional or international students and offer the chance of collaboration and partnership with other institutions, through course and student sharing.

The disaggregation of institutional activities required for the flexible delivery of programs in a world where content is free offers opportunities for rethinking how teaching and learning is funded.

Systematic faculty development and training

Since content will be freely accessible, institutions’ reputation and branding will increasingly depend on the way they support learners. This will put much greater emphasis on instructors having good teaching skills as well as subject expertise. Thus most universities and colleges will require faculty to have assessed teaching skills before tenure or permanent appointment, and equal attention will be given to teaching expertise as research in promotion. This will mean incorporating teaching practice and methods within most post-graduate subject areas, college instructors having compulsory pre-service teacher training, and regular faculty having systematic ongoing professional development as new technologies and new teaching approaches develop over time. The immediate benefit of this will be better student retention rates and higher quality learning outcomes.

Devolved decision-making and organizational models

A move to program-based funding, the need for effective course designs to attract students, the differentiation of services, the increased professionalism in teaching, and freely available open content will result in a move to systematic program planning and team teaching. A typical team will consist of a senior research professor, several junior or adjunct professors, an instructional designer/project manager and a media/web designer. The senior faculty member, in collaboration with the other team members, will be responsible for decisions about curriculum content, methods of learner support, and assessment standards. The team will develop assessment criteria and rubrics, and where necessary hire additional instructors for learner support and marking of assessments , under the supervision of the senior faculty members.

One consequence will be the disappearance of central centres for teaching and technology, except in small institutions. Instructional design staff will be located in program areas and will be responsible with academic faculty for faculty development activities, as well as with overall course design input. There will be increased demand for media designers, while instructional designers will be in less demand in the future, but still necessary to support faculty, especially as new learning technologies develop.

Student privacy, data security and student online behaviour will become more difficult

Learning will increasingly be delivered through student-owned devices, and learners will increasingly integrate social life, work and study in a seamless manner. Services will increasingly be delivered through the cloud. Security agencies, Internet-based companies and knowledge-based companies will constantly be seeking access to student data, especially student learning performance and online behaviour, as this information will be increasingly valuable for state security and commercial reasons. As a result it will become increasingly difficult for institutions to protect student data and their privacy. This may turn out to be the biggest challenge for students, institutions, and government in the next 20 years and could seriously inhibit the development of online learning in the future, if students or faculty lose trust in the system.

The future is about choices

This is my view about where we could be going with online learning in the next five to ten years. However, I will not be making the decisions, as I am retiring in April. If you do not like this vision, then you are in a position to influence a different kind of vision. Although as McLuhan says, we are shaped by our devices, we also shape the world around these devices. The worst thing we could do is to leave it to computer scientists to decide our future.

The value such a vision lies not in its detail, but in identifying some of the key choices or decisions that will need to be made. So here are the decisions that are thrown up by this vision for the future, for students, faculty, institutions and government (and some of these, such as those about campus facilities, should be being made right now):

Students and learners

  • at this point in my life, what are my learning goals? What is the best way to meet these? Where can I get advice for this?
  • do I need a qualification and if so, what kind?
  • what is the best way for me to access this learning? On-campus; online; or a mix of both?
  • what kind of learning support do I need?
  • how much do I want to – or must I – pay for these services?
  • what institution or other method of delivery will provide what I want? Where can I get independent advice on this?
  • how can I protect my privacy when I am online studying?

Faculty and instructors

  • why do students need to come to campus? What am I offering on-campus that they couldn’t get online? Have I looked up the research on this?
  • what teaching methods will lead to the kind of learning outcomes that students will need in life?
  • what should be my role if content is freely available online?
  • what kind of teaching spaces do I need for what I want to offer on campus?
  • how should I best use my time in teaching? In what kind of teaching activities can I really make a difference for students?
  • if I create new or original content for my teaching, should I make it openly available to anyone to use?
  • what methods of assessment should I use in a digital age? How do I assess prior or informal learning?
  • what kind of courses or programs should we be offering for lifelong learners?
  • what do I need to know about student data, and the protection of student privacy?
  • what training or professional development do I need to ensure that I can meet the learning needs of my students?

Institutions

  • what kind of campus will we need in 10 years time?
  • what proportion of course enrollments are likely to be accessed off-campus?
  • what will be the best way to accommodate more students – online learning or more buildings?
  • what kind and number of teaching spaces will we need?
  • what partnerships or strategies should we adopt to protect our enrollment base?
  • what are our strategies and policies regarding open educational resources?
  • what is our strategy for lifelong learning?
  • what financial models should we put in place to encourage innovation in teaching and to attract students?
  • how do we ensure that faculty have the skills necessary for teaching in a digital age?
  • how can we best reward innovation and high quality teaching?
  • what kind of organization and staff do we need to support faculty in their teaching?
  • how do we best protect student data and privacy (as well as our staff’s) in a digital age?

Government

  • what kind of post-secondary system, in terms of institutional differentiation, program delivery and innovations in teaching, do we need in a digital age?
  • how many, and what kind of, campuses do we need when students are also studying online? What is the best way to accommodate expansion in the system?
  • how can we best support system-wide open education, to reduce costs and increase quality?
  • how should we fund post-secondary education in a digital age? How much and what should ‘first-time’ students pay for themselves? What should lifelong learners who have already been through the system pay? What funding models would encourage innovation in teaching and help improve quality?
  • how can online learning help to increase the productivity of the post-secondary educational system? What can we do to encourage this?
  • what does government need to do to protect student data and student privacy?

What’s YOUR vision?

I won’t be around to make or influence these decisions, but most of you will. Are there decisions I’ve missed? What decisions would you make? What’s your vision for the future?

If you are willing to share just one response to any of these questions or decisions, this will be very much appreciated. Because the future will be increasingly about sharing knowledge.

Productivity and online learning redux

Listen with webReader
If only it was this simple: Image© Course Gateway, 2013

If only it was this simple: Image© Course Gateway, 2013

Summarizing the previous posts

In previous posts (see end of this post), I tried to identify a range of areas where online learning might enable productivity gains. In this post I will bring them together and state what I believe are the areas that offer the potential for the greatest gains, given current knowledge. At this stage my conclusions are very subjective but I hope they will provide a framework for further studies and for better and more systematic data collection.

Access

There are fairly strong arguments (but little hard data) to suggest that online learning can help governments boost participation rates more effectively than by building more campuses and funding more campus-based education. Productivity is increased by eventually getting more graduates than would otherwise have been possible without online learning and the flexibility it allows.

The reasons to support this argument are as follows:

  • Particularly in jurisdictions where there is already a high participation rate, increasing that rate further means reaching out to groups that have to date been largely excluded from post-secondary education. Since existing data indicates that online learning appeals particularly to lifelong learners, working adults and older learners, online learning is more likely to appeal to this target group. To date though there has been little attempt to measure the impact of reaching otherwise excluded groups through online learning. More hard data is needed to support this argument.
  • there is some evidence to suggest that online learning has lower overhead costs than campus-based education. If this is correct, it may be more productive to expand online learning rather than build new campuses when attempting to increase participation rates. Again though there are few studies that provide hard data to show that overhead costs are indeed lower for online learning.
  • it has been argued that hybrid and online learning provides existing students with more flexibility, allowing them to combine work with part-time study, thus allowing them either to complete studies they could not afford without part-time work, or to complete more quickly than they would without the flexibility that online learning provides. Of course the argument could work in reverse. By providing more flexibility, students may take longer to graduate. Once again, it should be possible to test either argument empirically, but again there are few if any studies that have looked at this.

Thus online learning offers the promise of increasing productivity through increasing participation and speed to graduation  at less cost, but there are few studies to date to either support or refute these claims. It should be noted though that in most cases, the data to test these arguments is within institutional databases; it has just not been extracted and analyzed for this purpose.

Wilfred Laurier University is proposing a campus in Milton Ontario - but would it be more productive to use online learning?

Wilfred Laurier University is proposing a campus in Milton Ontario – but would it be more productive to use online learning?

Free or massively scalable content

Nowhere in online learning is there such potential for increases in educational productivity as in content development and delivery. Once learning materials are created, they can be stored, accessed, delivered and used by an unlimited number of learners, thus potentially achieving large economies of scale and thereby reducing costs per learner (see graph at beginning of this post).

Another important factor contributing to economies of scale in online learning is the increasing availability of open educational resources. Particularly in foundational courses and many ‘standard’ undergraduate courses, ‘open’ material is already available and does not have to be re-created. The main cost is selecting and organizing existing open source materials, but this is likely to be less time-consuming for faculty than creating materials from scratch. Open online textbooks can have a direct and immediate impact on reducing student costs.

Nevertheless there are many impediments to achieving productivity gains through free or massively scalable content:

  • faculty often see themselves as  creating unique and original material in their teaching; this is true occasionally and needs to be respected, especially where faculty are teaching about their own and related research. Often though faculty are merely repackaging prior knowledge. That prior knowledge is increasingly being made available and open for anyone to use.
  • the shelf life of much academic material is increasingly short; thus content needs to be constantly maintained and updated
  • there may not be a massive market for many specialist online courses thus preventing economies of scale from being achieved. However, there are many ways to increase market reach with online learning, including going global, collaborating and sharing materials, courses and programs with partner institutions in the same or other jurisdictions, repackaging content for different markets (e.g. for casual learners, certificates, or degrees). Such strategies though will also require reviewing and often changing admission policies, intellectual property agreements and other practices that restrict access to ‘institutional’ content.
  • the quality of open educational resources developed by faculty working alone, without applying best course design practices, is often very low and such ‘open’ resources are often not considered suitable for re-use
  • content development and delivery is a relatively small proportion of the cost of credit-based online learning (from 15-20%); the main costs are in learner support.

Despite these impediments, in certain circumstances (i.e. where there is a large market and best practices are applied to content design), online content development and delivery is already resulting in increased productivity in post-secondary education, although it has yet to be well measured.

Course design based on sound pedagogical principles

One important reason for the success of many for-credit online courses and programs has been the introduction of best practices in course design, drawing on cognitive science research, best teaching practices, and prior experience of teaching students at a distance. These practices include situated learning, drawing on learners’ own work and life experiences, student time-management support, collaborative learning, student activities resulting in greater time on task, and regular and constructive feedback to students through continuous assessment.

In particular a focus in online courses on ’21st century skills’ development, such as knowledge management and independent learning, would have two benefits. It would improve outputs (turning out graduates with the skills needed). Second, content development and delivery becomes subsidiary to helping students find, analyze, organize and apply content themselves. Thus less time would be spent by instructors on course development and delivery.

Such practices of course could be be used in classroom teaching, but good online course design templates are more easily scaled and reproduced, and the technology lends itself to such approaches to learning. Productivity is improved through application of such quality course design because more students achieve higher levels of learning and more students complete courses and programs. Thus although it is not the technology itself that results in better outcomes, the technology facilitates the change to more effective teaching methods.

Once again though, while teachers and students who have been engaged in such new designs often claim better outcomes, there is still a lack of convincing empirical research to support these beliefs. Nevertheless, a focus on better design replicated on a large scale through online learning should have a major impact on improving productivity.

Learner support

What little research that has been done on costs of credit-based online learning indicates that course delivery (which includes both learner support and student assessment) accounted for the largest overall cost of an online program (37%), almost three times more than course development, over the life of an online program (Bates and Sangra, 2011).

Instructional MOOCs (xMOOCs) have basically removed learner support, at least in terms of human (instructor) support, but this has resulted in a very low number of MOOC learners passing end-of-course assessments of learning. Indeed, prior research into credit-based learning has established that instructor online ‘presence’ is a critical factor in retaining students. So far, it has proved difficult to scale up learner support on a massive scale, except through the use of computer technology, such as automated feedback. However, Carey and Trick (2013) and indeed faculty at elite institutions who are offering xMOOCs (see Thrun and ‘the Magic of the Campus‘) have argued that such computer support does not support ‘the learning that matters most’.

Although computer-based feedback and adaptive learning facilitate comprehension and technical mastery outcomes, computer-based approaches to learner support to date has been inadequate for formal assessment of higher order learning skills such as original, critical or strategic thinking, evaluation of strategies or alternative explanations. To assess such forms of learning, deep expertise and qualitative assessment is required, and to date not only human instructors, but instructors with a deep subject understanding and high levels of expertise, are needed to both develop and assess such high level skills. Given the long history of trying to apply artificial intelligence to instruction, immediate and major breakthroughs seem unlikely, at least in the short term.

A l'école, Jean Marc Cote, 1901.

A l’école, Jean Marc Cote, 1901.

However, there are other ways in which the productivity of learner support might be improved. In cMOOCs that are more like communities of practice and thus contain many participants with already high levels of expertise, that expertise and judgement can be provided by the participants themselves. (The issue then is how do people get to such high levels of expertise in the first place – we need more research/experience with cMOOCs to know whether they are also good for learners with initially low levels of knowledge in a particular subject domain. Some combination of expert/instructors plus a community of practice approach might be necessary for such learners, but might still operate successfully with much higher instructor:learner ratios than in conventional, credit-based learning.)

Also, credit-based online learning has achieved some economies of scale and scope by re-organizing the learner support process, through the hire and training in online learner support of lower-paid contact adjuncts who still have high level academic qualifications, under the supervision of a senior faculty member. In other words, team teaching approaches, with the senior academic working more as a teaching consultant, setting curriculum, designing assessments and creating rubrics, and supervising the learner support provided by a team of adjuncts, can help not only reduce costs but also achieve modest economies of scale in learner support, especially when combined with best practices in course design.

Innovation vs standardization

In industry, innovation is often another way of saying ‘investment in technology’. However, there is more to innovation than just replacing a human activity with a computer-based activity. What the technology usually brings about is a change in process at the same time.

Thus there is a natural tension between ‘best practice’, based on experience of doing things in an ‘old’ way, and innovation, which means doing something differently. Indeed Christensen (2008) distinguished between ‘sustaining’ innovation, which builds and improves on best practices, and ‘disruptive’ innovation, where a new technology results in sweeping away old ways of doing something. Real, sustainable innovation occurs then when new technology is combined with new processes.

In education, perhaps the main ‘process’ that we need to examine is the instructional model, particularly that based around the lecture system. I am not arguing that lectures no longer have a purpose. However, the teaching model based on three lectures a week over 13 weeks used primarily to deliver information to students is now redundant, given that information is ubiquitous and if not free, increasingly available at low cost over the Internet. Thus knowledge management becomes more important than mere access to knowledge. If we look at xMOOCs though we have taken a new technology – video lecture capture and Internet transmission – and applied it to an outdated model of teaching. True innovation requires a change of process or method as well as a change of technology.

Earlier though I argued that we need to apply best practices in course design to the use of technology. By definition, best practices are based on tried and true methods. However, in post-secondary education, these ‘best practices’ are not the prevailing teaching method on most campuses (except perhaps the very elite, where they can be applied on a face-to-face basis to small classes.). As public post-secondary education has become massified, the lecture has become the default model, because in a classroom based system, it has proved the only way to ‘scale up.’ Online learning offers an opportunity to break out of this redundant and increasingly less productive lecture model of teaching, as it does not develop the skills needed in the 21st century.

There are then really two routes to innovation. The least risky is the sustainable development approach, finding ways to incorporate and more importantly adapt ‘best practices’ on a massive scale through online learning. This will mean increasing productivity in relatively small steps. The advantage of this approach is that it is more likely to preserve and protect the core values of ‘higher’ education. The other route is ‘disruptive’ innovation – jumping into an entirely different way of doing something based around a new or emerging technology. This is more likely to bring much greater productivity gains, but the risks are much higher. It could well result in throwing the baby out with the bathwater.

In reality, institutions, and individuals within those institutions, cannot control disruptive innovation – it comes from outside. However, institutions can control sustainable innovation. Indeed if they do not they are much more vulnerable to disruptive innovation. Thus it is important to find new best practices that are easily scalable, while meeting the needs of 21st century learners at high quality. This is probably the most sensible way to bring about radically better productivity. But it’s not going to be easy.

Conclusions

So here are my personal views on online learning and productivity, based on this analysis.

1. Government and institutional leaders need to set improved productivity as a key goal for investment in learning technologies. This means setting benchmarks and implementing means of measuring success or otherwise in improving productivity through learning technologies/online learning. Data analytics now make this measurement more feasible than in the past, but it also requires models or a theoretical framework for assessing what constitutes productivity in a post-secondary educational setting.

2. Understanding the basic cost structures of online learning, compared to the costs of classroom teaching, is an essential first step to increasing productivity in post-secondary education. It is risky to assume that online learning is always more cost-effective or productive; the circumstances need to be right.

3.Content is only one component of teaching (and an increasingly less important component); other components such as learner support and assessment are even more important. Care is needed then because changes in methods of online content development and delivery could have negative knock-on cost and productivity consequences in other areas of course delivery, such as learner support and assessment. In looking at productivity issues, all these factors need to be examined together.

4. Any attempts at scaling or increasing economies of scope in content development and delivery need to be balanced by ensuring quality does not suffer. However, online course development has the potential, through good course design, to improve quality rather than reduce it.

5. The ‘learning that matters most’ mainly addresses university teaching, but also increasingly technical, vocational and corporate training; the aim is to develop the knowledge and skills needed in a knowledge-based society. Online learning can handle the ‘learning that matters most’ as well, in most cases, as on-campus teaching, although there will always be some exceptions.

6. However, there are major difficulties in scaling up the learner support and assessment activities that are needed for the learning that matters most, both online or on campus. The danger in scaling up is the loss of quality in terms of learning outcomes.

7. Adaptive learning software that helps individualize learning, and learning analytics, may help to a small degree in enabling instructors to handle slightly more students without loss of quality, but cannot as yet replace a skilled instructor, and probably never will. Higher education requires expertise and qualitative assessment for the learning that matters most, and that will need human instructors.

8. New online course designs built around the use of new technologies have greater potential for increases in productivity – through producing better learning outcomes – for the learning that matters most, than through scaling up, i.e. by increasing teacher:student ratios.

9. We need more empirical research on the relationship between teaching methods, mode of delivery, costs, and the type of learning outcomes that constitute the ‘learning that matters most’ (not to mention better definitions).

Now it’s your turn

As I’ve said, this has been a struggle for me to work through the issues of online learning and productivity. The whole purpose of this arduous exercise is to promote debate and discussion about productivity and online learning. So I’d really welcome your comments. It would be great to hear from people with experience in productivity in other areas besides education, and to hear from those in education about the potential or dangers of applying the thinking around productivity to online learning.

So go for it!

Other posts in the series

© Ann Helmond 2009

© Ann Helmond 2009

Improving productivity in online learning: can we scale ‘the learning that matters most’?

Listen with webReader
Can 'the magic of the campus' be replicated online - and at scale?

Can ‘the magic of the campus’ be replicated online – and at scale?

The story so far

This is a continuation of the discussion on whether online learning can increase educational ‘productivity.’ Previous posts in this series include:

There is a CIDER webinar presentation on the HEQCO report available from here

In the last post, I concluded:

  • there are major economies of scale in using computer-based feedback for facilitating comprehension and technical mastery outcomes
  • computer-based feedback, when well designed, can also be useful in providing student feedback for more complex forms of learning, such as alternative strategies, critical thinking and evaluation
  • however computer-based analyses to date are inadequate for formal assessment of these higher order learning skills, where deep expertise and qualitative assessment is required, and where learners may provide new insights or alternative explanations
  • redesign of courses with a greater focus on student discovery (finding, analyzing and applying content) within a learning design offers more modest but still significant potential for increases in productivity, mainly through better learning outcomes (development of 21st century skills) and through more effective use of senior research professors’ time.

Learner-instructor interaction and economies of scale

In this current post, I examine particularly the learner-instructor interaction, and discuss whether online learning can provide economies of scale in this area. This is particularly important, because research on credit-based online learning has shown that course delivery (which includes both learner support and student assessment) accounted for the largest overall cost of an online program (37%), almost three times more than course development, over the life of an online program (Bates and Sangra, 2011).

Can we scale ‘the learning that matters most’?

This important question has been raised in the HEQCO report by Tom Carey and David Trick. It is this issue I wish to address here, since scaling up the delivery of content, and learner-content interaction, through online learning is relatively easy, although both depend on good course design for effective learning.

What is more challenging is whether we can also scale the kind of ‘learning that matters most’, namely helping students when they struggle with new concepts or ideas, helping students to gain deep understanding of a topic or subject, helping students to evaluate a range of different ideas or practices, providing students with professional formation or development, understanding the limits of knowledge, and above all enabling students to find, evaluate and apply knowledge appropriately in new or ill-defined contexts.

Before looking at whether or not such activities can be scaled, it is important to challenge the view, such as Sanjay Sharma’s at MIT, that such forms of learning can only be achieved on campus. There is also more than a hint of this assumption in the HEQCO report, at least with respect to undergraduate education. Those of us who have taught online will know that it is possible to develop these kinds of learning outcomes online, especially but not exclusively at graduate level. Strategies such as scaffolding or supporting knowledge construction through online discussion and dialogue, student reflection through e-portfolios, and above all personal online interventions and communication between students and instructor, have all been found to lead to learning outcomes at least as equivalent to those of students studying the same subjects on-campus (see references below).

There will remain a relatively few learning activities that matter most that are best done on campus, such as the development of hands-on skills, but there will be others, such as knowledge management, that may well be best done online. More importantly, there will be some students who really need the environment provided by a campus, and others that will prefer an online environment.

The issue is not can the learning that matters most be done online, but can it be scaled up through online learning? Certainly, I would argue that the main criticism of xMOOCs is that they spectacularly fail to address this form of learning. However, cMOOCs, when they operate at the level of communities of practice with relatively shared levels of understanding and knowledge among the participants, do have at least the potential for such economies of scale while maintaining or even improving quality of learning outcomes. The challenge though is how one accounts for the hidden costs of the participation of experts in such professional sharing, which rely heavily on volunteering or ‘moonlighting’ from a paid job by those with the expertise. I suspect though that even if these costs were calculated, they would still prove more ‘productive’ than conventional campus-based classes for this type of learner. However, the cost-effectiveness research has yet to be done.

The challenge though is scaling up the kinds of interaction between students and instructors that enable diagnosis of a student’s learning difficulties, that facilitate deep understanding of a subject, that encourage creative and original thinking, especially within undergraduate education. Adaptive learning and learning analytics may help to some extent, but in my view cannot yet come close to matching the skill of an experienced and skilled instructor. If instructors are to have enough time to engage in these kinds of dialogue and communication with students, there is clearly a limit on the number of students they can handle. Thus there is a possibility of small increases in productivity, aided by developments such as adaptive learning and learning analytics, but not major ones, in this aspect of teaching and learning.

Scaling the assessment of ‘learning that matters most.’

When ‘the magic of the campus’ is raised, one of the implicit assumptions is that student assessment is more valid because of the personal knowledge that faculty develop of a student in their entirety, and not just in their formal academic work: how they conduct themselves in class discussion (not just what they say, but how they say it), their interests and knowledge outside the formal curriculum (e.g. do they read widely or participate in valued extra-curricula activities), and the impression students make in social activities with faculty. This ‘tacit’ knowledge of a particular student that faculty acquire on campus can heavily influence the final assessment of a student, beyond that of the final exam. As they say at Oxford University, ‘Is he one of us?’

I was fortunate to have done my undergraduate degree in a department where every ‘honours’ student was well known by every faculty member. We were told that in the final exam, we could not get a worse grade than was already determined, but we could improve on it by a really good performance. In other words, the final exam was more of a rite of passage – the assessment was already more or less in place. This was only possible because of the ‘deep’ knowledge that faculty had already gained of the students. The fear that many faculty have of of online learning is that this kind of knowledge of a student is impossible ‘at a distance.’

Again, however, at least some elements of this ‘getting to know students’ can be achieved online, through continuous assessment, the use of e-portfolios and participation in online discussions. Again, the similarities between online learning and campus teaching are often greater than the differences. The problem is scaling up this kind of in-depth academic relationship between student and instructor, both for classroom and online teaching. Although the actual ratio may be difficult to specify, it is clear that this kind of relationship cannot be built up if the instructor:student ratio is in the thousands.

The fact is though that undergraduate students in most public universities are not in the fortunate position that I was. Even in their final year, many find themselves are in classes of over 100 students. They will probably be better off in an online class of 30 students, and even in an online class of 100, they may have more personal interaction with the instructor than in a lecture theatre, if the course is well designed. However, scaling up much beyond this ratio is not going to enable the more personal intellectual relationship to develop that allows for the more informal ‘I know what this student is capable of’ relationship, either online or on campus.

In short, for assessment based on deep knowledge of a student’s progress and capabilities, the scope for economies of scale are limited. In this sense, teacher:student ratios do matter, so economies of scale through online learning will be difficult to achieve for these higher order learning skills.

Conclusions

This has been a particularly difficult blog to write which suggests I may still not be thinking clearly about this topic, so please help me out! However, here is where I stand on this issue so far:

1. The ‘learning that matters most’ mainly addresses university teaching, but I suspect also increasingly technical, vocational and corporate training; the aim is to develop the knowledge and skills needed in a knowledge-based society.

2. Online learning can handle the ‘learning that matters most’ as well, in most cases, as on-campus teaching, although there will always be some exceptions.

3. However, there are major difficulties in scaling up the learner support and assessment activities that are needed for the learning that matters most, both online or on campus. The danger in scaling up is the loss of quality in terms of learning outcomes.

4. Adaptive learning software that helps individualize learning, and learning analytics, may help to a small degree in enabling instructors to handle slightly more students without loss of quality, but cannot as yet replace a skilled instructor, and probably never will.

5. New online course designs built around the use of new technologies have greater potential for increases in productivity – through producing better learning outcomes – for the learning that matters most, than through scaling up, i.e. by increasing teacher:student ratios.

6. We need more empirical research on the relationship between teaching methods, mode of delivery, costs, and the type of learning outcomes that constitute the ‘learning that matters most’ (not to mention better definitions).

Your input

First I’d really welcome responses to this post. In particular:

  • Is ‘the learning that matters most’ a useful concept for university teaching? Do you agree with my descriptions of it?
  • Have I missed something obvious in the possibility for scaling these learner support and assessment activities?
  • Can adaptive learning software and learning analytics take some or all of the load off instructors in developing such learning outcomes?
  • What would new online course designs that increase productivity look like? Do you have actual examples that have been implemented?

Next

In my next post on this topic, I will discuss an area where I think there is huge potential for increasing productivity through online learning, and that is through savings in physical overheads.

References

Anderson, T., Rourke, L., Garrison, R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, Vol. 5, No.2.

Baker, C. (2010) The Impact of Instructor Immediacy and Presence for Online Student Affective Learning, Cognition, and Motivation The Journal of Educators Online Vol. 7, No. 1

Bates, A. and Sangrà, A. (2011) Managing Technology in Higher Education: Strategies for Transforming Teaching and Learning San Francisco: Jossey-Bass/John Wiley and Son

Garrison, D. R. & Cleveland-Innes, M. (2005). Facilitating cognitive presence in online learning: Interaction is not enough. American Journal of Distance Education, Vol. 19, No. 3

Harasim, L. (2012) Learning Theory and Online Technologies New York/London: Routledge

Jonassen, D., Davidson, M., Collins, M., Campbell, J. and Haag, B. (1995) ‘Constructivism and Computer-mediated Communication in Distance Education’, American Journal of Distance Education, Vol. 9, No. 2, pp 7-26.

Paloff, R. and Pratt, K. (2007) Building Online Learning Communities San Francisco: John Wiley and Co.

Richardson, J. C., & Swan, K. (2003). Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7 (1), 68-8 8.

Salmon, G. (2000) E-moderating London/New York: Routledge

Sheridan, K. and Kelly, M.  (2010) The Indicators of Instructor Presence that are Important to Students in Online Courses MERLOT Journal of Online Learning and Teaching, Vol. 6, No. 4

 

 

MOOCs, MIT and Magic

Listen with webReader

MOOC panel: Dan Hastings, Anant Agarwal, Tony Bates, Sanjay Sarma, John Daniel

In my previous post, there were two sessions at the LINC 2013 conference that referred specifically to MIT’s own strategies for technology-enabled learning within MIT. These resulted in my asking the following question towards the end of the conference:

Why is MIT ignoring 25 years of research into online learning and 100 years research into how students learn in its design of online courses?

This post then will discuss both why I think this is the case, based on MIT’s own presentations at the conference, and the broader implications for educational research and instructional design.

MOOCs, MIT and Magic

The first session at the LINC conference was on four perspectives on MOOCs. There were four speakers before the coffee break, then the four speakers formed a panel to respond to questions from the audience after the coffee break. All four presentations are available in full from here, so I will provide a very brief summary of the main points made by each presenter. The session presenters were introduced by Richard Larson, Director of LINC.

First though, MIT’s Chancellor, Eric Grimson, laid out the reasons why MIT is making such a large commitment to OpenCourseWare, MOOCs and edX, and these reasons were reinforced by other MIT speakers:

  • to rethink the campus experience in the light of developments in online learning
  • increase access to learning worldwide by making MIT resources and courses available to anyone, anywhere
  • to conduct research on learning, especially by mining and analyzing the large amount of data generated by MOOCs
  • Anant Agarwal, the Director of edX, also later added: to develop an open source platform for (massive) online learning.

Sanjay Sarma, Director of MIT’s Office of Digital Learning, opened the session. He made the distinction between MOOCs as open courses available to anyone, reflecting the highest level of knowledge in particular subject areas, and the ‘magic’ of the on-campus experience, which is distinctly different from the online experience. He argued that it is difficult to define or pin down the magic that takes place on-campus, but referred to ‘in-the-corridor’ conversations between faculty and staff, hands-on engineering with other students outside of lectures and scheduled labs, and the informal learning that takes place between students in close proximity to one another. Not mentioned but implicit was also the very high standard of students admitted to MIT, and the impact of continuous contact on campus between student and professor, none of which of course is available to MOOC students. MOOCs however as well as providing a route to high quality learning for self-directed learners can be also be re-used and incorporated by other instructors in other institutions for credit.

Sir John Daniel took a much more critical view of MOOCs, suggesting, using the Gartner ‘hype cycle’, that MOOCs would soon enter the ‘trough of disillusionment’ and reflected on whether or how MOOCs will reach the “plateau of productivity”. He also pointed out that open and virtual universities in both developed and developing countries have been providing open and distance learning on a massive scale for over 40 years, and these initiatives have provided high quality and recognized qualifications.

Professor Anant Agarwal, the President of edX, provided some facts and figures about edX MOOCs, and mentioned that MIT had awarded a scholarship to the 15 year old Mongolian student who scored 100% on the final exam of an MIT MOOC course (although he will not receive credit for it). He pointed out that although over 150,000 learners enrolled in edXs first MOOC, 26,000 did the first activity, and 7,000 went on to complete successfully the certificate based on an online exam. (This woud provide a completion rate of approximately 28%, which is probably the most valid way to calculate completion rates for MOOCs.) More importantly, Agarwal defined the pedagogical ‘innovations’ in MOOCs as follows:

  • active learning: short video lectures interspersed with student tests/activities
  • self-paced learning
  • instant feedback
  • simulations/online labs to teach design of experiments
  • peer-to-peer learning.

Some of this has been made possible by MIT engineers building original software for automatic grading or feedback, including enabling students to write formulae as answers.

Me, MOOCs and pedagogy

I was the last speaker in this session and focused on the pedagogy of MOOCs, and suggested some ways in which they could be improved, based on 25 years of research in online learning. In summary the basic points I made are as follows:

  • MOOCs face several challenges, in particular low completion rates, problems with student  assessment, especially for assessment that requires qualitative or essay-type answers, and poor Internet access in developing countries
  • there is 25 years of experience and research into what works and what doesn’t in online learning
  • by and large, this knowledge is not being applied to the design of edX or Coursera MOOCs, which are based mainly on video recordings of classroom lectures
  • paying more attention to pedagogical issues and instructional design could help mitigate some of the challenges
  • in particular more attention needs to be paid to skills development, knowledge construction/deep learning and learner support
  • research should focus on course designs that focus on skills development rather than the transmission of information, on how to scale up learner support and oncosting models that provide resources for improved learner support
  • MOOCs should not be ‘second best’ for developing countries, replacing more locally based provision
  • for all this to happen, computer specialists and educators/instructional designers need to work together as equals

A copy of my presentation can be obtained by sending me an e-mail (tony.bates@ubc.ca) and I will send you an invitation via Dropbox to download the slides.

Technology-enabled learning: what’s going on at MIT?

This was the title of another session that described in more detail MIT’s other technology-enabled activities besides MOOCs. First I need to describe how MIT organizes its technology-enabled teaching and learning, based on the Executive Director of OpenCourseware, Cecilia d’Oliveira’s, clear presentation about 10 years history and the organizational structure of educational technology initiatives at MIT.

The Office of Digital Learning

Most of the better known MIT activities in this area come under the umbrella of the Office of Digital Learning, whose Director is Dr. Sanjay Sarma, a Professor of Mechanical Engineering.

Within this division is MIT OpenCour.seWare, which collects and provides a portal for video recordings of lectures and support materials that faculty have agreed can be shared openly. Currently MIT OCW offers materials from 2,150 MIT courses, plus courses from more than 300 universities worldwide. However, these are open educational materials (OERs), not full courses. Cecilia d’Oliveira, whose background is mainly in IT, is the Exective Director of MIT OpenCourseWare.

Also within the Division of Digital Learning is MITx, which works with faculty and academic departments to develop MOOCs (massive online courses, including currently 16 available at the moment through edX), and is responsible for the platform used not only for its own online courses but also for other edX courses. Some of these courses are available to MIT students for credit, as well as being open to other learners (but without credit).

While edX uses the MITx platform (which is open source and open to other developers) for its courses, edX is a ‘portal’ or stage for bringing together the MOOCs from MIT, Harvard and other partners in edX, such as UC Berkeley. There are currently 26 universities contributing MOOCs to edX, which is a non-profit organization supported mainly by a grant of $60 million from MIT and Harvard. Professor Anant Agarwal is the President of edX, and is Professor of Electrical Engineering and Computer Science and was formerly Director of the Computer Science and Artificial Intelligence Laboratory.

Media Production Services has approximately 25 staff who help with the video capturing and production for online courses, OCW, and other technical services.

Lastly, the Office of Educational Innovation and Technology (OEIT) is also within the Office of Digital Learning. OEIT works with faculty, staff and students to enable and promote the development and dissemination of innovative uses of technology in teaching and learning. Its focus is on innovative software and hardware to support learning and teaching. For instance, the ARTEMiS project is developing high-quality visualizations by applying the principles of visual communication and using the tools of modern computer graphics to create visualizations that accurately portray scientific and technological concepts. OEIT also maintains four physical Experimental Learning Environments (ELE) and a small pool of laptops for flexible deployment for innovative curricula.  These spaces are intended as incubators for testing new or different technologically enhanced pedagogical paradigms.  These physical spaces host a suite of technologies, applications and tools. The Director of OEIT is Dr. M.S. Vijay Kumar, who has a doctorate in academic computing in education.

Office of the Dean for Undergraduate Education

Also, within the Office of the Dean for Undergraduate Education, the Teaching and Learning Laboratory (TLL) collaborates with faculty, teaching assistants, and students to promote excellence in teaching and learning throughout the Institute, assisting with MIT-wide innovations in pedagogy, curriculum, and educational technology in STEM teaching and learning.  It also conducts research in teaching at MIT. Dr. Leslie Breslow, Senior Lecturer, Sloan School of Management, is the Director.

The Office of Faculty Support, provides support to help the faculty develop and coordinate the undergraduate curriculum and educational programming and provides grants and opportunities for faculty development . Professor Diana Henderson, Professor of Literature, is the Director.

Ten years of learning technology innovation at MIT

Research into teaching and learning at OEIT

There has been over 10 years of research and experimentation in teaching with technology at MIT. Brandon Murumatsu of OEIT described two current research projects. The first was the development of an online version of an MIT ‘hard’ course in mechanics and materials, traditionally delivered in class mainly by one hour 25 minute lectures and supported by problem sets that are done as homework. In the distance version, the lectures are recorded with a TA taking notes on the different topics or ‘chunks’ addressed during the lecture. This enables  the online video to be embedded in an interactive web page that is indexed and linked to the different ‘chunks’ of the lecture (see diagram below). Students can therefore search quickly for different parts of the lecture, slow down, speed up or repeat each ‘chunk, etc. No information was given on how successful this was.

The second experiment was a take a ‘flipped’ lecture class and embed assessment with immediate feedback into the lecture, through the use of simple multiple choice questions. This enabled a large set of data to be collected and anlyzed about how students responded to the different parts of the lecture. It was reported  that students love or even dream of getting the green checkmark when they get the answers right.

MIT student responses to online learning

The last session was in some ways the best of the conference. Three MIT students presented on their experiences of online learning. Sam Shames reported how he used online learning for his project work, exploring the ‘universe’ of online, open resources (in particular OCW) to help with problem-solving and project work, and in particular the opportunity it provides for students to find individual pathways through online OERs.

Ethan Solomon reported on his experience of taking four MOOCs. The good:

  • the ability to go over materials again and again
  • ability to go at one’s own pace
  • immediate feedback

The bad:

  • the limitation of multiple choice questions
  • MOOCs are mainly just lectures
  • difficulty of organizing massive numbers of students, especially in discussions.

Comments

I found the conference fascinating, for many reasons, but here are the main points I came away with.

1. MIT is making genuine efforts to open up its teaching, its materials and opportunities for learning across the world. It has invested very heavily in this, and many institutions, instructors and learners outside MIT are taking advantage. The quality of the content is often outstanding.

2. MIT is still tied though to the lecture as the main means of delivery for online learning. In fact, the MIT students on the panel showed that they understand the need to adopt a different approach to online learning better than the faculty.

3. MOOCs are the consequence of lecture capture technology. This technology makes it easy to move teaching online, but without changing the design of the teaching. This usually results in information transmission becoming the primary pedagogy, without addressing the many limitations of lectures, except the ability for asynchronous access, which is an important improvement on the ‘live’ lecture.

4. MIT  is using a behaviourist approach to its online learning, based mainly on Skinnerian thinking and research. Long lectures are still a core part of its campus pedagogy as well, but there is additional ‘magic’ provided on campus (informal and experiential learning and close contact with faculty) which is not available to its online learners. In my view, it is a mistake to believe that such ‘magic’ cannot be created online. It can, but it needs good course design based on sound educational principles.

5. If instructional designers exist at MIT, they play a minor role or have little power. This shows in both the design of its MOOCs and in the research being conducted.

6. In my view, MIT will struggle to make an impact on educational research if it continues to ignore the potential contribution of educators. It is as if researchers such as Piaget, Bruner, Vigotsky, Carl Rogers, Gagné, and many later researchers had never existed. Can you imagine anyone trying to develop a new form of transportation while deliberately ignoring  Newtonian mechanics? Yet this is what MIT is doing in its educational research. In fact, as the research described above shows, they are re-inventing the wheel. It was admiited that many of the results they are getting are not new but have been known for many years.

For me this is a tragedy. MIT’s engineers have so much to offer in helping to improve educational technology but it needs to be informed and embedded in theories of learning, and must take account of prior research, for it to gain traction and be of value. This means working in a team with educators who have the design and research knowledge and experience, and working with them as equal partners.

Of course, MIT does not need this advice. It is immensely successful and will continue to produce great engineers. But it could also do so much more.

Having said all that, I learned a great deal from the conference, was treated with immense courtesy, and I am very grateful for the invitation to attend.