MOOC panel: Dan Hastings, Anant Agarwal, Tony Bates, Sanjay Sarma, John Daniel

In my previous post, there were two sessions at the LINC 2013 conference that referred specifically to MIT’s own strategies for technology-enabled learning within MIT. These resulted in my asking the following question towards the end of the conference:

Why is MIT ignoring 25 years of research into online learning and 100 years research into how students learn in its design of online courses?

This post then will discuss both why I think this is the case, based on MIT’s own presentations at the conference, and the broader implications for educational research and instructional design.

MOOCs, MIT and Magic

The first session at the LINC conference was on four perspectives on MOOCs. There were four speakers before the coffee break, then the four speakers formed a panel to respond to questions from the audience after the coffee break. All four presentations are available in full from here, so I will provide a very brief summary of the main points made by each presenter. The session presenters were introduced by Richard Larson, Director of LINC.

First though, MIT’s Chancellor, Eric Grimson, laid out the reasons why MIT is making such a large commitment to OpenCourseWare, MOOCs and edX, and these reasons were reinforced by other MIT speakers:

  • to rethink the campus experience in the light of developments in online learning
  • increase access to learning worldwide by making MIT resources and courses available to anyone, anywhere
  • to conduct research on learning, especially by mining and analyzing the large amount of data generated by MOOCs
  • Anant Agarwal, the Director of edX, also later added: to develop an open source platform for (massive) online learning.

Sanjay Sarma, Director of MIT’s Office of Digital Learning, opened the session. He made the distinction between MOOCs as open courses available to anyone, reflecting the highest level of knowledge in particular subject areas, and the ‘magic’ of the on-campus experience, which is distinctly different from the online experience. He argued that it is difficult to define or pin down the magic that takes place on-campus, but referred to ‘in-the-corridor’ conversations between faculty and staff, hands-on engineering with other students outside of lectures and scheduled labs, and the informal learning that takes place between students in close proximity to one another. Not mentioned but implicit was also the very high standard of students admitted to MIT, and the impact of continuous contact on campus between student and professor, none of which of course is available to MOOC students. MOOCs however as well as providing a route to high quality learning for self-directed learners can be also be re-used and incorporated by other instructors in other institutions for credit.

Sir John Daniel took a much more critical view of MOOCs, suggesting, using the Gartner ‘hype cycle’, that MOOCs would soon enter the ‘trough of disillusionment’ and reflected on whether or how MOOCs will reach the “plateau of productivity”. He also pointed out that open and virtual universities in both developed and developing countries have been providing open and distance learning on a massive scale for over 40 years, and these initiatives have provided high quality and recognized qualifications.

Professor Anant Agarwal, the President of edX, provided some facts and figures about edX MOOCs, and mentioned that MIT had awarded a scholarship to the 15 year old Mongolian student who scored 100% on the final exam of an MIT MOOC course (although he will not receive credit for it). He pointed out that although over 150,000 learners enrolled in edXs first MOOC, 26,000 did the first activity, and 7,000 went on to complete successfully the certificate based on an online exam. (This woud provide a completion rate of approximately 28%, which is probably the most valid way to calculate completion rates for MOOCs.) More importantly, Agarwal defined the pedagogical ‘innovations’ in MOOCs as follows:

  • active learning: short video lectures interspersed with student tests/activities
  • self-paced learning
  • instant feedback
  • simulations/online labs to teach design of experiments
  • peer-to-peer learning.

Some of this has been made possible by MIT engineers building original software for automatic grading or feedback, including enabling students to write formulae as answers.

Me, MOOCs and pedagogy

I was the last speaker in this session and focused on the pedagogy of MOOCs, and suggested some ways in which they could be improved, based on 25 years of research in online learning. In summary the basic points I made are as follows:

  • MOOCs face several challenges, in particular low completion rates, problems with student  assessment, especially for assessment that requires qualitative or essay-type answers, and poor Internet access in developing countries
  • there is 25 years of experience and research into what works and what doesn’t in online learning
  • by and large, this knowledge is not being applied to the design of edX or Coursera MOOCs, which are based mainly on video recordings of classroom lectures
  • paying more attention to pedagogical issues and instructional design could help mitigate some of the challenges
  • in particular more attention needs to be paid to skills development, knowledge construction/deep learning and learner support
  • research should focus on course designs that focus on skills development rather than the transmission of information, on how to scale up learner support and oncosting models that provide resources for improved learner support
  • MOOCs should not be ‘second best’ for developing countries, replacing more locally based provision
  • for all this to happen, computer specialists and educators/instructional designers need to work together as equals

A copy of my presentation can be obtained by sending me an e-mail (tony.bates@ubc.ca) and I will send you an invitation via Dropbox to download the slides.

Technology-enabled learning: what’s going on at MIT?

This was the title of another session that described in more detail MIT’s other technology-enabled activities besides MOOCs. First I need to describe how MIT organizes its technology-enabled teaching and learning, based on the Executive Director of OpenCourseware, Cecilia d’Oliveira’s, clear presentation about 10 years history and the organizational structure of educational technology initiatives at MIT.

The Office of Digital Learning

Most of the better known MIT activities in this area come under the umbrella of the Office of Digital Learning, whose Director is Dr. Sanjay Sarma, a Professor of Mechanical Engineering.

Within this division is MIT OpenCour.seWare, which collects and provides a portal for video recordings of lectures and support materials that faculty have agreed can be shared openly. Currently MIT OCW offers materials from 2,150 MIT courses, plus courses from more than 300 universities worldwide. However, these are open educational materials (OERs), not full courses. Cecilia d’Oliveira, whose background is mainly in IT, is the Exective Director of MIT OpenCourseWare.

Also within the Division of Digital Learning is MITx, which works with faculty and academic departments to develop MOOCs (massive online courses, including currently 16 available at the moment through edX), and is responsible for the platform used not only for its own online courses but also for other edX courses. Some of these courses are available to MIT students for credit, as well as being open to other learners (but without credit).

While edX uses the MITx platform (which is open source and open to other developers) for its courses, edX is a ‘portal’ or stage for bringing together the MOOCs from MIT, Harvard and other partners in edX, such as UC Berkeley. There are currently 26 universities contributing MOOCs to edX, which is a non-profit organization supported mainly by a grant of $60 million from MIT and Harvard. Professor Anant Agarwal is the President of edX, and is Professor of Electrical Engineering and Computer Science and was formerly Director of the Computer Science and Artificial Intelligence Laboratory.

Media Production Services has approximately 25 staff who help with the video capturing and production for online courses, OCW, and other technical services.

Lastly, the Office of Educational Innovation and Technology (OEIT) is also within the Office of Digital Learning. OEIT works with faculty, staff and students to enable and promote the development and dissemination of innovative uses of technology in teaching and learning. Its focus is on innovative software and hardware to support learning and teaching. For instance, the ARTEMiS project is developing high-quality visualizations by applying the principles of visual communication and using the tools of modern computer graphics to create visualizations that accurately portray scientific and technological concepts. OEIT also maintains four physical Experimental Learning Environments (ELE) and a small pool of laptops for flexible deployment for innovative curricula.  These spaces are intended as incubators for testing new or different technologically enhanced pedagogical paradigms.  These physical spaces host a suite of technologies, applications and tools. The Director of OEIT is Dr. M.S. Vijay Kumar, who has a doctorate in academic computing in education.

Office of the Dean for Undergraduate Education

Also, within the Office of the Dean for Undergraduate Education, the Teaching and Learning Laboratory (TLL) collaborates with faculty, teaching assistants, and students to promote excellence in teaching and learning throughout the Institute, assisting with MIT-wide innovations in pedagogy, curriculum, and educational technology in STEM teaching and learning.  It also conducts research in teaching at MIT. Dr. Leslie Breslow, Senior Lecturer, Sloan School of Management, is the Director.

The Office of Faculty Support, provides support to help the faculty develop and coordinate the undergraduate curriculum and educational programming and provides grants and opportunities for faculty development . Professor Diana Henderson, Professor of Literature, is the Director.

Ten years of learning technology innovation at MIT

Research into teaching and learning at OEIT

There has been over 10 years of research and experimentation in teaching with technology at MIT. Brandon Murumatsu of OEIT described two current research projects. The first was the development of an online version of an MIT ‘hard’ course in mechanics and materials, traditionally delivered in class mainly by one hour 25 minute lectures and supported by problem sets that are done as homework. In the distance version, the lectures are recorded with a TA taking notes on the different topics or ‘chunks’ addressed during the lecture. This enables  the online video to be embedded in an interactive web page that is indexed and linked to the different ‘chunks’ of the lecture (see diagram below). Students can therefore search quickly for different parts of the lecture, slow down, speed up or repeat each ‘chunk, etc. No information was given on how successful this was.

The second experiment was a take a ‘flipped’ lecture class and embed assessment with immediate feedback into the lecture, through the use of simple multiple choice questions. This enabled a large set of data to be collected and anlyzed about how students responded to the different parts of the lecture. It was reported  that students love or even dream of getting the green checkmark when they get the answers right.

MIT student responses to online learning

The last session was in some ways the best of the conference. Three MIT students presented on their experiences of online learning. Sam Shames reported how he used online learning for his project work, exploring the ‘universe’ of online, open resources (in particular OCW) to help with problem-solving and project work, and in particular the opportunity it provides for students to find individual pathways through online OERs.

Ethan Solomon reported on his experience of taking four MOOCs. The good:

  • the ability to go over materials again and again
  • ability to go at one’s own pace
  • immediate feedback

The bad:

  • the limitation of multiple choice questions
  • MOOCs are mainly just lectures
  • difficulty of organizing massive numbers of students, especially in discussions.

Comments

I found the conference fascinating, for many reasons, but here are the main points I came away with.

1. MIT is making genuine efforts to open up its teaching, its materials and opportunities for learning across the world. It has invested very heavily in this, and many institutions, instructors and learners outside MIT are taking advantage. The quality of the content is often outstanding.

2. MIT is still tied though to the lecture as the main means of delivery for online learning. In fact, the MIT students on the panel showed that they understand the need to adopt a different approach to online learning better than the faculty.

3. MOOCs are the consequence of lecture capture technology. This technology makes it easy to move teaching online, but without changing the design of the teaching. This usually results in information transmission becoming the primary pedagogy, without addressing the many limitations of lectures, except the ability for asynchronous access, which is an important improvement on the ‘live’ lecture.

4. MIT  is using a behaviourist approach to its online learning, based mainly on Skinnerian thinking and research. Long lectures are still a core part of its campus pedagogy as well, but there is additional ‘magic’ provided on campus (informal and experiential learning and close contact with faculty) which is not available to its online learners. In my view, it is a mistake to believe that such ‘magic’ cannot be created online. It can, but it needs good course design based on sound educational principles.

5. If instructional designers exist at MIT, they play a minor role or have little power. This shows in both the design of its MOOCs and in the research being conducted.

6. In my view, MIT will struggle to make an impact on educational research if it continues to ignore the potential contribution of educators. It is as if researchers such as Piaget, Bruner, Vigotsky, Carl Rogers, Gagné, and many later researchers had never existed. Can you imagine anyone trying to develop a new form of transportation while deliberately ignoring  Newtonian mechanics? Yet this is what MIT is doing in its educational research. In fact, as the research described above shows, they are re-inventing the wheel. It was admiited that many of the results they are getting are not new but have been known for many years.

For me this is a tragedy. MIT’s engineers have so much to offer in helping to improve educational technology but it needs to be informed and embedded in theories of learning, and must take account of prior research, for it to gain traction and be of value. This means working in a team with educators who have the design and research knowledge and experience, and working with them as equal partners.

Of course, MIT does not need this advice. It is immensely successful and will continue to produce great engineers. But it could also do so much more.

Having said all that, I learned a great deal from the conference, was treated with immense courtesy, and I am very grateful for the invitation to attend.

 

20 COMMENTS

  1. Dear Tony,

    I can’t speak about the impact educational research has on course design at MIT, although you sketch a sobering picture. About half a year ago, I attend a seminar by Harvard’s Katie Vale, who is tasked with doing research on the edX MOOC. I was quite impressed by what Harvard is doing, from what I understand theirs is a very serious effort to understand MOOCs. Their research certainly also is informed by instructional design principles and other insights that educational research has provided. I report more extensively on this in a blogpost. It is rather curious that Harvard and MIT are together in the edX business but do not seem to share research results (or, a possibility you suggest, MIT might choose to ignore them)

    http://pbsloep.blogspot.nl/2012/10/how-to-improve-teaching-with.html

  2. So telling is that the students can identify the problems, the gaps with edX MOOCs, yet edXs’ leadership cannot. edX and Coursera are recycling the same pedagogy that is not working in lecture halls, and packaging them us as ‘new and improved’.

    I watched a panel discussion [http://www.wgbhnews.org/post/college-20] recently with Anant Agarwal, and three other educators on Innovation Hub. Agarwal expressed his views, stating that MOOC platforms [edX] are the future of education, and are ‘transformative’. He appears to be not able to see the forest for the trees.

    Thanks for the post.

  3. I agree totally with your assessment of the xMOOCs you describe above. The x in xMOOCs represents transmission of knowledge (very engineering based meaning of the letter, but of course engineering is more than just that).

    I see that both you and others including myself have found it very easy to point out that ‘xMOOCs’ are behaviourist and they ignores the research done over several years with online learning. It also serves another point that face to face lectures are also not great teaching. But as they say, pointing 1 finger at others only leads to the rest of your 3 fingers pointing at yourself. xMOOCs has some how made it easy to point a finger at behaviourist approach/model of teaching and learning which has been difficult as that is what HE system mainly is made up of. So I am happy and hope that this will lead to reflections on the ways academics deliver their teaching and encourage them to do more face to face active learning.

    There is of course another type of MOOC known as cMOOC which is different. More open, more connectivist and more student centric and active. So it is not impossible to take into account the various theories that we all know exist that are not behaviourist.

    I am not defending MIT but I suspect that MIT has done it this way for a good reason. If they have not thought of this as follows then may be they should.
    1) they want to research behaviourist first – that is wha HE has been doing since so long – at least in the western world post gutenburg / post Plato.
    2) then they will have enough data to compare this with the cMOOCs and the likes
    3) there may also be technological limitations as Annant points out – the need to build a MOOC platform. This work is already developing.
    4) Mass education has always been ‘x’ – if you do it any other way you will fail miserably and wont get world attention – do it this way first and then improve.

    • Hi, manish

      Many thanks for your comments.

      However, I can’t agree that mass education must be behaviorist. The Open University used a variety of methods, some behaviorist, but others based on face-to-face discussion and active learning through summer schools.

      More recently credit-based online learning, through online discussion forums and increasingly the use of social media, have adopted more constructivist approaches. Mass education is not just about the size of a single course, but the numbers a whole system of education can reach. We have had mass k-12 education in most developed countries for a long time now, and with over 100 million students now studying credit online programs, one could argue that without MOOCs, online learning is helping to foster mass higher education.

      I believe that MOOCs can be a valuable addition to the mass higher education movement, but only if they help develop the skills and knowledge students will need in the 21st century. That I believe will require much more than a behaviourist approach to teaching.

      But thanks for bringing this up – it’s an important point for discussion,

      regards

      • Hi Tony,

        I should rephrase 4 to
        “Mass education has always been mainly ‘x’”

        I am an OU AL too, so I know what you are referring to. OU is also mass education provider but it is only one institution. Compared to so many other f2f institutions that do transmission of know-how/information.

        Prof. Mitra (Hole in the wall fame) is questioning that in future need for “Knowing” something (not knowledge) may become extinct (i.e. not needed any more). In that of course there is the threat to all who believe in the lecture or info. transmission as the best mass education activity. xMOOCs will become pointless then.

        Knowledge, or interrelations between information (held in memory or accessed online), will still be around and courses that engage people in knowledge building will stay useful. You can think of many OU courses or the cMOOCs as examples. In fact something in between xMOOC and cMOOCs will actually be practical and useful for most people for a very long time. We all know we still will need teachers, just their role wont be the same.

  4. Moocs could probably be better, but most of you must have been spoiled, because I can assure that for most people they are great… Maybe you should go to a free French university with 800 students in the classroom, and then you would understand that the lack of interaction with the teacher in the Moocs is very relative;)

  5. I interact with Ed Tech crowd a lot and they either don’t realize or simply don’t know that differentiated instruction in education is key to success – each subject, level, age group, culture group, and occasionally individual students require their own blend of teaching methods, something technology is simply not equipped to address. Online education can work for certain skills/subjects, but that success doesn’t transfer to everything else automatically.

    I could share so many stories about ed tech startup founders who don’t know the simplest things about education…

LEAVE A REPLY

Please enter your comment!
Please enter your name here