June 21, 2018

More developments in teaching science online

Screen shot from A101’s Virtual Reality of Human Anatomy (YouTube)

Matthews, D. (2018) Scepticism over Google plan to replace labs with virtual reality, Times Higher Education, June 7

The Harvard Gazette (2018) Virtual lab to extend reach of science education Harvard Gazette, June 6

It was interesting that I came across these two completely separate news announcements on the same day.

Google and Labster

The THE article is about a partnership between Google and the Danish virtual reality company, Labster. Among the 30 ‘virtual reality’ labs planned are ones allowing training in confocal microscopy, gene therapy and cytogenetics.

Arizona State University, one of the major online providers in the USA, will be the first institution to use the labs in VR this autumn, launching an online-only biological sciences degree. It has worked with Labster to develop the VR labs. Students will require access to their own VR headsets such as Google’s Daydream View, which costs US$99, used in combination with specific brands of smartphones. 

Harvard and Amgen

The second article from the Harvard Gazette announces a partnership between the Amgen Foundation and edX at Harvard University to establish a platform called LabXchange, ‘an online platform for global science education that integrates digital instruction and virtual lab experiences, while also connecting students, teachers, and researchers in a learning community based on sharing and collaboration.’ 

The term ‘virtual lab’ is used differently from the Google/Labster sense. Amgen, a major biotechnology company in the USA, is investing $6.5 million in grant funding to Harvard University to develop, launch and grow the LabXchange platform for teachers and students globally. LabXchange will include a variety of science content, such as simulated experiments, but more importantly it will provide an online network to connect students, researchers and instructors to enable ‘learning pathways’ to be built around the online materials.

Comment

It is interesting and perhaps somewhat unnerving to see commercial companies in the USA moving so strongly into online science teaching in partnership with leading universities.

Of course, the THE had to choose a snarky headline suggesting that you can’t teach science wholly online, rather than have the headline focus on the innovation itself. As with all innovation, the first steps are likely to be limited to certain kinds of online teaching or experiments, and in the end it will come down as much to economic factors as to academic validity. Can virtual labs and online science teaching scale economically better than campus-based courses and at the same quality or better?

More importantly I would expect that the technology will lead to new and exciting approaches not only to science teaching, but also to science research. Already some researchers are using virtual reality and mathematical modelling to explore variations in DNA sequences, for instance. Virtual and augmented reality in particular will lead to science being taught differently online than in physical labs, for different purposes.

At the same time, the two developments are very different. The Google/Labster/ASU partnership is pushing hard the technology boundaries in teaching science, using proprietal VR, whereas the Harvard/Amgen/edX partnership is more of a networked open educational resource, providing access to a wide range of online resources in science. Both these developments in turn are different from remote labs, which provide online access to controlling ‘real’ experimental equipment.

Lastly, both new developments are what I call ‘We’re gonna’ projects. They are announcements of projects that have yet to be delivered. It will be interesting to see how much the reality matches the hype in two year’s time. In the meantime, it’s good to see online learning being taken seriously in science teaching. The potential is fascinating.

How serious should we be about serious games in online learning?

An excerpt from the video game ‘Therapeutic Communication and Mental Health Assessment’ developed at Ryerson University

In the 2017 national survey of online learning in post-secondary education, and indeed in the Pockets of Innovation project, serious games were hardly mentioned as being used in Canadian universities or colleges. Yet there was evidence from the Chang School Talks in Toronto earlier this month that there is good reason to be taking serious games more seriously in online learning.

What are serious games?

The following definition from the Financial Times Lexicon is as good a definition as any:

Serious games are games designed for a purpose beyond pure entertainment. They use the motivation levers of game design – such as competition, curiosity, collaboration, individual challenge – and game media, including board games through physical representation or video games, through avatars and 3D immersion, to enhance the motivation of participants to engage in complex or boring tasks. Serious games are therefore used in a variety of professional situations such as education, training,  assessment, recruitment, knowledge management, innovation and scientific research. 

So serious games are not solely educational, nor necessarily online, but they can be both.

Why are serious games not used more in online learning?

Well, partly because some see serious games as an oxymoron. How can a game be serious? This may seem trivial, but many game designers fear that a focus on education risks killing the main element of a game, its fun. Similarly, many instructors fear that learning could easily be trivialised through games or that games can cover only a very limited part of what learning should be about – it can’t all be fun. 

Another more pragmatic reason is cost and quality. The best selling video games for instance cost millions of dollars to produce, on a scale similar to mainstream movies. What is the compelling business plan for educational games? And if games are produced cheaply, won’t the quality – in terms of production standards, narrative/plot, visuals, and learner engagement – suffer, thus making them unattractive for learners?

However, probably the main reason is that most educators simply do not know enough about serious games: what exists, how they can be used, nor how to design them. For this reason, the ChangSchoolTalks, organised each year by the School of Continuing Studies at Ryerson University, this year focused on serious games.

The conference

The conference, held on May 3rd in Toronto, consisted of nine key speakers who have had extensive experience with serious games, organised in three themes:

  • higher education
  • health care
  • corporate

The presentations were followed by a panel debate and question and answer session. The speakers were:

This proved to be an amazingly well-selected group of speakers on the topic. In one session run by Sylvester Arnab, he had the audience inventing a game within 30 seconds. Teams of two were given a range of  existing games or game concepts (such as Dictionary or Jeopardy) and a topic (such as international relations) and had up to two minutes to create an educational game. The winning team (in less than 30 seconds) required online students in political sciences to represent a country and suggest how they should respond to selected Tweets from Donald Trump.

I mentioned in an earlier blog that I suffered from such information overload from recent conferences that I had to go and lie down. It was at this conference where that happened! It has taken three weeks for me even to begin fully processing what I learned.

What did I learn?

Probably the most important thing is that there is a whole, vibrant world of serious games outside of education, and at the same time there are many possible and realistic applications for serious games in education, and particularly in online learning. So, yes, we should be taking serious games much more seriously in online learning – but we need to do it carefully and professionally.

The second lesson I learned is that excellent online serious games can be developed without spending ridiculous amounts of money (see some examples below). At the same time, there is a high degree of risk. There is no sure way of predicting in advance that a new game will be successful. Some low-cost simple games can work well; some expensively produced games can easily flop. This means careful testing and feedback during development.

For these and other reasons, research being conducted at Ryerson University and funded by eCampus Ontario is particularly important. Naza Djafarova and colleagues at Ryerson’s Chang School of Continuing Education are conducting research to develop a game design guide to enhance the process by which multidisciplinary teams, engaged in the pre-production stage, approach the design of a serious game. They have developed a process called the Art of Game Design methodology, for multidisciplinary teams involved in the design of serious games, and appraised in participatory workshops.

The Chang School has already developed a few prototype games, including:

  • Lake Devo, a virtual learning environment enabling online role-play activity in an educational context. Learners work synchronously, using visual, audio, and text elements to create avatars and interact in online role-play scenarios.
  • Skills Practice: A Home Visit that promotes the application of knowledge and skills related to establishing a therapeutic nurse-client relationship and completing a mental health assessment. Students assume the role of a community health nurse assigned to complete a home visit. Working with nurses and professors from George Brown College, Centennial College this project is working to establish a ‘virtual hospital’ with several serious games focused on maternity issues.

Thus serious games are a relatively high risk, high return activity for online learning. This requires building on best practices in games design, both within and outside education, sharing, and collaboration. However, as we move more and more towards skills development, experiential learning, and problem-solving, serious games will play an increasingly important role in online learning. Best to start now.

Active learning at the Royal Military College of Canada

The interior of Currie Hall, RMC

The RMC

Following my trip to the UK Open University, I visited the Royal Military College in Kingston, Ontario, where I was a keynote speaker at a one day conference on active learning.

The RMC is the military college of the Canadian Armed Forces, and is a degree-granting university training military officers. RMC was established in 1876 and is the only federal institution in Canada with degree-granting powers. Programs are offered at the undergraduate and graduate levels, both on campus as well as through the college’s distance learning programme via the Division of Continuing Studies. It has a total of about 3,000 students, with about one-third part-time/distance and about 300 taking post-graduate studies. It is fully bilingual.

Active learning at the RMC

This was the rough theme of the conference, and it was interesting to see how the College is working to make its programs, both on-campus and online, more learner focused and interactive. I don’t have space to cover all the presentations, which without exception were excellent, so I will focus just on those that were of particular interest to me.

The importance of retrieval-practice for learning

This was an interesting presentation by Dr. Mathieu Gagnon, a psychology instructor at the RMC, basically about effective learning methods. He drew attention to research (Gagnon and Cormier, 2018) that suggests that students who spend time writing down or retrieving what they learn from reading do better in long-term retention than students who re-read the same text multiple times. Another factor is that distributed learning, where students take breaks rather than study intensively, is also more effective in long-term retention. (I hope I have got this right, as I didn’t take notes during his presentation….)

The art and science of flying

I used to have my own small plane, a Cessna 172, which I have flown from the west coast to the east coast of Canada and back. I loved flying my own plane, and although I knew about stall speeds, the use of flaps and ailerons, and so on, I never really understood the basic principles of aeronautics (which is why it is probably fortunate that I have stopped flying now because of my age).

So imagine my delight when I heard Dr. Billy Alan and Dr. Steve Lukits discuss a radical inter-disciplinary course they had designed that combined English literature (books and writing about flying) with aeronautical engineering, capturing both the beauty and magic of flying and its downright practicalities. Unfortunately the course is no longer extant (too many challenges for the administration), but surely we need more such inter-disciplinary courses in higher education. 

Wi-fi on buses

Sawyer Hogenkamp is doing a master’s thesis at Queen’s University on the use of wi-fi on school buses. He presented some staggering figures:

  • 30 million students in the U.S. and Canada ride the school bus every day.
  • 40% of Canadian school students take a school bus every school day
  • the average commute time is one hour or more in each direction

Many school districts are now putting wi-fi on to their buses that connect to their networks so students can study to and from school. This is particularly important for students in rural areas who often have no or slow speed wi-fi access at home.

Google is rolling out a program across the United States called Rolling Study Halls that includes devices as well as connectivity for use on school buses. They claim they are ‘reclaiming’ more than 1.5 million study hours in this way. 

Hogenkamp is researching the impact on learning and behaviour of students on buses with wi-fi. He stated that the first person to notify the school district if the wi-fi fails is the school bus driver, because of the impact on bus behaviour. To see a great three minute video of Sawyer’s research on bullying on school buses, see: http://www.queensu.ca/3mt/results-and-galleries/videos-2018

Active learning classrooms

Queen’s University is also located in Kingston, and there is clearly a great deal of collaboration and cross-teaching and research between the RMC and Queen’s. Several instructors from RMC, Major Vicki Woodside-Duggins, Dr. Bernadette Dechecci, Lt. Glen Whitaker, and Mrs. Annie Riel, and from Queen’s University, Dr. Andrea Philpson, discussed their use of active classrooms at Queen’s University.

In 2014, Queen’s University installed three different types of active classrooms:

  • a small classroom (capacity 45) with flexible configuration, movable chairs with arm rests for tablets or notes, and extensive whiteboard all around the room, a podium and a projector with a screen
  • a medium size classroom (capacity 70), with round tables for groups of six with power outlets and connections to several interactive displays around the walls, enabling students to work in collaboration around a table or in presentation mode to the whole class, and a podium that connects to all the screens or can be switched to just one screen
  • a large classroom (capacity 136), with rectangular tables for groups of up to eight with a monitor at the end of each table, a and a podium connected to all the monitors with can be switched to just one screen.

The medium-sized active classroom at Queen’s University

A study was conducted in 2014 (Leger, Chen, Woodside-Duggins and Riel, 2014) and found:

Overall, both student and instructors had overwhelmingly positive expectations and experiences in all three classrooms across disciplines and course levels. Initial impressions and expectations about the rooms were optimistic with students expecting “active” courses and no lecturing, and most instructors immediately changing their typical teaching approaches to adapt to the new environment. The data collected at the end of the term suggests most learning expectations were met, with students being highly engaged throughout the term as a consequence of instructors using more active teaching approaches.

I had the good fortune to present in the medium-sized classroom to faculty and staff in 2016 and can personally attest to how the configuration of the room impacts on how one presents and engages the audience. I have already written about how the increased use of blended learning will require more active classroom designs and the RMC presentation strongly reinforced this.

Five active learning exercises

Dr. Holly Ann Garnett rounded up the conference with an interactive workshop where she got everyone to try five exercises for engaging students, including:

  • ball toss
  • pass-a-problem
  • students teach the class
  • think-pair-share
  • snowball

As these are all classroom exercises, I won’t go into detail but you can find them described more fully here.

What I found interesting is that best practices in online learning provide very different student engagement activities, such as online class discussion, student mini-assignments, and online tests with immediate feedback, which I believe have the advantage of being more authentic.

Conclusion

As always, I learn more than I teach when I’m a keynote presenter. The RMC has been doing distance education now for more than 20 years and it was good to connect with some of the RMC pioneers in distance education as well as the current Dean of Continuing Studies, Dr. Grace Scoppio, who was a delightful host. But I was also impressed with the quality and the enthusiasm of all the presenters. I am very fortunate to have such an interesting job!

References

Gagnon, Ma. and Cormier, S. (2018) Retrieval Practice and Distributed Practice: The Case of French Canadian Students, Canadian Journal of School Psychology, May, 2018

Leger, A., Chen, V., Woodside-Duggins, V., and Riel, A. (2014)  Active Learning Classrooms in Ellis Hall, Kingston ON: Queen’s University

 

Open and remote labs from the UK Open University

The Open University’s remote access electron microscope set-up

On my recent visit to the UK Open University, I had the privilege of a guided tour of the Open University’s remote labs. These allow students to log on from anywhere and conduct experiments remotely. The tour was courtesy of Professor Nick Braithwaite, Associate Dean (Academic Excellence), Faculty of Science, Technology, Engineering & Mathematics.

Note that remote labs are somewhat different from simulated online experiments, where students interact by entering data or clicking and dragging on screen items. With remote labs, the equipment being operated is real, with the students actually controlling the equipment in real time as well as recording and interpreting data. 

The OpenScience Laboratory

The OpenScience Laboratory is a means of conducting authentic and rigorous investigations using real data and is globally available. It is an initiative of the Open University and the Wolfson Foundation. It includes:

  • Remote Experiments
  • Virtual instruments and interactive screen experiments
  • Online field investigations
  • 3D Immersive environments
  • Citizen Science
  • Research and development 

There are altogether more than 50 self-contained open educational resource modules in experimental science, in the OpenScience Laboratory, each taking somewhere between one to three hours of study to complete.

As an example, there is an experiment to identify what causes variation in species of heather on English moorland. It is a combination of an online video recorded on site in English moorland and guided student activities, such as taking simulated measurements and calculating and interpreting data. The video is divided in to 23 parts, showing how measurements are made in the field, how to calculate slope, water flow, and organic soil depth, and how to take simulated measurements, to test the hypothesis that different types of heather are associated with different levels of slope in moorlands. This took me a couple of hours to complete.

The heather hypothesis

The OpenSTEM labs

The Open STEM Labs are part of the OpenScience Laboratory project.

The OpenSTEM Labs connect students to state-of-the-art instrumentation and equipment for practical enquiries over the internet, where distance is no barrier and where access to equipment is available 24 hours a day.

Students and teachers access the equipment via a web browser through which they can view the experiment, send real-time control commands, monitor real-time performance and download data for subsequent analysis. Using remotely accessible hardware for laboratory and exploratory studies, ranging from electronics to chemical synthesis and from microscopes to telescopes, students are able to access the various instruments and other remote controlled resources virtually anytime from anywhere with an internet connection.

The new facilities are available to students studying Open University modules and may be available by subscription to other institutions of higher education.

Figure 1 below indicates the relationship between the Open Science Labs, OpenSTEM Labs and remote labs.

The Open University’s remote labs

Below are links to some of the diverse range of equipment available. Simply click on a link and this will take you to that experiment’s landing page, as seen by the OU’s students. Here you will then be able to access the equipment. Please note that you may have to book a session if all pieces of that equipment are being used by others. If you do book a session you should enter the experiment through the booking system at the allotted time. This will take you straight through to the equipment. (Not all these are currently operational at any one time and you may need to register first to get access).

The OU also has scanning electron microscopes, an auto-titrator, and a radio telescope available on request from those with direct experience of these curriculum areas. Please email OpenSTEM to arrange access and further briefing.

A student’s desktop view of the eye of a fly seen through the OU’s electron microscope. The student can manipulate the electron microscope to get different degrees of magnitude.

Many of the remote lab experiments are part of the Open University’s MSc in Space Science and Technology.  This includes student remote control of a model ‘Mars Rover’ operated in a mock-up of the surface of Mars.

The OU’s model of the Mars Rover

Comments

The Open University has added a new set of quality online resources in experimental science and technology to those currently offered by, among others:

I would welcome suggestions for other sources for high quality OER in experimental science and technology..

However, many more are still needed. We are still a long way from being able to build an entire high quality experimental science or technology curriculum with open educational resources. As well as increasing quantity, we need better quality resources that enable student activity and engagement, that include clearly understandable instructions, and that result in a high level of scientific inquiry. The Open University resources meet these standards, but not all other OER in this field do. Also there are issues of scalability. One needs enough students to justify the investment in software, production and equipment, especially for remote labs and quality simulations. Sharing of resources between institutions, and between departments within institutions, is therefore highly desirable.

Thus there is still a long way to go in this field, but progress is being made. If you teach science or engineering I recommend you look carefully at the Open University’s resources. It may stimulate you not only to integrate some of these resources into your own teaching, but also to create new resources for everyone.

Learning analytics, student satisfaction, and student performance at the UK Open University

There is very little correlation between student satisfaction and student performance. Image: Bart Rienties. Click on image to see the video.

Rienties, B. and Toetenel, L. (2016) The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules, Computers in Human Behaviour, Vol. 60, pp.333-341

Li, N. et al. (2017) Online learning experiences of new versus continuing learners: a large-scale replication study, Assessment and Evaluation in Higher Education, Vol. 42, No. 4, pp.657-672

It’s never too late to learn

It’s been a hectic month with two trips from Vancouver to Ontario and back and one to the UK and back, a total of four keynotes, two panel sessions and two one day consultancies. By the time I got to the end of the month’s travels, I had learned so much that at a conference in Toronto I had to go to my room and lie down  – I just couldn’t take any more!

At my age, it takes time to process all this new information, but I will try to summarise the main points of what I learned in the next three posts.

Learning analytics at the Open University

The Open University, with over 100,000 students and more than 1,000 courses (modules), and most of its teaching online in one form or another, is an ideal context for the application of learning analytics. Fortunately the OU has some of the world leaders in this field. 

At the conference on STEM teaching at the Open University that I attended as the opening keynote, the closing keynote was given by Bart Rienties, Professor of Learning Analytics at the Institute of Educational Technology at the UK Open University. Rienties and his team linked 151 modules (courses) and 111,256 students with students’ behaviour, satisfaction and performance at the Open University UK, using multiple regression models. 

His whole presentation (40 minutes, including questions) can be accessed online, and is well worth viewing, as it provides a clear summary of the results published in the two detailed papers listed above. As always, if you find my summary of results below of interest or challenging, I strongly recommend you view Bart’s video first, then read the two articles in more detail. Here’s what I took away.

There is little correlation between student course evaluations and student performance

This result is a bit of a zinger. The core dependent variable used was academic retention (the number of learners who completed and passed the module relative to the number of learners who registered for each module). As Rientes and Toetenel (p.340) comment, almost as an aside, 

it is remarkable that learner satisfaction and academic retention were not even mildly related to each other….Our findings seem to indicate that students may not always be the best judge of their own learning experience and what helps them in achieving the best outcome.’

The design of the course matters

One of the big challenges in online and blended learning is getting subject matter experts to recognise the importance of what the Open University calls ‘learning design.’ 

Conole (2012, p121) describes learning design as:

a methodology for enabling teachers/designers to make more informed decisions in how they go about designing learning activities and interventions, which is pedagogically informed and makes effective use of appropriate resources and technologies. LD is focussed on ‘what students do’ as part of their learning, rather than the ‘teaching’ which is focussed on the content that will be delivered.

Thus learning design is more than just instructional design.

However, Rienties at al. comment that ‘only a few studies have investigated how educators in practice are actually planning and designing their courses and whether this is then implemented as intended in the design phase.’ 

The OU has done a good job in breaking down some of the elements of learning design. The OU has mapped the elements of learning design in nearly 200 different courses. The elements of this mapping can be seen below (Rientes and Toetenal, 2016, p.335):

Rientes and Toetenel then analysed the correlations between each of these learning design elements against both learner satisfaction and learner performance. What they found is that what OU students liked did not match with learner performance. For instance, students were most satisfied with ‘assimilative’ activities, which are primarily content focused, and disliked communication activities, which are primarily social activities. However, better student retention was most strongly associated with communication activities, and overall, with the quality of the learning design.

Rientes and Toetenel conclude:

although more than 80% of learners were satisfied with their learning experience, learning does not always need to be a nice, pleasant experience. Learning can be hard and difficult at times, and making mistakes, persistence, receiving good feedback and support are important factors for continued learning….

An exclusive focus on learner satisfaction might distract institutions from understanding the impact of LD on learning experiences and academic retention. If our findings are replicated in other contexts, a crucial debate with academics, students and managers needs to develop whether universities should focus on happy students and customers, or whether universities should design learning activities that stretch learners to their maximum abilities and ensuring that they eventually pass the module. Where possible, appropriate communication tasks that align with the learning objectives of the course may seem to be a way forward to enhance academic retention.

Be careful what you measure

As Rientes and Toetenel put it:

Simple LA metrics (e.g., number of clicks, number of downloads) may actually hamper the advancement of LA research. For example, using a longitudinal data analysis of over 120 variables from three different VLE/LMS systems and a range of motivational, emotions and learning styles indicators, Tempelaar et al. (2015) found that most of the 40 proxies of simple” VLE LA metrics provided limited insights into the complexity of learning dynamics over time. On average, these clicking behaviour proxies were only able to explain around 10% of variation in academic performance.

In contrast, learning motivations, emotions (attitudes), and learners’ activities during continuous assessments (behaviour) significantly improved explained variance (up to 50%) and could provide an opportunity for teachers to help at-risk learners at a relatively early stage of their university studies.

My conclusions

Student feedback on the quality of a course is really important but it is more useful as a conversation between students and instructors/designers than as a quantitative ranking of the quality of a course.  In fact using learner satisfaction as a way to rank teaching is highly misleading. Learner satisfaction encompasses a very wide range of factors as well as the teaching of a particular course. It is possible to imagine a highly effective course where teaching in a transmissive or assimilative manner is minimal, but student activities are wide, varied and relevant to the development of significant learning outcomes. Students, at least initially, may not like this because this may be a new experience for them, and because they must take more responsibility for their learning. Thus good communication and explanation of why particular approaches to teaching have been chosen is essential (see my comment to a question on the video).

Perhaps though the biggest limitation of student satisfaction for assessing the quality of the teaching is the often very low response rates from students, limited evaluation questions due to standardization (the same questions irrespective of the nature of the course), and the poor quality of the student responses. This is no way to assess the quality of an individual teacher or a whole institution, yet far too many institutions and governments are building this into their evaluation of teachers/instructors and institutions.

I have been fairly skeptical of learning analytics up to now, because of the tendency to focus more on what is easily measurable (simple metrics) than on what students actually do qualitatively when they are learning. The focus on learning design variables in these studies is refreshing and important but so will be analysis of student learning habits.

Finally, this research provides quantitative evidence of the importance of learning design in online and distance teaching. Good design leads to better learning outcomes. Why then are we not applying this knowledge to the design of all university and college courses, and not just online courses? We need a shift in the power balance between university and college subject experts and learning designers resulting in the latter being treated as at least equals in the teaching process.

References

Conole, G. (2012). Designing for learning in an open world. Dordrecht: Springer

Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: learning analytics in a data-rich context. Computers in Human Behavior, 47, 157e167. http://dx.doi.org/10.1016/j.chb.2014.05.038.