June 22, 2018

Open and remote labs from the UK Open University

The Open University’s remote access electron microscope set-up

On my recent visit to the UK Open University, I had the privilege of a guided tour of the Open University’s remote labs. These allow students to log on from anywhere and conduct experiments remotely. The tour was courtesy of Professor Nick Braithwaite, Associate Dean (Academic Excellence), Faculty of Science, Technology, Engineering & Mathematics.

Note that remote labs are somewhat different from simulated online experiments, where students interact by entering data or clicking and dragging on screen items. With remote labs, the equipment being operated is real, with the students actually controlling the equipment in real time as well as recording and interpreting data. 

The OpenScience Laboratory

The OpenScience Laboratory is a means of conducting authentic and rigorous investigations using real data and is globally available. It is an initiative of the Open University and the Wolfson Foundation. It includes:

  • Remote Experiments
  • Virtual instruments and interactive screen experiments
  • Online field investigations
  • 3D Immersive environments
  • Citizen Science
  • Research and development 

There are altogether more than 50 self-contained open educational resource modules in experimental science, in the OpenScience Laboratory, each taking somewhere between one to three hours of study to complete.

As an example, there is an experiment to identify what causes variation in species of heather on English moorland. It is a combination of an online video recorded on site in English moorland and guided student activities, such as taking simulated measurements and calculating and interpreting data. The video is divided in to 23 parts, showing how measurements are made in the field, how to calculate slope, water flow, and organic soil depth, and how to take simulated measurements, to test the hypothesis that different types of heather are associated with different levels of slope in moorlands. This took me a couple of hours to complete.

The heather hypothesis

The OpenSTEM labs

The Open STEM Labs are part of the OpenScience Laboratory project.

The OpenSTEM Labs connect students to state-of-the-art instrumentation and equipment for practical enquiries over the internet, where distance is no barrier and where access to equipment is available 24 hours a day.

Students and teachers access the equipment via a web browser through which they can view the experiment, send real-time control commands, monitor real-time performance and download data for subsequent analysis. Using remotely accessible hardware for laboratory and exploratory studies, ranging from electronics to chemical synthesis and from microscopes to telescopes, students are able to access the various instruments and other remote controlled resources virtually anytime from anywhere with an internet connection.

The new facilities are available to students studying Open University modules and may be available by subscription to other institutions of higher education.

Figure 1 below indicates the relationship between the Open Science Labs, OpenSTEM Labs and remote labs.

The Open University’s remote labs

Below are links to some of the diverse range of equipment available. Simply click on a link and this will take you to that experiment’s landing page, as seen by the OU’s students. Here you will then be able to access the equipment. Please note that you may have to book a session if all pieces of that equipment are being used by others. If you do book a session you should enter the experiment through the booking system at the allotted time. This will take you straight through to the equipment. (Not all these are currently operational at any one time and you may need to register first to get access).

The OU also has scanning electron microscopes, an auto-titrator, and a radio telescope available on request from those with direct experience of these curriculum areas. Please email OpenSTEM to arrange access and further briefing.

A student’s desktop view of the eye of a fly seen through the OU’s electron microscope. The student can manipulate the electron microscope to get different degrees of magnitude.

Many of the remote lab experiments are part of the Open University’s MSc in Space Science and Technology.  This includes student remote control of a model ‘Mars Rover’ operated in a mock-up of the surface of Mars.

The OU’s model of the Mars Rover

Comments

The Open University has added a new set of quality online resources in experimental science and technology to those currently offered by, among others:

I would welcome suggestions for other sources for high quality OER in experimental science and technology..

However, many more are still needed. We are still a long way from being able to build an entire high quality experimental science or technology curriculum with open educational resources. As well as increasing quantity, we need better quality resources that enable student activity and engagement, that include clearly understandable instructions, and that result in a high level of scientific inquiry. The Open University resources meet these standards, but not all other OER in this field do. Also there are issues of scalability. One needs enough students to justify the investment in software, production and equipment, especially for remote labs and quality simulations. Sharing of resources between institutions, and between departments within institutions, is therefore highly desirable.

Thus there is still a long way to go in this field, but progress is being made. If you teach science or engineering I recommend you look carefully at the Open University’s resources. It may stimulate you not only to integrate some of these resources into your own teaching, but also to create new resources for everyone.

Spotlight on online experiential learning at Ryerson University

Lake Devo is one of several e-learning initiatives at Ryerson University

Lake Devo is one of several e-learning initiatives at Ryerson University

A week or so ago, I had the opportunity to visit the Digital Education Strategies team at the G. Raymond Chang School of Continuing Education at Ryerson University, Toronto.

Ryerson is well known for its DMZ (formerly the Digital Media Zone), one of Canada’s largest business incubators for emerging tech start-ups, but it is by no means the only centre of innovation at Ryerson. As well as being responsible for the design of online learning courses at Ryerson, the Centre for Digital Education Strategies (CDES) has several very interesting e-learning initiatives. 

Online courses

The ‘bread and butter’ work of the CDES is the over 400 online courses, including around 300 degree-credit online and hybrid courses, four part-time degree online and blended programs, 23 fully online certificates, and 22 blended certificates. CDES serves roughly 23,000 online course enrolments a year. Ryerson recently moved from Blackboard to Desire2Learn learning management system to support most of its online courses.

Because of its expertise in online course design, Chang School’s Digital Education Strategies team has been engaged in a number of other innovative e-learning initiatives. The DES team has also built business efficiency tools and interactive learning applications. Each of these deserves a blog post on its own, but in this post I want to give a quick overview of some of the other work of the Centre.

1. Lake Devo

Lake Devo is a virtual learning environment enabling online role-play activity in an educational context. Learners work synchronously, using visual, audio, and text elements to create avatars and interact in online role-play scenarios.

The Lake Devo environment is fully equipped to allow an instructor to set up his/her class as an online collaborative community. He/she may enter students’ information, configure working groups and have the system issue login information to all users.

Lake Devo has been used by a total of ten online instructors, for at least eight different courses, involving over 35 sections of students. Students have developed over 100 different scenarios in Lake Devo (see “Gallery” for examples). 

 2. The Law Practice Program

This unique alternative to traditional articling was established by the Law Society of Upper Canada (LSUC) and Ryerson University to provide new options and flexibility to meet the legal profession’s licensing requirements for law graduates in Ontario.

The program features interactive web-based collaboration tasks that replicate the experience of working in a law firm. This virtual firm activity is combined with expert guidance and mentorship to equip candidates with the skills and competencies required for effective practice. For a promo video, see: https://www.youtube.com/watch?v=eKsu6P3ZUVQ

 3. Serious games

Mental health assessment during a home visit’ is a video-based game in which users practice their skills in a setting that is realistic and allows the user to make clinical choices within a safe environment.

This is another collaborative project involving Ryerson nursing faculty and professors from George Brown College and Centennial College.

4. Professional Development for Online Instructors

 As part of its commitment to offer high quality learning experiences for students, the CDES offers professional development for online instructors. Teaching Adult Learners Online (TALO) is a four-week, hands-on program designed to model effective facilitation techniques, and provide instructors with insight into the learning experiences of online students, while promoting an engaging community of practice.

Drawing on promising practices in online pedagogy and examples from leading open resources such as CU Open, TALO offers a unique experience that is helping to increase online instructor capacity and diversity.

I will do a more complete blog post on each of these initiatives over the next week or so.

Other initiatives

The Centre for Digital Education Strategies is involved in many other e-learning initiatives, including:

  • Providing training on foundations of instructional design principles to Pearson Canada Inc. employees.
  • Free multi-media e-learning modules to help Canadians boost their financial knowledge and plan for their future financial security for the Financial Consumer Agency of Canada (see: http://www.fcac-acfc.gc.ca/Eng/resources/educationalPrograms/financialBasics/Pages/elearning-apprligne.aspx)
  • A project for the Bombay Stock Exchange to design a train-the-trainer program for effective delivery of a hybrid curriculum on intercultural communication skills for the workplace.
  • A partnership with the University of the West Indies provided students in 12 Caribbean countries with access to a high-quality online programming for their Bachelor of Science in Nursing (BScN). 
  • Entrepreneurial mentor training through an online seminar using interactive case studies and role play.

Further information

 More details of the work of the Centre for Digital Education Strategies can be found here: http://ce-online.ryerson.ca/ce/default.aspx?id=3665

More detailed posts on each of the four projects listed above will follow shortly.

Can you do experiential learning online? Assessing design models for experiential learning

Loyalist College's virtual border crossing

Loyalist College’s virtual border crossing

One of the frustrating things about writing a book (Teaching in a Digital World) is that just when you think you’ve finished a chapter, you realise you have missed out something really important. I thought I’d covered the main learning design models when I became aware that I hadn’t covered experiential learning.

Nevertheless, I still remember at one of my presentations to faculty on online learning, a faculty member saying that you can’t do experiential learning online. I felt at the time that this was so mistaken a view, that I need to address it in my book. Indeed, experiential learning is like most design models: it is independent of the mode of delivery. What matters is how well it is done.

Now while I have had some experience of doing project-based learning in an elementary school in England many years ago, I don’t consider myself a specialist in experiential learning, especially at a post-secondary level, but I do have a bias towards embedding teaching and learning within real world contexts, where appropriate, while recognising that academic learning is about thinking in abstractions and generalisations. However, these need to be empirically based, and students need to move easily from the concrete to the abstract and back again, and done well, experiential learning should assist that process.

So here’s my stab at describing the various design models for experiential learning.

What is experiential learning?

Definition

Simon Fraser University defines experiential learning as:

the strategic, active engagement of students in opportunities to learn through doing, and reflection on those activities, which empowers them to apply their theoretical knowledge to practical endeavours in a multitude of settings inside and outside of the classroom.” 

There is a wide range of design models that aim to embed learning within real world contexts, including:

  • problem-based learning
  • case-based learning
  • project-based learning
  • inquiry-based learning
  • cooperative (work- or community-based) learning
  • apprenticeship.

I will be focusing on the first four of these design models (for a discussion of the apprenticeship model, and lab or studio work, see Chapter 4, Section 4.) The focus here is on some of the main ways in which experiential learning can be designed and delivered, with particular respect to the use of technology, and in ways that help develop the knowledge and skills needed in a digital age. (For a more detailed analysis of experiential learning, see Moon, 2004).

Core design principles

Experiential learning is a major form of teaching at the University of Waterloo. Its web site lists the conditions needed to ensure that experiential learning is effective, as identified by the Association for Experiential Education:

  • Experiential learning occurs when carefully chosen experiences are supported by reflection, critical analysis and synthesis.
  • Experiences are structured to require the student to take initiative, make decisions and be accountable for results.
  • Throughout the experiential learning process, the student is actively engaged in posing questions, investigating, experimenting, being curious, solving problems, assuming responsibility, being creative and constructing meaning.
  • Students are engaged intellectually, emotionally, socially, soulfully and/or physically. This involvement produces a perception that the learning task is authentic.
  • The results of the learning are personal and form the basis for future experience and learning.
  • Relationships are developed and nurtured: student to self, student to others and student to the world at large.
  • The instructor and student may experience success, failure, adventure, risk-taking and uncertainty, because the outcomes of the experience cannot totally be predicted.
  • Opportunities are nurtured for students and instructors to explore and examine their own values.
  • The instructor’s primary roles include setting suitable experiences, posing problems, setting boundaries, supporting students, insuring physical and emotional safety, and facilitating the learning process.
  • The instructor recognizes and encourages spontaneous opportunities for learning.
  • Instructors strive to be aware of their biases, judgments and pre-conceptions, and how these influence the student.
  • The design of the learning experience includes the possibility to learn from natural consequences, mistakes and successes.

Ryerson University in Toronto is another institution with extensive use of experiential learning, and also has an extensive web site on the topic, also directed at instructors. The next section examines different ways in which these principles have been applied.

Experiential design models

There are many different design models for experiential learning, but they also have many features in common.

Problem-based learning

The earliest form of systematised problem-based learning (PBL) was developed in 1969 by Howard Barrows and colleagues in the School of Medicine at McMaster University in Canada, from where it has spread to many other universities, colleges and schools. This approach is increasingly used in subject domains where the knowledge base is rapidly expanding and where it is impossible for students to master all the knowledge in the domain within a limited period of study. Working in groups, students identify what they already know, what they need to know, and how and where to access new information that may lead to resolution of the problem. The role of the instructor (usually called a tutor in classic PBL) is critical in facilitating and guiding the learning process.

Usually PBL follows a strongly systematised approach to solving problems, although the detailed steps and sequence tend to vary to some extent, depending on the subject domain. The following is a typical example

Figure 6. The Maastricht Seven-Jump PBL Tutorial Process (Gijeselaers, 1995)

Figure 6.10  (derived from Gijeselaers, 1995)

Traditionally, the first five steps would be done in a small face-to-face class tutorial of 20-25 students, with the sixth step requiring either individual or small group (four or five students) private study, with a the seventh step being accomplished in a full group meeting with the tutor. However, this approach also lends itself to blended learning in particular, where the research solution is done mainly online, although some instructors have managed the whole process online, using a combination of synchronous web conferencing and asynchronous online discussion.

Developing a complete problem-based learning curriculum is challenging, as problems must be carefully chosen, increasing in complexity and difficulty over the course of study, and problems must be chosen so as to cover all the required components of the curriculum. Students often find the problem-based learning approach challenging, particularly in the early stages, where their foundational knowledge base may not be sufficient to solve some of the problems. (The term ‘cognitive overload’ has been used to describe this situation.) Others argue that lectures provide a quicker and more condensed way to cover the same topics. Assessment also has to be carefully designed, especially if a final exam carries heavy weight in grading, to ensure that problem-solving skills as well as content coverage are measured.

However, research (see for instance, Strobel and van Barneveld, 2009) has found that problem-based learning is better for long-term retention of material and developing ‘replicable’ skills, as well as for improving students’ attitudes towards learning. There are now many variations on the ‘pure’ PBL approach, with problems being set after initial content has been covered in more traditional ways, such as lectures or prior reading, for instance.

Case-based learning

With case-based teaching, students develop skills in analytical thinking and reflective judgment by reading and discussing complex, real-life scenarios.

University of Michigan Centre for Research on Teaching and Learning,

Case-based learning is sometimes considered a variation of PBL, while others see it as a design model in its own right. As with PBL, case-based learning uses a guided inquiry method, but usually requires the students to have a degree of prior knowledge that can assist in analysing the case. There is usually more flexibility in the approach to case-based learning compared to PBL. Case-based learning is particularly popular in business education, law schools and clinical practice in medicine, but can be used in many other subject domains.

Herreid (2004) provides eleven basic rules for case-based learning.

  1. Tells a story.
  2. Focuses on an interest-arousing issue.
  3. Set in the past five years
  4. Creates empathy with the central characters.
  5. Includes direct quotations from the characters.
  6. Relevant to the reader.
  7. Must have pedagogic utility.
  8. Conflict provoking.
  9. Decision forcing.
  10. Has generality.
  11. Is short.

Using examples from clinical practice in medicine, Irby (1994) recommends five steps in case-based learning:

  • Anchor teaching in a (carefully chosen) case
  • Actively involve learners in discussing, analysing and making recommendations regarding the case
  • Model professional thinking and action as an instructor when discussing the case with learners
  • Provide direction and feedback to learners in their discussions
  • Create a collaborative learning environment where all views are respected

Case-based learning can be particularly valuable for dealing with complex, interdisciplinary topics or issues which have no obvious ‘right or wrong’ solutions, or where learners need to evaluate and decide on competing, alternative explanations. Case-based learning can also work well in both blended and fully online environments. Marcus, Taylor and Ellis (2004) used the following design model for a case-based blended learning project in veterinary science:

Figure 6.  Blended learning sequence involving online learning resources , Marcus, Taylor and Ellis, 2004

Figure 6.11 Blended learning sequence involving online learning resources, Marcus, Taylor and Ellis, 2004

Other configurations are of course also possible, depending on the requirements of the subject.

Project-based learning

Project-based learning is similar to case-based learning, but tends to be longer and broader in scope, and with even more student autonomy/responsibility in the sense of choosing sub-topics, organising their work, and deciding on what methods to use to conduct the project. Projects are usually based around real world problems, which give students a sense of responsibility and ownership in their learning activities.

Once again, there are several best practices or guidelines for successful project work. For instance, Larmer and Mergendoller (2010) argue that every good project should meet two criteria:

  • students must perceive the work as personally meaningful, as a task that matters and that they want to do well.
  • a meaningful project fulfills an educational purpose.

They then list seven essential elements of ‘meaningful’ projects:

  • need to know: provide students with a compelling ‘event’ (a video, news item, picture, guest lecturer) that they are asked to engage with
  • a driving question: ‘a good driving question captures the heart of the project in clear, compelling language, which gives students a sense of purpose and challenge. The question should be provocative, open-ended, complex, and linked to the core of what you want students to learn.
  • student voice and choice: students should be engaged in initial ‘brainstorming’ about the driving question, and have some choice in how they approach answering this question, both in terms of methods of inquiry, and in how the results of their study will be presented. Different sub-groups in the class then may operate in different ways.
  • 21st century skills: encourage the development of skills, particularly collaboration, team-building, role differentiation, oral, written and multimedia communication and reflection through journals or e-portfolios. Both instructor and students should be involved with assessment, which should include measurement of these skills.
  • inquiry and innovation: in order to tackle the driving question, students refine their own questions and line of inquiry, then seek out the information they need to answer their questions, then test their own ideas through discussion and further research. This may well lead to innovative suggestions for dealing with the issue under research.
  • feedback and revision: students should be encouraged to share their work with other students and be willing to give feedback and help each other. The instructor should structure more formal feedback so students are receiving help and guidance throughout the project, and encourage external forms of feedback from outside the institutional context, such as responses from relevant businesses or government agencies.
  • a publicly presented product: ideally the end product from the class should be presented to an external audience that has a major stake or interest in the issue under study, and should as far as possible offer constructive suggestions or conclusions.

The main danger with project-based learning is that the project can take a life of its own, with not only students but the instructor losing focus on the key, essential learning objectives, or important content areas may not get covered. Thus project-based learning needs careful design and monitoring by the instructor.

Inquiry-based learning

Inquiry-based learning (IBL) is similar to project-based learning, but the role of the teacher/instructor is somewhat different. In project-based learning, the instructor decides the ‘driving question’ and plays a more active role in guiding the students through the process. In inquiry-based learning, the learner explores a theme and chooses a topic for research, develops a plan of research and comes to conclusions, although an instructor is usually available to provide help and guidance when needed.

Banchi and Bell (2008) suggest that there are different levels of inquiry, and students need to begin at the first level and work through the other levels to get to ‘true’ or ‘open’ inquiry as follows:

Figure 6.x from Banchi and Bell (2008)

Figure 6.x from Banchi and Bell (2008)

It can be seen that the fourth level of inquiry describes the graduate thesis process, although proponents of inquiry-based learning have advocated its value at all levels of education.

Experiential learning in online learning environments

Advocates of experiential learning are often highly critical of online learning, because, they argue, it is impossible to embed online learning in real world examples. By its nature, online learning is not ‘real world.’ However, this is an oversimplification, and there are contexts in which online learning can be used very effectively to support or develop experiential learning, in all its variations:

  • blended or flipped learning: although group sessions to start off the process, and to bring a problem or project to a conclusion, are usually done in a classroom or lab setting, students can increasingly conduct the research and information gathering by accessing resources online, by using online multimedia resources to create reports or presentations, and by collaborating online through group project work or through critique and evaluation of each other’s work
  • fully online: increasingly, instructors are finding that experiential learning can be applied fully online, through a combination of synchronous tools such as a web conference, asynchronous tools such as discussion forums and/or social media for group work, and e-portfolios and multimedia for reporting.

Indeed, there are circumstances where it is impractical, too dangerous, or too expensive to use real world experiential learning. Online learning can be used to simulate real conditions and to reduce the time to master a skill. Flight simulators have long been used to train commercial pilots, enabling trainee pilots to spend less time mastering fundamentals on real aircraft. Commercial flight simulators are still extremely expensive to build and operate, but in recent years the costs of creating realistic simulations has dropped dramatically.

For instance, instructors at Loyalist College have created a ‘virtual’ fully functioning border crossing and a virtual car in Second Life to train Canadian Border Services Agents. Each student takes on the role of an agent, with his/her avatar interviewing the avatars of the travellers wishing to enter Canada. All communication is done by voice communications in Second Life, with the people playing the travellers in a separate room from the students. Each student interviews three or four travellers and the entire class observes the interactions and discusses the situations and the responses. A secondary site for auto searches features a virtual car that can be completely dismantled so students learn all possible places where contraband may be concealed. This learning is then reinforced with a visit to the auto shop at Loyalist College and the search of an actual car. The students in the customs and immigration track are assessed on their interviewing techniques as part of their final grades. Students participating in the first year of the Second Life border simulation achieved a grade standing that was 28 per cent higher than the previous class who did not utilize a virtual world. The next class, using Second Life, scored a further 9 per cent higher. More details can be found here.

Staff in the Emergency Management Division at the Justice Institute of British Columbia have developed a simulation tool called ExPod that helps to bring critical incidents to life by introducing real-world simulations into training and exercise programs. Because participants can access ExPod via the web, it provides the flexibility to deliver immersive, interactive and scenario-based training exercises anytime, anywhere. A typical emergency might be a major fire in a warehouse containing dangerous chemicals. ‘Trainee’ first responders, who will include fire, police and paramedical personnel, as well as city engineers and local government officials, are ‘alerted’ on their mobile phones or tablets, and have to respond in real time to a fast developing scenario, ‘managed’ by a skilled facilitator, following procedures previously taught and also available on their mobile equipment. The whole process is recorded and followed later by a face-to-face debriefing session.

Once again, design models are not in most cases dependent on any particular medium. The pedagogy transfers easily across different delivery methods.

Strengths and weaknesses of experiential learning models

How one evaluates experiential learning designs depends partly on one’s epistemological position. Constructivists strongly support experiential learning models, whereas those with a strong objectivist position are usually highly sceptical of the effectiveness of this approach. Nevertheless, problem-based learning in particular has proved to be very popular in many institutions teaching science or medicine, and project-based learning is used across many subject domains and levels of education. There is evidence that experiential learning, when properly designed, is highly engaging for students and leads to better long-term memory. Proponents also claim that it leads to deeper understanding, and develops ’21st century’ skills such as problem-solving, critical thinking, improved communications skills, and knowledge management. In particular, it enables learners to manage better highly complex situations that cross disciplinary boundaries, and subject domains where the boundaries of knowledge are difficult to manage.

Critics though such as Kirschner, Sweller and Clark (2006) argue that instruction in experiential learning is often ‘unguided’, and pointed to several ‘meta-analyses’ of the effectiveness of problem-based learning that indicated no difference in problem-solving abilities, lower basic science exam scores,  longer study hours for PBL students, and that PBL is more costly. They conclude:

In so far as there is any evidence from controlled studies, it almost uniformly supports direct, strong instructional guidance rather than constructivist-based minimal guidance during the instruction of novice to intermediate learners. Even with students with considerable prior knowledge, strong guidance when learning is most often found to be equally effective as unguided approaches.

Certainly, experiential learning approaches require considerable re-structuring of teaching and a great deal of detailed planning if the curriculum is to be fully covered. It usually means extensive re-training of faculty, and careful orientation and preparation of students. I would also agree with Kirschner et al. that just giving students tasks to do in real world situations without guidance and support is likely to be ineffective.

However, many forms of experiential learning can and do have strong guidance from instructors, and one has to be very careful when comparing matched groups that the tests of knowledge include measurement of the skills that are claimed to be developed by experiential learning, and are not just based on the same assessments as for traditional methods, which often have a heavy bias towards memorisation and comprehension.

On balance then, I would support the use of experiential learning for developing the knowledge and skills needed in a digital age, but as always, it needs to be done well, following best practices associated with the design models.

References

Banchi, H., and Bell, R. (2008). The Many Levels of Inquiry Science and Children, Vol. 46, No. 2

Gijselaers, W., (1995) ‘Perspectives on problem-based learning’ in Gijselaers, W, Tempelaar, D, Keizer, P, Blommaert, J, Bernard, E & Kapser, H (eds) Educational Innovation in Economics and Business Administration: The Case of Problem-Based Learning. Dordrecht, Kluwer.

Herreid, C. F. (2007). Start with a story: The case study method of teaching college science. Arlington VA: NSTA Press.

Irby, D. (1994) Three exemplary models of case-based teaching Academic Medicine, Vol. 69, No. 12

Kirshner, P., Sweller, J. amd Clark, R. (2006) Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching Educational Psychologist, Vo. 41, No.2

Larmer, J. and Mergendoller, J. (2010) Seven essentials for project-based learning Educational Leadership, Vol. 68, No. 1

Marcus, G. Taylor, R. and Ellis, R. (2004) Implications for the design of online case-based learning activities based on the student blended learning experience: Perth, Australia: Proceedings of the ACSCILITE conference, 2004

Moon, J.A. (2004). A Handbook of Reflective and Experiential Learning: Theory and Practice. New York: Routledge Falmer.

Strobel, J. , & van Barneveld, A. (2009). When is PBL More Effective? A Meta-synthesis of Meta-analyses Comparing PBL to Conventional Classrooms. Interdisciplinary Journal of Problem-based Learning, Vol. 3, No. 1

Comments, please

I’m really hoping to get some input from specialists in experiential learning. In particular:

1. Is what I’ve written an accurate description of the main models of experiential learning design?

2. Have I missed anything significant (given that this section is already too long)?

3. Do you agree that experiential learning can be done just as well online as in classrooms or in the field?

4. If you have had either really good or really bad experiences with experiential learning, what worked well and what didn’t?

Next

I have a final, wrap-up section on design models for learning, where I compare the different models’ strengths and weaknesses. Then the first draft of the chapter on design models will be complete and will be published.

Adapting student assessment to the needs of a digital age

Assessment 2

The story so far

Chapter 5 of my open textbook, ‘Teaching in a Digital Age’ is about the design of teaching and learning, which I am currently writing and publishing as I go.

I started Chapter 5 by suggesting that instructors should think about design through the lens of constructing a comprehensive learning environment in which teaching and learning will take place. I have been working through the various components of a learning environment, focusing particularly on how the digital age affects the way we need to look at some of these components.

I briefly described some of the key components of an effective learning environment in a series of blog posts:

In this post, I examine the assessment of students as a key component, and how assessment methods need to be adapted to meet the needs of a digital age. This is the last component I’m discussing, but it will be followed by a final post that discusses the value of designing teaching and learning through the lens of a comprehensive learning environment.

Learner assessment

‘I was struck by the way assessment always came at the end, not only in the unit of work but also in teachers’ planning….Assessment was almost an afterthought…

Teachers…are being caught between competing purposes of …assessment and are often confused and frustrated by the difficulties that they experience as they try to reconcile the demands.’

Earle, 2003

Learner assessment in a digital age

Because assessment is a huge topic, it is important to be clear that the purpose of this section is (a) to look at one of the components that constitute an effective and comprehensive learning environment, and (b) briefly to examine the extent to which assessment is or should be changing in a digital age. Assessment will be a recurring theme in this book, so in this section the treatment is deliberately cursory.

Probably nothing drives the behaviour of students more than how they will be assessed. Not all students are instrumental in their learning, but given the competing pressures on students’ time in a digital age, most ‘successful’ learners focus on what will be examined and how they can most effectively (i.e. in as little time as possible) meet the assessment requirements. Therefore decisions about methods of assessment will in most contexts be fundamental to building an effective learning environment.

The purpose of assessment

There are many different reasons for assessing learners. It is important to be clear about the purpose of the assessment, because it is unlikely that one single assessment instrument will meet all assessment needs. Here are some reasons (you can probably think of many more):

  1. to improve and extend students’ learning
  2. to assess students’ knowledge and competence in terms of desired learning goals or outcomes
  3. to provide the teacher/instructor with feedback on the effectiveness of their teaching and how it might be improved
  4. to provide information for employers about what the student knows and/or can do
  5. to filter students for further study, jobs or professional advancement
  6. for institutional accountability and/or financial purposes.

I have deliberately ordered these in importance for creating an effective learning environment. In terms of the needs of a digital age, assessment needs to focus on both developing and assessing skills. This means that continuous or formative assessment will be as important as summative or ‘end-of-course’ assessment.

A question to be considered is whether there is a need for assessment of learning in the first place. There may be contexts, such as a community of practice, where learning is informal, and the learners themselves decide what they wish to learn, and whether they are satisfied with what they have learned. In other cases, learners may not want or need to be formally evaluated or graded, but do want or need feedback on how they are doing with their learning. ‘Do I really understand this?’ or ‘How am I doing compared to other learners?’

However, even in these contexts, some informal methods of assessment by experts, specialists or more experienced participants could help other participants extend their learning by providing feedback and indicating the level of competence or understanding that a participant has achieved or has yet to accomplish. Lastly, students themselves can extend their learning by participating in both self-assessment and peer assessment, preferably with guidance and monitoring from a more knowledgeable or skilled instructor.

Methods of assessment

The form the assessment takes, as well as the purpose, will be influenced by the instructors’ or examiners’ underlying epistemology: what they believe constitutes knowledge, and therefore how students need to demonstrate their knowledge. The form of assessment should also be influenced by the knowledge and skills that students need in a digital age, which means focusing as much on assessing skills as knowledge of content.

There is a wide range of possible assessment methods. I have selected just a few to illustrate how technology can change the way we assess learners in ways that are relevant to a digital age:

  • computer-based multiple-choice tests: good for testing ‘objective’ knowledge of facts, ideas, principles, laws, and quantitative procedures in mathematics, science and engineering etc., and are cost-effective for these purposes. This form of testing though tends to be limited  in assessing high-level intellectual skills, such as complex problem-solving, creativity, and evaluation, and therefore less likely to be useful for developing or assessing many of the skills needed in a digital age.
  • written essays or short answers: good for assessing comprehension and some of the more advanced intellectual skills, such as critical thinking, but are labour intensive, open to subjectivity, and are not good for assessing practical skills. Experiments are taking place with automated essay marking, using developments in artificial intelligence, but so far automated essay marking still struggles with reliably identifying valid semantic meaning (for a balanced and more detailed account of the current state of machine grading, see Mayfield, 2013Parachuri, 2013).
  • project work: either individual but more commonly group-based, project work encourages the development of authentic skills that require understanding of content, knowledge management, problem-solving, collaborative learning, evaluation, creativity and practical outcomes. Designing valid and practical project work needs a high level of skill and imagination from the instructor.
  • e-portfolios (an online compendium of student work): enables self-assessment through reflection, knowledge management, recording and evaluation of learning activities, such as teaching or nursing practice, and recording of an individual’s contribution to project work (as an example, see  the use of e-portfolios in Visual Arts and Built Environment at the University of Windsor.); usually self-managed by the learner but can be made available or adapted for formal assessment purposes or job interviews
  • simulations, educational games (usually online) and virtual worlds: facilitate the practice of skills, such as complex and real time decision-making, operation of (simulated or remote) complex equipment, the development of safety procedures and awareness, risk taking and assessment in a safe environment, and activities that require a combination of manual and cognitive skills (see the training of Canadian Border Service officers at Loyalist College, Ontario). Currently expensive to develop, but cost-effective with multiple use, where it replaces the use of extremely expensive equipment, where operational activities cannot be halted for training purposes, or  where available as open educational resources.
Virtual world border crossing, Loyalist College, Ontario

Virtual world border crossing, Loyalist College, Ontario

It can be seen that some of these assessment methods are both formative, in helping students to develop and increase their competence and knowledge, as well as summative, in assessing knowledge and skill levels at the end of a course or program.

In conclusion

Nothing is likely to drive student learning more than the method of assessment. At the same time, assessment methods are rapidly changing and are likely to continue to change. Assessment in terms of skills development needs to be both ongoing and continuous as well as summative. There is an increasing range of digitally based tools that can enrich the quality and range of student assessment. Therefore the choice of assessment methods, and their relevance to other components, are vital elements of any effective learning environment.

Over to you

Your views, comments and criticisms are always welcome. In particular:

  • are there other methods of assessment relevant to a digital age that I should have included?
  • there is still a heavy reliance on computer-based multiple-choice tests in much teaching, mainly for cost reasons. However, although there are exceptions, in general these really don’t assess the high level conceptual skills needed in a digital age. Are there other methods that are equally as economical, particularly in terms of instructor time, that are more suitable for assessment in a digital age? For instance, do you think automated essay grading is a viable alternative?
  • would it be helpful to think about assessment right at the start of course planning, rather than at the end? Is this feasible?

Or any other comments on assessment as a critical component of a learning environment, please!

Next up

Why thinking in terms of a comprehensive learning environment is necessary but not sufficient when designing a course or program.

 

References

Earle, L. (2003) Assessment as Learning Thousand Oaks CA: Corwin Press

Mayfield, E. (2013) Six ways the edX Announcement Gets Automated Essay Grading Wrong, e-Literate, April 8

Parachuri, V. (2013) On the automated scoring of essays and the lessons learned along the way, vicparachuri.com,  July 31

 

Game-based and immersive courses in a community college system

SimSprayBradley, P. (2014) Getting in the game: Colorado colleges develop game-based, immersive courses Community College Weekly, March 3

The Colorado Community College System (CCCS) is one of the leading community college systems in exploring new online technologies. I have already reported on their use of remote labs for teaching introductory science courses at a distance. This article looks at the extensive use of immersion and game-based learning in the CCCS:

CCCS set aside $3 million through its Faculty Challenge Grant Program to encourage the development of courses and curriculum focusing on immersion and game-based learning (IGBL). Grants were awarded to 15 projects. The intent was that they would be “lighthouse projects,” illuminating the way for others to follow. Each solution would be scalable, shared with other institutions throughout the 13-college system.

Some of the 15 projects

Projects from this investment include the following:

  • CSI Aurora (Aurora CC) teaches the reality of forensic work through an immersive learning exercise involving a mock crime scene and mock criminal trial, with student participation from the archaeology, forensic anthropology, criminal justice, paralegal and science departments.
  • the Auto Collision Repair program at Morgan Community College purchased a SimSpray immersive virtual reality painting simulation unit, designed to assist in the teaching of spray painting and coating fundamentals. Using SimSpray decreases the expense of paint used to teach spray painting and prevents exposure to potentially dangerous fumes. The 3D SimSpray experience allows students to practice painting before ever stepping into the paint bay (I think in this case the real thing would be more fun!)
  • At Front Range Community College, Project Outbreak is a series of augmented reality scenarios in which microbiology students track and follow a potential epidemic in their local area to its source across international borders. Students use their mobile devices, the TagWhat geolocation app, Google Hangout and Google maps. Scenarios are designed to meet core competencies, promote global connectedness and give students a global perspective in solving real-world problems
  • the Community College of Aurora’s film school is in the process of using a $100,000 grant to create a virtual economy designed to mirror the reality of the studio system, from writing scripts to luring investors to screening the film in front of a real-life audiences. Over the past seven years, the film school has developed proprietary software that allows students to experience — virtually — every aspect of the filmmaking experience. The cost of rental housing in Los Angeles, New York and Denver can be accessed with a few clicks of a mouse. The cost of obtaining equipment can easily be calculated. Students working within a set budget can see how much to devote to paying actors and directors, producers and key grips.
  • an instructor at the the Community College of Denver is using ACCESS, a web-based game modelled after the board game “Life”, whixh simulates a person’s travels through his or her life, from college to retirement, with jobs, marriage, and possible children along the way. ACCESS teaches the course in a flipped format, allowing students to receive information through videos, podcasts, downloadable lectures and social media, and then discuss the materials in class. The course is designed to help students successfully complete remedial coursework.

Results

The article offers the following results from a ‘consultant’s report’ but I couldn’t find any corroboration:

  • where the ACCESS game was used, scores on quizzes jumped 14 percent and 71 percent of students completed the course, compared to 60 percent enrolled without the gaming component
  • students exhibited nearly identical pass/fail rates as non-IGBL courses.
  • 69 percent of students across semesters indicated that they were either more or much more satisfied with their IGBL course, as compared to other courses; 85 percent of students indicated that they were either more or much more satisfied with their IGBL instructor, as compared to other instructors.
  • students indicated that their IGBL course did a better or much better job (as compared to non-IGBL courses) of helping them achieve a variety of learning outcomes, including: having fun while learning (83 percent/73 percent); applying learning to new situations (81 percent/72 percent); staying engaged in learning (79 percent/73 percent); feeling involved in the college (69 percent/60 percent); working well with other students (67 percent/61 percent).

Over to you

Contact North has descriptions of a number of immersive learning projects under its ‘Pockets of Innovation‘ such as Loyalist College’s Border Simulation in Second Life.

See also:

Games-and-learning-in-digital-worlds-en-francais/

More news of video games

Games to defeat obesity, Napoleon, and students’ learning, and other games’ news

I’d be interested to hear from others who are using game-based immersive learning in the two year college system.