April 21, 2014

What is more important than grades?

Listen with webReader

Photo of building

McMaster University Faculty of Medicine

In an earlier posting on the OECD’s PISA tests, there was some discussion about what was NOT measured in standardized tests of reading, science and math.

The Faculty of Medicine  at McMaster University, in industrial Hamilton, 65 kilometres south west of Toronto, has always been a leader in innovation in medical teaching, many years ago pioneering problem-based learning.

In yesterday’s Globe and Mail, there was an interesting article about a new online assessment of personal characteristics (CASPer = Computer-based Assessment for Sampling Personal Characteristics) that is used for selecting students for admission to its medical school. CASPer is a computerized version of a previously face-to-face interviewing schedule called the Multiple Mini Interview developed by Professor Harold reiter at McMaster that is now used by 12 of the 17 Canadian medical schools. CASPer uses videoed scenarios to which students must quickly respond as part of a two hour test.

What jumped out at me is that now grades counted for only one third of a student’s admission score. CASPer is now weighted double the students’ grade point average.

What are these computerized tests measuring? Good decision-making, ethics, communication skills, cultural sensitivity, etc.

Now it should be noted that students still have to have the highest standard in grade point average to even get to be interviewed by CASPer, so it is already choosing from the very bright, but it is an interesting example of the importance of measuring students on a wide range of factors and not just test scores alone.

Bradshaw, J. (2010) Brains alone won’t get you into med school Globe and Mail, December 13, p. A11

Innovative e-learning in the Vancouver area

Listen with webReader

I worry about the often negative tone of many of my posts. It was therefore a great pleasure to attend the joint Justice Institute of British Columbia (JIBC) and Vancouver Community College (VCC) ‘Online showcase’ at JIBC in New Westminster, just south-east of the City of Vancouver, and see demonstrations of some great uses of e-learning for education and training.

The showcase provided an opportunity for local universities and colleges to demonstrate what they are doing regarding online learning. There were presentations from Kwantlen Polytechnic University, British Columbia Institute of Technology, University of British Columbia, JIBC, Emily Carr University of Art and Design, and VCC. I wasn’t able to see all the presentations, so my apologies to those presenters that I missed.

JIBC: Emergency management

JIBC is a unique education and training institution, Canada’s leading public safety educator. It provides training for police, paramedics, prison staff, probation officers, and so forth. It is partly funded by a grant from the BC provincial government and student tuition fees, but most of its revenues comes from training contracts with its main clients. The JIBC offers a range of applied and academic programs that span the spectrum of safety – from prevention to response and recovery. The JIBC’s main campus is located in New Westminster, but regional campuses allow students to study closer to home. It has a long history of using technology for the content and delivery of its programs.

The JIBC’s Emergency Management Division offers over 50 courses in this area, covering topics such as Incident Command, Emergency Operations Centre, Exercise Design and more. The Winter Olympics in Vancouver provided a challenge in terms of emergency response preparation, involving over 20 municipalities, several local police forces and the RCMP, fire and ambulance services, the Canadian (and US) military, and a host of other agencies, depending on the nature of the eventual emergency.

Jerome Rodriguez and Rosamaria Fong gave a demonstration of the materials created not only for the formal courses in emergency management offered by JIBC, but also made publicly available over the Internet and through mobile technology, such as iPhones and iPads. These resources enable all services involved in emergency response to have common and shared information about procedures, contacts and terminology. Indeed, you can see these materials by logging in to My Emergency Management Resources. The mobile learning component was assisted by a grant of $130,000 for the Inukshuk Fund, but a condition was that the material must be open access.

The Emergency Division has created open access resources such as downloadable forms that need to be completed in emergency situations, short 2-3 minutes videos of the various functional units in an Incident Command System, interactive walk throughs of a virtual emergency scene (clicking on ‘bubbles’ around the scene describes the functions of each of the units represented by bubbles), and some short video news reels of accidents or incidents to be used in training exercises. Some of these materials can be repurposed – for instance, the fire in the virtual walk through below could be moved to a high rise building and the ‘bubbles’ reconfigured.

A virtual walk through of an emergency scene © JIBC, 2010

The Division also offers WCDM 2010 – an “Immersive Simulation Technology” Workshop. Although delivered in a classroom, the immersive simulations make use of technologies such as mock video newscasts, Blackberry messages, and plotting first responder movements into GIS-enabled smart-phones using Google Earth. None of these reseources replaces the formal training provided by the JIBC, but these are low-cost, open access materials that are now available for use by training organizations across North America.

JIBC: Corrections

The Corrections and Social Justice Division trains professionals who work with adult and youth offenders in institutional and community settings, to manage the risk they pose to the public. It also trains individuals who work with families going through separation and divorce.

Rob Chong emphasised in his presentation the importance of context in designing programs. Part of the mandate of his division is to train 500 probation officers and 1500 prison guards scattered across the province. To do this, the division uses a mix of online and face-to-face learning.

There are three elements to the courses: self-study, with learners interacting with Blackboard, JIBC’s LMS; guided learning, with learners interacting with an instructor; and cohort learning, with learners interacting with other learners. Learners generally access their materials in the workplace, in prisons and local probation offices.

One example he gave was of personal safety awareness training for probation officers. Short video clips are used of simulated/acted situations, and in a self-study mode, learners are asked for how they would respond to the situation. These posts are collected then the learners meet with their managers in local offices to discuss the scenarios. As well as Blackboard and video clips, 360 degree interactive images are used, so the whole context can be seen (for instance, the design of the reception area in a probation office to highlight security). Also used are Webinars via Adobe Connect, for instance for training in interview skills. The aim is to ensure that the design and delivery of the teaching matches the context in which the learning will take place.

UBC: Using social media in a formal course

ETEC 522: UBC

One of the courses in UBC’s fully online Master in Educational Technology is ETEC 522, Ventures in Learning Technology, taught by David Vogt and David Porter. To enable students to understand the success of entrepreneurial or intrapreneurial ventures involving learning technologies, the course provides an online immersion in global learning technologies products, services and initiatives in public and commercial domains. ETEC 522 is delivered from a venture and market analysis perspective, with a particular focus on emerging markets and real-world ventures. Jeff Miller, the instructional designer for the course, gave a presentation on ‘Creating coherence with social media.’

Quite apart from the subject matter, there are a number of innovative elements in this course. First, even though UBC is the home of WebCT, this course does not use a learning management system, but WordPress and MediaWiki, because the students as much as the instructors are creating content. Second, student’s ‘final’ work is public. Their final assignment is a multimedia ‘pitch’ for an e-learning product, service or business. These ‘pitches’ may take the form of slide or video presentations. Some of the videos can be found on YouTube. (Jeff made the interesting comment that ‘universities should be like kindergartens: students’ work should be posted on the wall.’). The ‘open’ part of the course can be seen here.

One of the challenges Jeff mentioned is drawing the line between open and closed aspects of the teaching. Although the ‘scholarship’ is public, non-registered viewers can ‘see but not touch’. The interaction between instructors and students is private; the finished work is made public. Another challenge  is archiving students’ work in a secure way while enabling it to be used by new students in the current  version of the course. For instance, the course makes use of students’ work in earlier versions of the course. (I will be writing a review of a new book on ‘Content management in E-Learning’, which looks in detail at the question of content management in e-learning.)

It is clear that moving away from a learning management system offers lots of opportunities for student engagement and student generated content, but there are also challenges in ensuring coherence and the management of their workload. This course is truly dynamic, changing each year, and continually pushing the frontiers of e-learning.

UBC: Designing online courses in science for non-science students

All UBC Arts students must take at least six credits in science as part of the Bachelor of Arts. This results in large classes for a limited number of online science courses. Most popular are the courses in Earth and Ocean Sciences, some with over 200 students per course section. Each course will have a senior instructor, usually a tenured faculty member, supported by up to four teaching assistants (usually graduate students).

The design challenge is to create science courses for students with little or poor numeracy and quantitative skills for large online classes. Chris Crowley, Josefina Rosado and Sunah Cho from UBC’s Office of Learning Technology described how they used Flash 3D images and animations within Web CT Vista to help students understand the scientific principles that explain coastal upwelling in oceans.

The senior instructor role was identified as facilitator, stimulator, monitor, subject specialist, and evaluator.

Despite the value of using interactive graphics and simulations to improve understanding, I had many questions, both about the policy (good intention but can you really train someone in science in two one semester courses?) and the design. For instance can you teach science without an understanding of and experience in experimental design?

Emily Carr University of Art and Design: Science 202

Jane Slemon offers an interesting online version of a course also offered on campus called: Heart, Mind Health: Learning from the Human Body. This course offers comprehensive understanding of the shape and function of the organs of the human body and invites creative consideration to the metaphors relative to the body that abound in culture, language and design. She showed some of the outstanding student work inspired by their understanding of human biology, reflected in metaphors of asthma, dyslexia, autism, HIV, and other areas of human suffering.

Vancouver Community College

Karen Belfer presented on VCC’s online automotive collision repair course for unqualified apprentices in the work force. (Fewer than 50% complete full-time apprenticeship training in BC, resulting in large numbers of unqualified tradespeople in the BC workforce.)  VCC used to offer this program over seven weeks on campus, requiring 30 hours a week of class attendance. This caused many problems for both apprentices (who often lost wages and unemployment insurance and would have to travel to Vancouver) and employers, who had to manage without staff during this period. The course, which is 80% theory and 20% practice/hands-on), was redesigned for study over 16 weeks online (mainly while learners were at work) and the last two weeks full-time on campus in Vancouver. Here they are tested in their practical skills, and assessed on their knowledge.

Although VCC used its Moodle LMS for this course, it found apprentices are not prepared for large amounts of reading, so efforts were made to the use industry standard online content with a high graphics, video and audio content, and to reduce the amount of text through the use of audio, video clips, graphics and cartoons, with a good deal of online interaction with materials, such as moving online objects. This hybrid course has proved to be very successful, bot with employers and learners.

Some reflections on the showcase

1. I find such ‘show and tell’ sessions extremely valuable. They reflect what people are actually doing now, and you need to see what has been created and how the program works to fully evaluate it. Such sessions are also extremely valuable for showing faculty and instructors what is possible using learning technologies. Unfortunately, there were not many instructors present during this showcase, most being instructional designers.

2. The session also emphasised the value of having learning material publicly available. Open resources provide a good indication of the quality of the course or program. I think all institutions now offering hybrid or fully online courses should have ‘sample’ resources of each course on their course web sites, so potential students can be better informed about the courses they are having to make decisions about. Also, the open educational resources in both the Emergency Response and ETEC 522 courses are very different from the very didactic and lengthy OER’s offered by MIT, Carnegie Mellon and the Open University, or from meta-tagged learning objects, and hence, in my humble view, are very much more re-usable.

3. In almost all the cases, the course designers were ‘stretching’ the functions of an LMS, or, in one case, going outside it altogether. Flash animations and short video clips were evident in several of the cases. Video now is cheap and easy to make, and adds considerable value to courses, particularly where process or procedures need to be demonstrated or where authenticity is required for training purposes. LMSs are still useful for helping students and instructors to organize learning, but they need increasingly to accommodate more multimedia functions. The main limitation of LMSs is that they require time-consuming adaptations or additions and specialist multimedia staff if students are to freely create and organize their multimedia learning. However, going without an LMS and relying entirely on web 2.0 tools presents challenges in enabling both students and instructors to manage their work within a formal course structure.

4. These cases showed a mix of approaches to the design of courses, and emphasised in particular the importance of designing for the context of learning. The diversity of learners’ needs, and the wide range of technologies now available, challenges the idea of ‘standardized’ course design, such as the traditional ‘ADDIE’ model of course design. The most innovative of the cases (Emergency Response training and ETEC 522) both used very dynamic, almost ‘on-the-fly’ course design, taking advantage of learning opportunities, new technologies and changing contexts as they arose. Interestingly, though, these courses still used project management and instructional designers.

5. The only thing missing for me in these cases was some formal evaluation of their success, partly because they were often work in progress. It could be argued that building in evaluation from the start would slow down innovation, but if the ‘system’ is to change, it will be really important to have good data and information about the success or otherwise of such projects.

I would like to end by congratulating Tannis Morgan (JIBC) and Karen Belfer for organizing this showcase. It’s made me much more optimistic about the future of e-learning heading into a new year. I believe that BC Campus has recorded the showcase, and if so, I will let you know how to access this when it is ready.

For other excellent posts (well, theirs are excellent) on this showcase see Tannis Morgan’s:

Showcase Wrapup – Extended LMS

Showcase Wrapup-Instructional Design

and Leva Lee’s Online Course Showcase

November issue of journal ‘Distance Education’

Listen with webReader

The November 2010 edition of ‘Distance Education‘, Vol. 31, No 3. is now out, but not as an open access document. If you are fortunate to have access to a university library, or are a subscriber, you will find this edition does not have a specific theme, but several of the articles, in the words of the editor, report on

‘a design-based approach to learning and teaching enhancement….It comprises foremost, a clear definition of the learning outcomes for students, careful design and orchestration of appropriate learning and teaching experiences comprising assessment of learning achievement, and systematic evaluation of the impacts of the designed learning experiences on a range of learning and teaching outcomes.’

Bethel and Bernard’s paper: ‘Developments and trends in synthesizing diverse forms of evidence’ describes a range of models for synthesizing diverse forms of research evidence, with a focus on distance education and online learning.’ This is a useful paper for anyone placing their trust in meta-analysis of the kind recently done by the US Department of Education on comparisons between online and classroom teaching (Means et al. 2008), or for research students struggling to make sense of conflicting research studies. It did nothing though to ease my distrust of syntheses of research in the field of online and distance education; the contexts and variables are so great.

Pettinger and Doering’s paper ‘The influence of motivational design on completion rates in online self-study pharmacy-content courses’ examined why, contrary to general belief, the completion rates were high on these online courses offered by the University of Minnesota. ‘Motivational design utilizes educational scaffolding to provide clear directions and purpose to keep students engaged, while also creating assessments that efficiently clarify learning objectives.‘ In essence, active learning was built into the design of the learning materials; students in these courses did NOT participate in student-student or instructor-led discussion but still completed the courses.

Bolliger and Shepherd of the University of Wyoming’s paper: ‘Student perceptions of ePortfolio integration in online courses’ explored students’ perceptions regarding the integration of e-portfolios in two online graduate courses in an instructional technology program. The results: basically students liked e-portfolios, unless they had little previous experience of reflecting on their learning.

Beckett et al.in their paper ‘Students use of asynchronous discussions for academic discourse socialization’ analyzed online forum postings from seven graduate hybrid courses and compared native English speakers’ responses with those of non-native English speakers in the group. The results ‘reveal that participants perceived OADs (online asynchronous discussions) highly positively… findings also suggested that students experienced some frustrations and disappointments regarding professorial presence and grading‘ (Basically the profs didn’t turn up in the online discussions.)

You have probably already concluded that I was unimpressed by any of the main papers in this edition, which seem to have travelled over already well trodden ground, and on the way, they make some challengeable statements which are then contradicted by their own findings.

It was therefore a pleasure to read Jon Baggeley’s clever ‘reflection’ on Luddites and educational technology. It is not what you expect it to be, and well worth reading, and because it takes a contrarian view, I will not spoil the fun by trying to synthesise his arguments (especially after reading the Bethel and Bernard paper.)

The edition ends with an excellent review by Terry Evans (Deakin University) of John Daniel’s latest book: ‘Mega-schools, technology and teachers: achieving education for all.’

Generally I have a high respect for this journal, but this edition is disappointing. Don’t rush out and buy it.

Distance Education and Mobile Learning

Listen with webReader

The Australian-based Distance Education journal has just brought out a special edition on distance education and mobile learning.

The articles between them provide a good deal of insight into the main issues around mobile learning.

John Traxler, of the University of Wolverhampton, UK, is the guest editor, and I strongly recommend his editorial article. In it he discusses some of the issues around the definition of mobile learning, the similarities and differences between mobile learning and distance education, and, particularly useful, a summary of the main ‘affordances’ of mobile learning, under two general headings:

  • enhance, extend and enrich the concept and activity of learning, beyond earlier conceptions of learning
  • take learning to individuals, communities and countries that were previously too remote or distant

He draws attention also to the growing lack of connection between practitioners and educational researchers, opening up mobile learning to the criticism that it is under-theorised and project- and technology-based, rather than a consistent, focused approach to meeting specific educational challenges.

This theme is also taken up in another excellent article by Tiffany Koszalka and G.S. Ntloedibe-Kuswani, of Syracuse University, USA. This is a review of the literature and 10 case-studies of mobile learning. The authors build on Stead’s work around the safe and disruptive learning potentials of mobile technologies. The article provides useful statistics on access to portable phones. They conclude that most of the studies they reviewed were poorly designed as research studies. As a result, although all the studies suggest that m-learning may be supportive of the teaching and learning process, it is questionable whether much has been learned about the use of m-learning as a way to enhance learning. It is unclear whether m-technologies or changes in pedagogy are the root of outcomes. (This conclusion mirrors many preceding studies of other educational technologies.)  Nevertheless the article does indicate that there are clear motivational and access benefits from m-learning.

The other four articles are all reports on different m-learning projects.

Elizabeth Beckmann of the Australian National University reports on an m-learning project for a post-graduate program aimed at development workers, who by their nature are scattered in remote parts of the world. Some of the conclusions are generalisable, such as the critical importance of high quality, reliable Internet access, the importance of building rich social practices into the design of teaching and learning, the value of developing a community of learners, in this case development workers in different countries, and lastly, that many lessons learned from the use of past educational technologies, such as the need to focus on the pedagogy and design of learning rather than the technology, need to be adopted.

Taylor et al (all authors from the universities in the north of England) report on a project aimed at health and social care workers in England.

Balasubramanian et al. from the Commonwealth of Learning report on the use of mobile phones to promote lifelong learning among rural women in Southern India.

Vyas et al. (the authors are from Tufts University, USA and the Christian Medical College, Vellore, India) report on clinical training at remote sites in India.

I enjoyed reading and learned from all the articles in this special edition. I highly recommend the edition, even though it is not an open access journal.

SAIDE newletter on open and distance education in Africa

Listen with webReader

The South African Institute for Distance Education (SAIDE) has published its latest newsletter. Two articles caught my eye.

Mhlanga, E. (2010) SOFIE (Strengthening Open and Flexible Learning for Increased Education) Project Findings SAIDE Newsletter, Vol. 16, No. 2

The aim of this study was to increase access to education and learning for young people affected by HIV and AIDS in Malawi and Lesotho by developing, trialing and evaluating an expanded model of schooling which used open, distance and flexible learning (ODFL) to complement conventional schooling. The key design is the development of ‘circles of support’ that provide learning materials and teacher support for students who miss classes because of the need to care for family with HIV/AIDS. Although these are only preliminary results, the findings indicate that the intervention was successful, leading to less drop-out from school, and improved academic performance. (Note: this is not e-learning but print based support).

Preston, D. and Moore, A. (2010) The Use of Open Education Resources at the University of Malawi SAIDE Newsletter, Vol. 16, No. 2

During 2009 the University of Malawi (UNIMA) embarked on two OER projects, one at the Kamuzu College of Nursing and the second at the Bunda College of Agriculture. These projects were co-funded by the Open Society Initiative for Southern Africa (OSISA) and the William and Flora Hewlett Foundation, and facilitated jointly by SAIDE’s OER Africa Initiative and the International Association for Digital Publications (IADP).

At Kamuzu College of Nursing an interface was designed to hold the digital resources together in a manner that promoted the use of Problem-based Learning. This courseware is licensed under a creative commons licence and is freely-available on the OER Africa website.

At the Bunda College of Agriculture, the Language and Communication for Development Department decided to develop a textbook to address problems of staff and students not having access to the same set of textbooks despite their Communications Skills classes being a core course for all first year students. Secondly, students could not always access the recommended readings as the College Library did not have, or had an insufficient number of the texts. The 102 page Communication Skills textbook, which was created exclusively from OER, was released in early-2010 and has been offered back to the OER community. It is available from both the Bunda College website and the OER Africa website.

Again it is early days and immediate results for both the OER projects have been somewhat mixed but with promise for the future.

Also in this issue, the launch of the  African Health OER Network was announced. The network provides a platform for African health academics to freely access and share educational materials as well as to debate key issues around the future of health education for healthcare workers in Africa. The aim of the network is to share and circulate health-related educational materials by building links to existing resources (e.g., programmes, modules, courses), which authors have shared under a Creative Commons licence.