January 20, 2018

Learning analytics and learning design at the UK Open University

Maxim Jean-Louis (President, Contact North) and myself outside Walton Hall, the headquarters of the UK Open University, in 2012. It was my first visit since I left the OU in 1989.

Maxim Jean-Louis (President, Contact North) and myself outside Walton Hall, the headquarters of the UK Open University, in 2012. It was my first visit since I left the OU in 1989.

The Open University (2016) Developing learning design and analytics for student success, Connections, Vol. 21, No. 3

The latest edition of the Commonwealth of Learning’s magazine, Connections, has an interesting if brief article of the effective use of learning analytics. There are four key points that I noted:

  • the OU has scaled up its predictive use of learning analytics to cover over 45,000 students, and it works as well in traditional universities as in the OU
  • learning analytics is used in connection with learning design to identify not only students at risk but also to improve the design of the learning materials:

    the OU for the first time can empirically analyse the design of its modules. By linking learning designs with student satisfaction and success measures, it became possible to systematically identify, measure and improve critical aspects of students’ learning experience.

  • the OU is the first university in the world to develop and adopt a policy relating to the ethical uses of student data for
    learning analytics, involving students themselves in the development of the policy. This makes the adoption and use of learning analytics much easier
  • the OU has appointed a reader in learning analytics, Dr. Bart Rienties: that is treating learning analytics really seriously.

Unfortunately there were no links or ways to follow up the article.

Towards an open pedagogy for online learning

Image: © University of Victoria, BC

Image: © University of Victoria, BC

The problems with OER

I was interviewed recently by a reporter doing an article on OER (open educational resources) and I found myself being much more negative than I expected, since I very much support the principle of open-ness in education. In particular, I pointed out that OER, while slowly growing in acceptance, are still used for a tiny minority of teaching in North American universities and colleges. For instance, open textbooks are a no brainer, given the enormous savings they can bring to students, but even in the very few state or provincial jurisdictions that have an open textbook program, the take-up is still very slow.

I have written elsewhere in more detail about why this is so, but here is a summary of the reasons:

  • lack of suitable OER: finding the right OER for the right context. This is a problem that is slowly disappearing, as more OER become available, but it is still difficult to find exactly the right kind of OER to fit a particular teaching context in too many instances. It is though a limitation that I believe will not last for much longer (for the reasons for this, read on).
  • the poor quality of what does exist. This is not so much the quality of content, but the quality of production. Most OER are created by an individual instructor working alone, or at best with an instructional designer. This is the cottage industry approach to design. I have been on funding review committees where institutions throughout a province are bidding for funds for course development or OER production. In one case I reviewed requests from about eight different institutions for funds to produce OER for statistics. Each institution (or rather faculty member) made its proposal in isolation of the others. I strongly recommended that the eight faculty members got together and designed a set of OER together that would benefit from a larger input of expertise and resources. That way all eight institutions were likely to use the combined OER, and the OER would likely be of a much higher quality as a result.
  • the benefits are less for instructors than students. Faculty for instance set the textbook requirement. They don’t have to pay for the book themselves in most cases. With the textbook often comes a whole package of support materials from the publisher, such as tests, supplementary materials, and model answers (which is why the textbook is so expensive). This makes life easier for instructors but it is the students who have to pay the cost.
  • OER take away the ‘ownership’ of knowledge from the instructor. Instructors do not see themselves as merely distributors of information, a conveyor belt along which ‘knowledge’ passes, but as constructors of knowledge. They see their lecture as unique and individual, something the student cannot get from someone else. And often it is unique, with an instructor’s personal spin on a topic. OER’s take away from instructors that which they see as being most important about their teaching: their unique perspective on a topic.
  • and now we come to what I think is the main problem with OER: OER do not make much sense out of context. Too often the approach is to create an OER then hope that others will find applications for it. But this assumes that knowledge is like a set of bricks. All you have to do is to collect bricks of knowledge together, add a little  mortar, and lo, you have a course. The instructor chooses the bricks and the students apply the mortar. Or you have a course but you need to fill some holes in it with OER. I suggest these are false metaphors for teaching, or at least for how people learn. You need a context, a pedagogy, where it makes sense to use open resources.

Towards an open pedagogy

I am making three separate but inter-linked arguments here:

  • OER are too narrowly defined and conceptualized
  • we need to design teaching in such a way that it is not just sensible to use OER but unavoidable
  • we should start by defining what we are trying to achieve, then identify how OER will enable this.

So I will start with the last argument first.

Developing the knowledge and skills needed in the 21st century

Again I have written extensively about this (see Chapter 1 of Teaching in a Digital Age), but in essence we need to focus specifically on developing core ‘soft’ or ‘intellectual’ skills in our students, and especially the core skills of independent learning and knowledge management. Put in terms of learning outcomes, in a world where the content component of knowledge is constantly developing and growing, students need to learn independently so they can continue to learn after graduation, and students also need to know how to find, analyse, evaluate, and apply knowledge.

If we want students to develop these and other ‘soft’ skills such as problem-solving, critical thinking, evidence-based argumentation, what teaching methods or pedagogy should we adopt and how would it differ from what we do now?

The need for teaching methods that are open rather than closed

The first thing we should recognise is that in a lecture based methodology, it is the instructor doing the knowledge management, not the student. The instructor (or his or her colleagues) decide the curriculum, the required reading, what should be covered in each lecture, how it should be structured, and what should be assessed. There is little independence for the learner – either do what you are instructed to do, or fail. That is a closed approach to teaching.

I am suggesting that we need to flip this model on its head. It should ultimately be the students learning and deciding what content is important, how it should be structured, how it can be applied. The role of the instructor then would not be to choose, organise and deliver content, but to structure the teaching to enable students to do this effectively themselves.

This also should not be a sudden process, where students suddenly switch from a lecture-based format as an undergraduate to a more open structure as a post-graduate, but a process that is slowly and increasingly developed throughout the undergraduate program or a two-year college program where soft skills are considered important. One way – although there are many others – of doing this is through project- or problem-based learning, where students start with real challenges then develop the knowledge and skills needed to address such challenges.

This does not mean we no longer need subject specialists or content experts. Indeed, a deep understanding of a subject domain is essential if students are to be steered and guided and properly assessed. However, the role of the subject specialist is fundamentally changed. He or she is now required to set their specialist knowledge in a context that enables student discovery and exploration, and student responsibility for learning. The specialist’s role now is to support learning, by providing appropriate learning contexts, guidance to students, criteria for assessing the quality of information, and quality standards for problem-solving, knowledge management and critical thinking, etc.

A new definition of open resources

Here I will be arguing for a radical change: the dropping of the term ‘educational’ from OER.

If students are to develop the skills identified earlier, they will need access to resources: research papers, reports from commissions, case-study material, books, first-hand reports, YouTube video, a wide range of opinions or arguments about particular topics, as well as the increasing amount of specifically named open educational resources, such as recorded lectures from MIT and other leading research universities.

Indeed, increasingly all knowledge is becoming open and easily accessible online. All publicly funded research in many countries must now be made available through open access journals, increasingly government and even some commercial data (think government commission reports, environmental assessments, public statistics, meteorological models) are now openly accessible online, and this will become more and more the norm. In other words, all content is becoming more free and more accessible, especially online.

With that comes of course more unreliable information, more false truths, and more deliberate propaganda. What better preparation for our students’ future is there than equipping them with the knowledge and skills to sift through this mass of contradictory information?  What better than to make them really good at identifying the true from the false, to evaluate the strength of an argument, to assess the evidence used to support an argument, whatever the subject domain? To do this though means exposing them to a wide range of openly accessible content, and providing the guidance and criteria, and the necessary prior knowledge, that they will need to make these decisions.

But we cannot do this if we restrict our students to already ‘approved’ OER. All content eventually becomes an educational resource, a means to help students to differentiate, evaluate and decide. By naming content as ‘educational’ we are already validating its ‘truth’ – we are in fact closing the mind to challenge. What we want is access to open resources – full stop. Let’s get rid of the term OER and instead fight for an open pedagogy.

Report on SFU’s experiences of teaching with technology

Simon Fraser University (on a rare day when it wasn't raining)

Simon Fraser University’s Burnaby campus (on a rare day when it wasn’t raining)

I always enjoy going to a university or college and seeing how they are using learning technologies. I am always a little surprised and I am also usually intrigued by some unexpected application, and today’s DemoFest at Simon Fraser University was no exception.

About Simon Fraser University

SFU has just over 35,000 students with campuses in Burnaby, Vancouver downtown, and Surrey, all in the lower mainland of British Columbia, Canada.

For a long time it has had the largest distance education program in British Columbia, but the rapid development of fully online and blended learning in other BC and Canadian institutions means that other institutions are rapidly gaining ground. It is also the academic base for Linda Harasim, who is a Professor of Communications at SFU.

As with many Canadian universities, most of the DE programs are run out of the Centre for Online and Distance Learning in Continuing Studies at SFU. However, the university also has a large Teaching and Learning Centre, which provides a range of services including learning technology support to the faculty on campus.

The university recently adopted Canvas as its main LMS.

I was spending most of the day at SFU for two reasons:

  • to identify possible cases for Contact North’s ‘pockets of innovation’ project
  • to report on the survey of online learning in Canadian post-secondary institutions.

I will be giving more information on both these projects in separate blog posts coming shortly.

The DemoFest

DEMOfest 2016 is about how instructors are using ….technologies in ways that produce exciting and original educational experiences leading to student engagement and strong learning outcomes.

Making lectures interactive

Not surprisingly, several of the short, 10 minute presentations were focused on tools used in classroom teaching or lecturing. In particular, the tools are going mobile, in the form of apps that students can use on their mobile phones, tablets or laptops. I was particularly impressed with TopHat, which incorporates online quizzes and tests, attendance checks, and  discussion. REEF Polling is a similar development developed by iClicker, which is effectively a mobile app version of iClicker. Both provide students and instructors with an online record of their classroom activity on the app.

There was also a couple of sessions on lecture theatre technologies. As in other universities, lecturers can find a range of different interfaces for managing lecture theatre facilities. SFU has a project that will result in a common, simple interface that will be available throughout the different campuses of the universities, much to the relief of faculty and visiting speakers who at the moment have no idea what to expect when entering an unfamiliar lecture theatre or classroom.. There was also another session on the limits of lecture capture and how to use video to make learning more engaging.

Online learning development

However, I found nothing here (or anywhere else, for that matter) that has convinced me that there is a future in the large lecture class. Most of the technology enhancements, although improvements on the straight ‘talk’ lecture, are still just lipstick on a pig.

The online learning developments were much more interesting:

  • online proctoring: Proctorio. This was a demonstration of the ingenuity of students in cheating in online assessment and even greater ingenuity in preventing them from doing it. Proctorio is a powerful web-based automated proctoring system that basically takes control of whatever device the student is using to do an online assessment and records their entire online activity during the exam. Instructors/exam supervisors though have options as to exactly what features they can control, such as locked screens, blocking use of other urls, etc.. Students just sign in and take the exam at any time set by the instructor. Proctorio provides the instructor with a complete record of students’ online activity during the exam, including a rating of the ‘suspiciousness’ of the student’s online exam activity.
  • peer evaluation and team-based learning: SFU has a graduate diploma in business where students are required to work in teams, specifically to build team approaches to problem-solving and business solutions. Although the instructor assesses both the individual and group assignments, students evaluate each other on their contribution to the team activities. The demonstration also showed how peer assessment was handled within the Canvas LMS. It was a good example of best practices in peer-to-peer assessment.
  • Dialectical Map: an argument visualization tool developed at SFU. Joan Sharp, Professor of Biological Sciences, and her research colleague, Hui Niu, have developed a simple, interactive, web-based tool that facilitates the development of argumentation for science students. Somewhat to my surprise, research evidence shows that science students are often poor at argumentation, even in the upper years of an undergraduate program. This tool enables a question to be posed by an instructor at the tope of the map, such as ‘Should the BC government allow fracking for oil?’ or ‘Should the BC government stop the culling of wolves to protect caribou?’ The online map is split into two parts, ‘pro’ and ‘con’, with boxes for the rationale, and linked boxes for the evidence to support each rationale offered. Students type in their answers to the boxes (both pro and con) and have a box at the bottom to write their conclusion(s) from the argument. Students can rate the strength of each rationale. All the boxes in a map can be printed out, giving a detailed record of the arguments for and against, the evidence in support of the arguments and the student’s conclusion.  Hui Niu has done extensive research on the effectiveness of the tool, and has found that the use of the tool has substantially increased students’ performance on argument-based assignments/assessment.

General comments

I was very grateful for the invitation and enjoyed nearly all the presentations. The Teaching and Learning Centre is encouraging research into learning technologies, particularly developing a support infrastructure for OERs and looking at ways to use big data for the analysis and support of learning. This practical, applied research is being led by Lynda Williams, the Manager of the Learn tech team, and is being done in collaboration with both faculty and graduate students from different departments.

Students and a professor of computer science worked with the IT division and Ancillary Services to develop a student app for the university called SFU Snap, as part of a computer science course. This not only provides details of the bus services to and from SFU at any time, but also provides students with an interactive map so they can find their classrooms. Anyone who has tried to find their way around SFU (built at multi-levels into a mountain) will understand how valuable such an app must be, not just to students but also to visitors.

So thank you, everyone at the Teaching and Learning Centre at SFU for a very interesting and useful day.

 

What I learned at Drexel University in National Distance Learning Week

A street protester in Philadelphia on election day

A street artist in Philadelphia on election day

Fear and loathing in Philadelphia

On Tuesday and Wednesday last week, I found myself in Philadelphia on U.S. Presidential Election day, and even more importantly, the day after, as the results became known. I was there, not to ‘rig’ the election, as some have rumoured, but to visit one of the leaders in online learning in the USA, Drexel University.

I’m not going to say much more about the election, except to note that as in the rest of the country, Pennsylvania was deeply split, with cities such as Philadelphia and Pittsburg voting strongly for Clinton, and suburban areas, smaller towns and rural areas voting in sufficiently large enough numbers for Trump to just about win the state and its electoral votes. So the election results have caused a certain amount of fear and loathing in Philadelphia, particularly among the university community.

Why Drexel?

Drexel University is a private, nonprofit university ranked among the top 100 universities in the USA. In 2016 it was ranked the 8th most innovative university in the USA by US News and World Report. It has about 26,000 students.

Drexel University was founded in 1891 as the Drexel Institute of Art, Science and Industry, by Philadelphia financier and philanthropist Anthony J. Drexel. The original mission of the institution was to provide educational opportunities in the “practical arts and sciences” for women and men of all backgrounds. It is famed for its co-op education program and its close links to local industry and businesses, and in the past for its acceptance and encouragement of low income students. However in recent years its focus has changed, partly driven by the perceived need to increase its ranking. Today it has very high student tuition fees and a highly selective admission process.

I was there to visit Drexel University Online (DUO), an internal division within the university that serves those students at Drexel taking online courses and programs.

Drexel Online

Drexel University has more than 7,000 online students from all 50 states and more than 20 countries. It offers 140 fully accredited master’s degrees, bachelor’s degrees and certificate programs in a wide range of disciplines. Nursing in particular has a very strong set of online programs. Drexel was an early pioneer of online learning, offering its first fully online master’s degree in 1996.

Drexel University founded National Distance Learning Week, in conjunction with the United States Distance Learning Association, in 2007, and has won several national awards for institution-wide excellence in online education.

As part of Drexel’s contribution to National Distance Learning Week, I was invited as a guest speaker, to talk about ’21st century knowledge and online learning: re-designing teaching for a digital age.’ While at Drexel, I also took the opportunity to see what Drexel is doing with advanced learning technologies.

Advanced use of technologies at Drexel Online

DUO offers faculty a technology lending library, where faculty can try out new devices and evaluate their potential for teaching. This includes an augmented reality headset that combines a cheap ($10-$15), easily assembled cardboard frame into which a mobile phone can be inserted in front of the eyes, enabling augmented reality programs to be delivered at very low cost to the student (provided they already have a mobile phone).

DUO has also developed a very interesting web site, called VirtuallyInspired.org, which showcases a number of innovations in online learning from institutions across North America and around the world.

Here I will describe briefly just a few of Drexel’s own innovative projects, which I hope will inspire you to look in more detail at the VirtuallyInspired web site.

Tina the Avatar

Tina the Avatar

Tina the Avatar

Tina is an avatar of a 28 year old woman in a virtual world who not only responds to questions asked by students but can also be physically examined and will respond according to how she is being treated. The teaching around Tina is broken down into 10 modules, each of which correlate with a body system that students learn about in class. The program serves not only as reinforcement for the principles taught in the course, but also to develop interpersonal skills needed by clinical professionals. Professors are able to view the type of questions asked by the student and how the student reacts to Tina’s responses. They are then able to give the student advice and make recommendations for interpersonal skill improvement.

Synchronous online teaching

Drexel is experimenting with the use of low-cost (US$450) robots (Kubi) combined with iPads to improve the ‘telepresence’ of students in online webinars. In the classroom where the instructor is located, there is an iPad for each remote student locked into a robot that each student can remotely move around the instructor’s classroom. Using Skype and the camera on the student’s computer, the student’s face appears on the iPad. In this way the instructor can see the faces and hear each individual student via the iPad, and the students at home can also see on their screen not only the instructor but also the iPad images of all the other students in the class. This system is already in use at the Michigan State University.

Using Kubi for telepresence at Michigan State University

Using Kubi for telepresence at Michigan State University

Forensic investigation

Students taking a course on forensic investigation can use a branching video sequence to search for clues at a crime scene. Students can do a virtual walk around and inside a house and are asked to observe and interpret what they see, followed by a debriefing afterwards.

These are just a few of the several innovations that Drexel is experimenting with. Others include the use of video simulations in law and nursing, dealing with critical incidents in practice.

Innovation and operations

Drexel University is to be congratulated for two reasons: it has an extensive, ongoing online program that delivers a wide range of courses on a daily basis to over 7,000 students. For most of these courses, the challenges are common to all online post-secondary programs: ensuring that the programs are of high quality and that students succeed. This means applying well known best practices and procedures, using standard tools such as a learning management system, and ensuring that students are well supported by instructors.

At the same time, DUO is investing some of its energy and resources to investigating new ways of designing and delivering online teaching. This means finding like-minded faculty partners who can see the potential of new technologies and who are willing to put in the time and effort to do something different. The challenge here is to evaluate each innovation, to integrate such innovations into regular teaching, and then to ensure the diffusion of successful innovations into a wider range of courses and programs.

Getting the right balance between on-going operations and innovation is a challenge but one that Drexel Online seems more than able to handle.

And lastly, I cannot express enough my appreciation for the kindness and attention paid to me by Susan Aldridge, the Director of DUO, and all her staff during my visit. Elections may come and go, but American hospitality continues for ever.

Scary tales of online learning and educational technology

The Centre for Digital Media, Vancouver BC

The Centre for Digital Media, Vancouver BC

The Educational Technology Users Group (ETUG) of British Columbia held an appropriately Halloween-themed get together today called ‘The Little Workshop of Horrors’ at which participants were encouraged to share tales of failure and horror stories in the use of learning technologies.

This seemed to me a somewhat risky strategy but it actually worked really well. First the workshop was held in ‘the Hangar’, a large, covered space in (or rather beside) the Centre for Digital Media, a shared building used by UBC, Simon Fraser University, BCIT and the Emily Carr University of Art and Design. The Centre itself is a good example of collaboration and sharing in developing media-based programs, such as its Master of Digital Media. The Hangar lent itself to a somewhat spooky atmosphere, enhanced by a DJ who often accompanied presenters with ghoulish music.

Audrey’s Monsters

The workshop got off to an excellent start with a brilliant keynote from Audrey Watters on the Monsters of Educational Technology (The link will take you to her book on the subject). She identified a range of monsters (the examples are partly Audrey’s, partly mine):

  • Frankenstein’s monster that went wrong because its (hir?) master failed to provide it (em?) with love or social company (teaching machines?): in Audrey’s word’s ‘a misbegotten creature of a misbegotten science’,
  • vampires that suck the blood of students, e.g. by using their personal data (learning analytics?),
  • zombies, i.e. technologies or ed tech ideas that rise and die then rise again (e.g. technology will remove the need for schools, an idea that goes back to the early 1900s),
  • giants that become obsolete and die (Skinner, Merrill)
  • the Blob, which grows bigger and bigger and invades every nook and cranny (MOOCs?)
  • and the dragons, are the libertarian, free-market, Silicon-valley types that preach the ‘destruction’ and ‘re-invention’ of education.

Audrey Watters’ larger point is that if we are not careful, educational technology easily turns itself into a monster that drives out all humanity in the teaching and learning process. We need to be on constant watch, and, whenever we can, we need to take control away from large technology corporations whose ultimate purpose is not educational.

Not only was it a great, on topic, presentation, but it was also such a pleasure to meet at last Audrey in person, as I am a huge fan of her blog.

He was a monster, not because he was a machine, but because he wasn't loved

Confessions

Then came the confessional, at which a series of speakers confessed their sins – or rather, classic failures – about educational technology, often in very funny ways. What was interesting though about most of the tales was that although there was a disaster, in most cases out of the disaster came a lot of good things. (As one speaker said, ‘Success is failing many times without losing your optimism’; or ‘ A sailor gets to know the sea only after he has waded ashore.’).

One presenter reported going to a university to ‘sell’ Blackboard but was so nervous that her presentation was so bad they ended up going with Canvas (you see what I mean about some good coming out of these disasters!) Another described how over 20 years she has been trying to move faculty into more interactive and engaging technology than learning management systems, yet here she is still spending most of her time supporting faculty using an LMS.

One talked about spending years trying to promote IMS-based learning objects, only to find that Google’s search engine made meta-data identification redundant. Revealingly, he felt he knew at the time that the meta-data approach to learning objects was too complex to work, but he had to do it because that was the only way he could get funding. More than one speaker noted that Canada in the past has spent millions of dollars on programs that focused heavily on software solutions (anyone remember EduSource?) but almost nothing on evaluating the educational applications of technology or on research on new or even old pedagogies.

Another spoke about the demise of a new university, the Technical University of British Columbia, that was a purpose-built, new university deliberately built around an “integrated learning” approach, combining heavy use of on-line learning with mixed face-to-face course structures – in 1999. However, by 2002 it had only about 800 FTEs, and a new incoming provincial government, desperate to save money and eager to diminish the previous government’s legacy, closed the university and transferred the students (but not the programs) to Simon Fraser University. Nevertheless, the legacy did live on, with many of the learning technology staff moving later into senior positions within the Canadian higher education system.

I see instructional designers, educational technologists or learning ecology consultants (which was a new title for me) as the Marine Corps of the educational world. They have seen many battles and have (mostly) survived. They have even learned how to occasionally win battles. That’s the kind of wisdom of which academic leaders and faculty and instructors should make much better use.

One participant had such a bad experience at Simon Fraser University that she thinks of it as 'the haunted house on the hill.'

One participant had such a bad ed tech experience at Simon Fraser University that she thinks of it as ‘the haunted house on the hill.’

Happy Halloween, everyone!