January 20, 2018

The importance of redesign in online learning: the NCAT experience

© yourdecoratinghotline.com, 2012

Twigg, C. (2012) Improving Learning and Reducing Costs: Initial Project Outcomes from Changing the Equation The Learning MarketSpace, July

The results of a major program aimed at re-designing math curricula in two year colleges are announced in the National Centre for Academic Transformation‘s July Newsletter .

The program

Changing the Equation is a major program to engage the community colleges in the USA in a successful redesign of their remedial/developmental math sequences. There were 38 participating two-year colleges.

Each participant in Changing the Equation redesigned its entire developmental math sequence–all sections of all developmental courses offered–using NCAT’s Emporium Model and commercially available instructional software. Each redesign modularized the curriculum, allowing students to progress through the developmental course sequence at a faster pace if possible or at a slower pace if necessary, spending the amount of time needed to master the course content. Institutions piloted their redesign plans in spring 2011 and fully implemented their plans in fall 2011.


Preliminary results are in for 20 of the 38 colleges. The main results are as follows:

Improved learning outcomes

Twenty institutions redesigned a total of 54 developmental math courses. Student learning outcomes in the traditional and redesigned formats of the courses were measured by comparing performance on common final examinations, common exam items and/or gains on pre- and post-tests with the following results:

  • 46 courses (85%): showed significant improvements
  • 6 courses (13%): showed no significant difference
  • 2 courses (2%): insufficient data was collected to make a comparison

Course Completion

NCAT asked each institution to compare course-by-course completion rates (grades of C or better or grades of P in a P/F system) between the traditional and redesigned formats. Twenty institutions redesigned a total of 54 developmental math courses with the following course-by-course completion results:

  • 7 courses (13%): had higher completion rates, 3 of which were significantly higher
  • 5 courses (9%): showed no significant difference in completion rates
  • 28 courses (52%): had lower completion rates, 16 of which were significantly lower
  • 12 courses (22%): completion could not be calculated due to the collapse of multiple courses into one
  • 2 courses (4%): insufficient data were collected to make a comparison

This suggests that in terms of completion, the redesign was unsuccessful. However, the reasons put forward by NCAT to explain this are quite interesting:

  • prior grade inflation: pass rates in the traditional format were inflated by prior inconsistencies in grading practices.
  • mastery learning requirement in the redesign: students were required to master all of the content of all of the courses.
  • the redesigned courses were more difficult: more content and more activities such as tests, quizzes which meant students had to work longer/harder; it should be remembered too that what matters is the final completion rate, how many succeed at graduation. NCAT argues that students who succeed in the redesigned courses do better at graduating, thus boosting the final graduation rates. (Evidence to support this argument is not yet available).

Cost reductions

With regard to cost savings, the CTE results were almost universally successful. All but one of the 20 CTE projects reduced their costs.

The actual reduction in the cost-per-student for the 20 institutions was about 20%.

  • 4 institutions (20%) reduced the cost-per-student between 30% and 40%.
  • 10 institutions (50%) reduced the cost-per-student between 15% and 30%.
  • 5 institutions (25%) reduced the cost-per-student 15% or less.
  • 1 institution (5%) did not produce cost savings.

There were two primary ways that cost reduction was achieved: 1) increasing section size or 2) increasing the number of sections that full-time and adjunct faculty counted toward their load (e.g., teaching two redesigned sections rather than one section for one workload credit.) Both of these strategies were implemented without increasing faculty workload because of the elimination of repetitive tasks such as hand-grading homework, quizzes and exams.


Hybrid learning is not new. Since 1999, Carol Twigg and the National Center for Academic Transformation have been plugging away at the redesign of core or foundational courses in higher education institutions in the USA. (Go the the NCAT site for more results and reports of other projects).

This program has had some remarkable successes, without any of the mass media hype and hysteria around MOOCs. It appears from these results  – and other NCAT programs in universities – that the NCAT redesign program results in higher quality learning at less cost (although the average cost per graduating student may be affected by the lower completion rates – it is impossible to tell from the report).

Course redesign is not easy. It requires hard work and discipline. Institutions sometimes find the requirements for course redesign too onerous, or simply ignore some of the conditions that NCAT demands. Also, NCAT’s results are not independently published or verified, and some institutions have complained in the past that they had to absorb many of the costs. There are also other possible redesign models than the NCAT model that would better fit other learning philosophies and subject requirements.

Nevertheless, what the NCAT projects demonstrate in spades is the importance of redesign. It can lead to both better student performance and reduced costs. Yes, the use of computers and online learning is a critical component, but it is the redesign and the more effective use of faculty time that makes the difference. Just adding technology to the classroom or replicating the classroom through lecture capture will not produce the same results.

Examples from Ontario colleges of faculty development in online learning

This is the third in a series of guest blogs on innovative developments in online learning in Ontario post-secondary institutions. (The first was examples of hybrid learning and the second was examples of virtual worlds. simulations and mobile apps.)

In this post, Judith Tobin of Contact North| Contact Nord focuses on examples of faculty and professional development. Here is her guest post:


In my third guest blog based on the Pockets of Innovation Series from Contact North, Ontario’s distance learning and training network, I am focusing on the professional development and training opportunities offered by some Ontario colleges to prepare their faculty to take advantage of online and hybrid learning. 

 Georgian College

Georgian College in Barrie provides two programs with the students and their learning at the core of the instructional design. Following some mandatory online training prior to attending the Designing for Online Learning Series, faculty then meet and discuss learning theories, online course design principles, and online learning tools, supplemented with some hands-on experience.  The subsequent program, Online Course Development Workshop, offers online modules and personal support as the faculty work through the tools and structures for developing one of their courses for online delivery. Faculty were initially disappointed that the first program did not teach them all they needed to know about technological tools but have come to appreciate the emphasis on learning and theory. The course development workshop incorporates discussion groups so that faculty can form communities to support each other during and after the course.

Niagara College

At Niagara College in Welland and Niagara-on-the-Lake, Hybrid Course Development is offered as a four credit Continuing Education course. The course is taught in hybrid mode so that the faculty have the same experience as their students in adapting to and benefitting from hybrid learning. The faculty learn about the technological tools by using them and applying them to their course development. The goal is to have the faculty appreciate and be able to exploit the variety and potential effectiveness of technologically-based alternatives to lectures.

Mohawk College 

Mohawk College of Applied Arts and Technology in Hamilton is working to position itself as a blended learning institution, using eLearn@Mohawk. Each course is required to post information, course, and assignment information, resources, discussion boards, and the grade book – all of which is facilitated by tools created by the Centre for Teaching and Learning. To support sustainability, paperless assignments and online communication and collaborative learning are to be integrated. Blended learning with core and supplemental materials online and a reduction in face-to-face class time is the third step. An intensive blended learning course has been developed so that faculty can become familiar with the pedagogy and the technological tools of blended learning. To engage the faculty, the course begins by looking at what they like least about teaching the course they are re-designing and then suggesting how blended learning can address that as well as provide effective teaching and learning.

Faculty Cyber Connections

Six colleges in eastern Ontario – Algonquin, Durham, Fleming, La Cité collégiale, Loyalist, and St. Lawrence – have established Faculty Cyber Connections so that faculty can collaborate online with their colleagues. To support faculty development, nine online modules are offered. Sir Stanford Fleming College in Peterborough had developed the module on Classroom Management, which features videos, discussion boards, readings, and other activities.  The course instructor also models classroom management principles in her facilitation of the discussion board, providing useful models for the faculty. 

Humber Institute of Teaching and Advanced Learning

The Centre for Teaching and Learning (CTL) at the Humber Institute of Teaching and Advanced Learning has set up an eLearning Roadshow during which displays and demonstrations are set up in the high traffic areas of the different Schools or departments at Humber.  Bringing the technology to the faculty, CTL highlights wikis, podcasts, white boards, e-portfolios, and all manner of other tools available for online teaching and learning.  Stressing the student success factor and student demand for flexibility, the Roadshow attracts faculty who are unfamiliar with technology and online pedagogy and starts the conversation about new ways of teaching.

Strengths and challenges

Each of these initiatives builds from the pedagogy, new ways of thinking about and facilitating teaching and learning, to the technology and its capabilities.  This is both the strength and the greatest challenge of faculty training and development.  Starting from the pedagogy places learning effectiveness and outcomes at the centre of the thinking and keeps the student as the focus. However, many of the faculty would prefer to focus on the technologies. The challenge is to blend these two approaches, providing an understanding of how online pedagogy and tools need to work together for enhanced learning.  

Keeping faculty committed to course development provides other challenges as well.  Despite release time, it is often difficult to keep faculty motivated throughout the often lengthy and complex process of re-thinking and re-structuring a course they have taught for years.  Concerns about work load are common as faculty are not prepared to deal with the new demands of online teaching and communication, especially with such tasks as monitoring discussion boards. Institutions, in some cases, are working to develop better understanding of the demands for faculty training, support, release time, and work load implications for online or hybrid learning. Change is never an easy process. Faculty training, development, and ongoing support are essential components to the integration of online learning and teaching in any institution.

In the face of these opportunities and challenges, the college staff who contributed to the Pockets of Innovation on faculty training and development all expressed their willingness to share their experiences with other colleagues and to learn from each other.


A project using e-readers in Africa

The WorldReader project in Ghana © Wired Magazine, 2010

Trucano, M. (2012) An update on the use of e-readers in Africa EduTech, March 16

Michael Trucano’s excellent World Bank blog here reports on the use of e-readers in Africa, based mainly on a Kindle-based project from an NGO called WorldReader.

Dust and breakage were a problem. Most low cost e-readers are just not robust enough for climatic and usage challenges by children in Africa. (Incidentally, this is a problem the very lost Aakash tablet has run into in India).

However, WorldBreaker has a number of other lessons that it has learned from this project, some of which are in fact recurring themes in many ICT projects in developing countries:

  • lack of cheap content: not enough African-originated material; traditional book publishers are not willing to make texts available for free; need for a rights business model that allows for low cost use ($1 a book?) – to date only 250 African books are available for this project
  • need for support from local education officials
  • need for  support from teachers
  • a need to give reading a higher social currency in many  local cultures, especially those that have very strong oral traditions
  • dedicated ‘face time’ in schools
  • buy in from local support structures at the community level
  • funding to scale up from a pilot to a mid-sized project that can transferred eventually on a larger scale across countries.

Despite these difficulties, there are signs that the project is encouraging greater reading, especially in Grades 4-5.

This project also reminds me of Professor Fred Litto’s project, ‘Escola do Futuro‘ in Brazil in the late 1990s, where he created one of the first open source models for books in Portuguese for Brazilian schools. This project is still running successfully almost 20 years later.

Thanks to Stephen Downes for directing me to this. See also:

Sorrel, C. (2010) Kindle comes to classroom in Ghana Wired Gadget Lab, March 16

Bertelsmann Stiftung (2011) Worldreader brings e-readers to Ghanaian classrooms Future Challenges, July 11

Sniderman, Z. (2011) E-Readers in Africa: Non-Profit Brings Thousands of Books to Ghanaian Children Mashable Social Media, January 26


40 years of comparative research on technology for teaching: ‘weak’ results

© Merlot Biology

Tamim, R. et al. (2011) What Forty Years of Research Says About the Impact of Technology on Learning: A Second-Order Meta-Analysis and Validation Study Review of Educational Research, Vol. 81, No. 1

This study found that looking at studies over 40 years, there is a slight tendency for students who study with technology to do better than students who study without technology.

Don’t get too excited about this. This is a ‘second-order meta-analysis.’ A meta-analysis collects a wide range of research publications on the topic, aggregates the data then runs statistical tests to see if across all the studies there are consistent statistically valid results, even if individual studies didn’t find clear differences or produced mixed or contradictory results. Meta-analyses sometimes find differences ‘missed’ in the original studies, because the larger the sample, the smaller the differences needed to be statistically significant. The results of the meta-analysis depend heavily on the criteria used to chose the original studies.

Note though that this paper  is a meta-analysis of previous meta-analyses, so that although the aggregated sample size of participants is very large, the study is now two levels of analysis away from the original research. You begin to wonder what this really means, especially since that even at the second-order level of analysis, the measured difference between ‘effect’ (studying with technology) and ‘control’ (studying without technology) is quite weak (0.35 on a range of .00 to 1.0). As the researchers themselves conclude:

‘It is important to note that these average effects must be interpreted cautiously because of the wide variability that surrounds them. We interpret this to mean that other factors, not identified in previous meta-analyses or in this summary, may account for this variability….Thus, it is arguable that it is aspects of the goals of instruction, pedagogy, teacher effectiveness, subject matter, age level, fidelity of technology implementation, and possibly other factors that may represent more powerful influences on effect sizes than the nature of the technology intervention.’

Right on, but, as always, I recommend you read the article in full if you believe the results could be important.

The African Health OER Network

Mawayo, M. and Tlaka, M. (2011) The African Health OER Network, SAIDE Newsletter, Vol. 17, No. 5

A useful short description of the African Health OER Network.

The role of the Network is to:

  • Aggregate the results of multiple health education initiatives by collecting, classifying, indexing, and then actively distributing African-initiated resources with the global health community;
  • Facilitate discussion of how these resources can best be used;
  • Share best practices, e.g., OER production and advocacy;
  • Aggregate content to develop and deliver a critical mass of learning materials; and
  • Work through institutions and associations to advocate the principles of openness and of sharing educational materials. This includes helping institutions to create an enabling policy environment for OER production and use.

The African Health OER Network has a nice new web site, with nearly 300 resources for free downloading. About half are in the Public and Community Health area.