From the abstract:

Proponents of postsecondary online education were recently buoyed by a meta-analysis sponsored by the U.S. Department of [...]]]> Smith Jaggars, S. and Bailey, T. (2010) Effectiveness of Fully Online Courses for College Students: Response to a Department of Education Meta-Analysis New York: Community College Research Center, Teachers College, Columbia University.

From the abstract:

Proponents of postsecondary online education were recently buoyed by a meta-analysis sponsored by the U.S. Department of Education suggesting that, in many cases, student learning outcomes in online courses are superior to those in traditional face-to-face courses. This finding does not hold, however, for the studies included in the meta-analysis that pertain to fully online, semester-length college courses; among these studies, there is no trend in favor of the online course mode. What is more, these studies consider courses that were taken by relatively well-prepared university students, so their results may not generalize to traditionally underserved populations. Therefore, while advocates argue that online learning is a promising means to increase access to college and to improve student progression through higher education programs, the Department of Education report does not present evidence that fully online delivery produces superior learning outcomes for typical college courses, particularly among low-income and academically underprepared students. Indeed some evidence beyond the meta-analysis suggests that, without additional supports, online learning may even undercut progression among low-income and academically underprepared students.

Comment

This is a study that should be read by all those proposing to research the differences between online and face-to-face teaching. It is a very nice dissection of the U.S. Department of Education study:

Means, B. et al. (2009) Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies Washington, DC: US Department of Education

The Columbia University researchers found that of the 28 ‘rigorous, scientific studies’ that compared online vs face-to-face teaching in the Department of Education study, only seven looked at ‘typical’ semester-long courses, and these were all in universities. Close examination of these seven studies by the Columbia University group found (p.10) that ‘Overall, then, the online courses showed no strong advantage or disadvantage in terms of learning outcomes among the samples of students under study.’

More interestingly, Smith Jaggers and Bailey argued that the U.S. Department of Education study produced no evidence to indicate that online learning is widening access to low-income or other disadvantaged learners, nor is there anything else in the literature to support this argument for online learning (a dangerous conclusion, because it only needs one study to come to light to challenge the Columbia University conclusion – in other words, never say never in research. Does anyone know of such a study that looks at whether online learning widens access to disadvantaged groups?).

None of this is surprising. Anyone wanting to do comparative research between face-to-face and technology delivered teaching should say over and over again what Wilbur Schramm said as long ago as 1974: we should not ask whether one means of delivery is better than another, but what are the conditions that determine the appropriate choice of technology.

My view of meta-analysis research is that it should be dumped in the garbage, as far as education is concerned. The in-condition variables are always greater than the between-condition variables, and these are lost in meta-analyses. In other words, it depends on how well the teaching is done, whatever the medium.

So, coming back to the Smith Jaggers and Bailey conclusion: can online learning widen access to disadvantaged groups? Well, first, this may be a bit of a straw man. I don’t recall much of the rationale for online learning in the literature being that it can recruit students from high school who can’t get into college – or don’t want to go. This is a tough market to reach, by any means, and often these students do not have the confidence or independent learning skills to study wholly online. The main justification from my perspective for fully online learning (apart from the skills it develops) is that it suits best adult and lifelong learners who cannot otherwise access college, but have pretty good learning skills already (and may well already have a conventional community college or university education). Online learning may be used to help some kinds of high school drop-outs, but it would need to be combined with a whole range of other strategies, such as personal counselling and one-on-one learner support.

So can online learning widen access to disadvantaged groups? Yes, probably, if the right conditions are there, such as quality learner support, subsidized programs, etc. However, the same conditions could equally apply to face-to-face programs – or different conditions. Neither the U.S. Department of Education nor the Columbia University study provides any evidence on this question, and trying to make this sort of comparison is a waste of time. So focus on the conditions that support teaching and learning, accepting that there likely to be some conditions where online teaching is more or less appropriate than other conditions.

Now try to explain this to your professor who wants a simple answer to the question as to whether online teaching is ‘as good as’ face-to-face teaching. The answer is of course, ‘Yes, provided that…..’ or ‘it all depends….’

See also:

Wojciechowska, I. (2010) Continuing debate over online education Inside Higher Education, July 16

This article suggests a change in corporate tactics regarding the commercialization of online learning. The article argues that due to the increased pressure from the US Department of Education on for-profit online institutions such as Kaplan and [...]]]> Parry, M. (2010) Outsourced Ed: Colleges Hire Companies to Build Their Online Courses Chronicle of Higher Education, July 18

This article suggests a change in corporate tactics regarding the commercialization of online learning. The article argues that due to the increased pressure from the US Department of Education on for-profit online institutions such as Kaplan and the University of Phoenix, investors are turning to companies that will outsource e-learning for not-for-profit private and public campus-based institutions, such as the University of Southern California, Northeastern University and Boston University.

Comment

For many who work as learning technology support staff in public universities, this is often a nightmare scenario. How do you deal with a VP Development or VP Academic who thinks the outsourcing of online learning is a great idea? (I know, because I worked for one where this happened).

For a start, for some institutions without any previous track record of developing online learning, it may be a sensible option, particularly for continuing education or extension departments that are expected to generate revenue, but don’t have experience in this area. It may even make sense where there is a history of online teaching, but it’s not been a good one, in terms of quality or student satisfaction.

However, here is a set of questions that need to be asked before any ‘deal’ is signed:

1. Will online learning remain a peripheral activity in this institution, or is it likely to become a core part of our teaching? The more integral online learning becomes to an institution, the weaker the case for outsourcing. (Another way of putting this is: would it make sense for an institution to outsource its teaching so it can concentrate on research? If not, then why outsource online learning, which is just one way of teaching?)

2. How important is it for the institution to develop its own expertise in online learning, from a teaching and learning perspective? If the institution is to remain entirely classroom or campus based in all its programs, outsourcing may make sense. In this case, though, will the outsourced online courses have the same status and respect as the campus-based programs?

3. Is online learning a way to gain market share and enhance the reputation of the university, or is it seen mainly as a way to generate extra revenues? If extra revenues are the main goal, outsourcing may make sense, but then:

4. If the aim is to make money from online offerings, will we make more money by developing our own in-house expertise than by outsourcing? Do we have business cases for both options? What is the best business case over the long-term compared to the short-term (one to five years)?

5. If the aim is to increase market share or enhance the reputation of the university, what impact will outsourcing have on the control and management of teaching? What role if any will tenured faculty play if online learning is outsourced? What impact will that have on the reputation of the university?

6. How are key stakeholders (faculty, students, employers, the board, the government) likely to respond to outsourcing? Will they support or oppose it?

7. What controls will the institution have over the quality of the outsourced teaching? For instance, who will select instructors to teach in the program and on what basis?

I am not arguing against outsourcing on principle. Some outsourcing companies are flexible enough to develop contracts that will ensure high quality and a good return to the institution, and there are situations where this will make sense. The key question though is: what is your ‘core’ mission? Are we a teaching and learning institution, and if so, why would we contract this out to someone else? If they can do teaching and learning better than us, why are we doing it? Or is our mission to become just an accreditation agency that also does research?

Underlying all this is the belief that online teaching is second rate and therefore can be contracted out without loss to the institution. Big mistake.

Reference

Bok, D. (2003) Universities in the Marketplace: The Commercialization of Higher Education Princeton NJ: Princeton University Press

Fusch, D. (2010) Integrating e-portfolios in your assessment strategy Academic Impressions, July 16

Chen, H. and Light, T. (2010) Electronic Portfolios and Student Success: Effectiveness, Efficiency, and Learning Washington DC: Association of American Colleges and Universities

Trent Batson provides a critique of existing testing practices, with their emphasis [...]]]> Batson, T. (2010) The testing straightjacket Campus Technology, July 7

Fusch, D. (2010) Integrating e-portfolios in your assessment strategy Academic Impressions, July 16

Chen, H. and Light, T. (2010) Electronic Portfolios and Student Success: Effectiveness, Efficiency, and Learning Washington DC: Association of American Colleges and Universities

Trent Batson provides a critique of existing testing practices, with their emphasis on memorization, and suggests that e-porfolios provide a more authentic form of assessment. David Fusch interviews Tracey Penny Light, of the University of Waterloo, who offers several steps for integrating e-portfolios into an instructor’s assessment strategy.

Helen Chen and Tracey Light have just published a short handbook (44 pp) that ‘presents an overview of electronic portfolios and ways individuals and campuses can implement e-portfolios to enhance and assess student learning, recognizing that learning occurs in many places, takes many forms, and is exhibited through many modes of representation. It is organized around eight issues central to implementing an e-portfolio approach: defining learning outcomes; understanding your learners; identifying stakeholders; designing learning activities; including multiple forms of evidence; using rubrics to evaluate e-portfolios; anticipating external uses of evidence; and evaluating the impact of e-portfolios. This work is illustrated through multiple campus case study examples.’

Batson, T. (2010) Innovation in Higher Education: It’s Not the Technology Campus Technology, June 2

There is an increasing awareness that for technology to be used effectively, there has to be changes in the way people work. This ‘truth’ is only [...]]]> Miller, B. (2010) The Course of Innovation: Using Technology to Transform Higher Education Washington DC: EducationSector

Batson, T. (2010) Innovation in Higher Education: It’s Not the Technology Campus Technology, June 2

There is an increasing awareness that for technology to be used effectively, there has to be changes in the way people work. This ‘truth’ is only slowly penetrating the post-secondary education sector.

The National Center for Academic Transformation has for over 10 years now been working with 100 major universities and colleges in the USA to redesign large first and second year lecture classes. From all the reports from NCAT, these redesigns are highly successful, reducing costs and leading to better learning outcomes.

However, if these interventions have been so successful, how comes that without the financial incentives provided through foundation grants for most of the NCAT projects, the redesign process has not been adopted generally throughout the system? This question is explored in a report from EducationSector, (a non-profit, non-partisan independent think tank) which states the problem as follows:

Despite the worst fiscal environment for higher education in a generation and mountains of evidence that NCAT-style reforms are effective, just over a hundred colleges out of nearly 7,000 nationwide have worked with the center to transform a course. This failure has broad implications for the way state and national leaders should think about the pressing challenge of helping more students earn an affordable college degree.

A major conclusion from this study is:

Reluctance to change is hardwired into many of the structural features that define today’s colleges and universities, and it will be very difficult to achieve large-scale reforms of any sort without dealing with them directly. The root of the dilemma lies with the decentralized and inherently conservative nature of the modern higher education institution. (p.13)

The report goes on to draw some interesting conclusions from the NCAT experience and makes recommendations to support change in institutions.

Trent Batson, always a realistic optimist (if that’s not an oxymoron), does see signs of gradual change, stemming from a growing realization that:

the myth that the technology does something itself to bring about significant human change in teaching/learning/assessment practices has been “busted.” Campuses are instead accepting the obvious truth that some human change must come first, that time and human commitment to a sustainable support system must precede technology adoption, and that educators themselves must lead technology initiatives.

Nevertheless, having read these two excellent articles/reports, I have two conclusions (see also Keith Hampson’s Higher Education Management Group blog for his thoughtful reflections on the EducationSector report):

1. At what point will faculty themselves take responsibility for institutional change, without having to be bribed to do it? Is there really no integrity or ‘greater purpose’ in university and college teaching these days that would drive faculty to do things differently, for the benefit of students and the general tax payer or are they really so self-centered on their academic careers? This does not square with my knowledge of many faculty, who do care and do want to make things better (there’s just not enough of them to overwhelm the rest). I believe that it comes down in the end to five major underlying barriers to change:

• lack of effective leadership, not so much in intent, but more so in knowing how to manage institutional change successfully
• lack of training of both administrators in how to manage change (see above); and particularly lack of training for faculty in how to teach effectively. Without a good grounding in change management and pedagogical theory and practice, making the necessary changes is impossible
• academic career incentives that reward research and punish innovation in teaching and learning
• lack of successful alternative models of governance for public post-secondary institutions, partly because of government and academic unions’ reluctance to create or allow radically different institutions
• complacency with the current dominant teaching paradigm, which places the instructor at the centre of the teaching and learning process, not the student.

2. Change will be impossible without changing the governance of universities and colleges. This is an issue sorely in need of research and discussion, because it is not a simple matter. It is not really a question of centralising power in the executive or board of governors, but of finding better ways to encourage faculty and administrators to see the benefits of change and take responsibility and reward them for doing it. However, our current methods of devolved and decentralized governance (which at its extreme is: let faculty do what they want, and at its best is: don’t upset the deans) are clearly inadequate for the challenges being faced.

Thanks to Keith Hampson, Ryerson University, and Gary Munro, Justice institute of BC, for drawing my attention to these articles.

If you go to Helen Barrett’s website you will get a two-for-one: a very helpful diagram of the two aspects of e-portfolios (learning and collaboration; and showcasing achievement), plus a copy of the article.

Thanks to Richard Elliott’s [...]]]> Barrett, H. (2010). Balancing the Two Faces of ePortfolios. Educação, Formação & Tecnologias, Vol. 3,No. 1, pp. 6-14.

If you go to Helen Barrett’s website you will get a two-for-one: a very helpful diagram of the two aspects of e-portfolios (learning and collaboration; and showcasing achievement), plus a copy of the article.

Thanks to Richard Elliott’s e-Learn Watch for directing me to this.

Baker, S. (2010) Hefce gives out extra places and takes back £20m from teaching funds Times Higher Education, June 25

Calhoun, T. (2010) Re-imagining Higher Education, Post-Recession SCUP Links Blog, June 27

Kamenetz, [...]]]> It’s funny how reports on the same issue arrive from completely different directions. These four all deal with the issue of innovation and higher education.

Baker, S. (2010) Hefce gives out extra places and takes back £20m from teaching funds Times Higher Education, June 25

Calhoun, T. (2010) Re-imagining Higher Education, Post-Recession SCUP Links Blog, June 27

Kamenetz, A. (2010) Online education an the laying on of hands Huffington Post, June 29

OECD (2010) The OECD Innovation Strategy: Getting a Head Start on Tomorrow Paris: OECD

Let’s start with a focus on the financial conditions that universities and colleges will be facing in North America and Europe over the next five years. One clear outcome from the recent G20 meeting in Toronto (apart from torched police cars and broken windows) is that governments are moving away from stimulus funding to deficit and debt reduction. This will vary from country to country. In Britain, many government departments are looking at a 25% reduction in funding. The Times Higher Education article highlights the first steps in Britain: a £20 million ($30 million) reduction in funding for university places. This comes on top of earlier cuts of £900 million ($1.35 billion) in December, 2009.However, it should be noted that $10 million was ‘protected’ for another 10,000 places, and the bulk of this money went to the Open University and the rest to ‘newer, teaching focused universities’.

The Calhoun article is an interview with Donald Norris and Linda Baer, who state that:

‘data from SHEEO (State Higher Education Executive Officers, who manage state funding to universities in the USA) demonstrate that the deficits facing states will be deeper than previous recessions, will last longer, and that there will be no bouncing back to normal like after the recessions of the past 30 years. The new normal will be diminished state appropriates, on average about 20% down over the next three years. This will require institutions not just to muddle through, but to reimagine themselves for the new normal.’ If higher education hasn’t established genuine financial sustainability through reinvention by 2020, we will have missed our chance to shape our future. Others will do it for us.

This is a multi-year campaign, not a single quick fix in response to mid-year budget cuts. It begins with establishing the need for establishing a sustainable vision for 2020 – financially, programmatically, organizational, and politically.  We expect that institutions will need to use the 2010-2013 period to launch processes of reimagination and reinvention, then progressively redirect their energies so that by 2020 they have leveraged innovations, achieved greater levels of academic and administrative productivity, fresh revenues, and an appreciation for the value propositions required in the new normal.  This is a tall order, but we cannot escape the implications of the times.

Then we have the response from the institutions. Anna Kamenetz, the author of KDIY U: Edupunks, Edupreneurs, and the Coming Transformation of Higher Education has an interesting blog in the Huffington Post, where she describes the negative reaction she got to her book from a panel at a conference in San Diego. Her response was:

If people who care about both quality and equality in higher education don’t get deeply involved in the use of technology to stretch the resources we have in order to educate everyone to the best of our ability and their abilities, then the future will be shaped by people with worse motives and visions.

In other words, she is arguing that public institutions will have to improve their productivity if they are to maintain quality with less rather than more money. This requires innovation in teaching and learning, the main message from our book about the strategic management of technology.

What is needed to support innovation? The OECD report argues that

‘in economically advanced countries future growth must increasingly come from innovation-induced productivity growth. Innovation encompasses a wide range of activities in addition to R&D, such as organisational changes, training, testing, marketing and design. Innovation is defined as the implementation of a new or significantly improved product (good or service), or process, a new marketing method, or a new organisational method in business practices, work-place organisation or external relations.’ [Does this not sound like e-learning?]

Although the OECD report is more focused on businesses, these statements apply equally well to universities and colleges. There are two aspects:

• preparing their students in such a way that students can foster innovation in the workplace when they leave
• ensuring that the necessary internal changes take place within institutions to support innovation in teaching. research and administration

Among the OECD policy principles for innovation are:

1. Empowering people to innovate

• Education and training systems should equip people with the foundations to learn and develop the broad range of skills needed for innovation in all of its forms, and with the flexibility to upgrade skills and adapt to changing market conditions. [In other words, faculty need to be trained in new skills, and new approaches to teaching.]

To transform ideas and inventions into innovation requires a range of activities, including organizational changes, organizational-level training, testing, marketing and design.

I believe that these policy principles apply equally well to our post-secondary educational institutions. Look also at what the OECD says about how to develop a culture of innovation and ask yourself if this would apply to your own institution:

People generate the ideas and knowledge that power innovation, and they apply this knowledge and the resulting technologies, products and services in the workplace and as consumers. Innovation requires a wide variety of skills, as well as the capacity to learn, adapt or retrain, particularly following the introduction of radically new products and processes. Empowering people to innovate relies not only on broad and relevant education, but also on the development of wide-ranging skills that complement formal education.

Does this apply to faculty?

Lastly, the OECD report also focuses on how curricula need to change to encourage the development of skills that lead to innovation in the workplace:

Formal education is the basis for forming human capital, and policy makers should ensure that education systems help learners to adapt to the changing nature of innovation from the start. This requires curricula and pedagogies that equip students with the capacity to learn and apply new skills throughout their lives. Emphasis needs to be placed on skills such as critical thinking, creativity, communication, user orientation and team- work, in addition to domain-specific and linguistic skills. The acquisition of skills is a lifelong process; it does not end with formal education. Schools lay a base for lifelong learning, but ongoing skills acquisition needs to be encouraged. This involves recognising all forms of learning and making them visible, including through qualification systems. Rewarding lifelong learning and making it attractive may help enhance participation.

Universities, colleges and vocational training centres are essential nodes in the innovation system, both producing and attracting the human capital needed
for innovation. These institutions act as essential bridges between players –businesses, governments and countries – in broader and more open systems of innovation. They also contribute to the local quality of life and thus can help to attract the highly skilled from around the globe. World-class institutions can be the anchor for clusters of innovative activity. The major policy challenge is to recognise the essential role of universities in the innovation enterprise rather than view them, as is all too commonly the case, simply as providers of essential public goods. This requires a greater focus of policy makers on ensuring independence, competition, excellence, entrepreneurial spirit and flexibility in universities.

In short, universities and colleges are critical to developing graduates that can support innovation in the work-place. Perhaps even more importantly, though, the institutions themselves will have to find ways to innovate to provide quality services with less money. The appropriate use of technology, as with innovation in other domains, will be an essential component of that process.

So some questions, dear readers:

1. Do our (public) institutions really need to change, or is this just the usual North American hype and hyperbole?

2. If they do need to change, are they up to it? Do they have the will, skills, knowledge and attitude to make the changes necessary?

3. Is e-learning an essential component of any needed changes, or could the institutions manage the necessary changes without a heavy reliance on e-learning?

4. What is needed to bring about any necessary changes in our institutions?

Over to you.

I found this a thoughtful if somewhat predictable reflection (how appropriate) on e-portfolios. Batson asks (and tries to answer) the question:

‘For the vast majority of …. students…, what is the value of their collection of evidence once they are beyond the course or [...]]]> Batson, T. (2010) Is portfolio evidence useful? Campus Technology May 19

I found this a thoughtful if somewhat predictable reflection (how appropriate) on e-portfolios. Batson asks (and tries to answer) the question:

‘For the vast majority of …. students…, what is the value of their collection of evidence once they are beyond the course or departmental level?’

There are some other questions that could be asked about e-portfolios:

How will external people know that it is the student’s own work, especially after they have left the institution? I am sure there is a technical answer to this question, but I would like to know the details. And does it matter? (I suspect it does).

Will the institution need to keep a copy of the authenticated e-portfolio in perpetuity? (The library in the University of British Columbia is already collecting and archiving, with student’s permission, copies of their digital work – see http://www.tonybates.ca/2010/01/31/a-personal-view-of-e-learning-at-the-university-of-british-columbia/). Will or should this be available to the public without the student’s permission? What if an employer for instance wants to see the archived version as a check? Who should be responsible for managing the storage of university/college-assessed e-portfolios: the Registrar’s Office, the Library, or the student?

If it is to be a lifelong learning tool, then the student will need to be able to go in and change it after graduation – will this impact on the authenticity of the e-portfolio after it has been ‘accredited’ by an institution as the student’s own work? Does this mean there will be multiple versions of an e-portfolio?

How will employers overall react to e-portfolios? Has anyone done any research on this? Will they need to be ‘educated’ about them, and if so, what’s the best way to do this?

I am sure that there are other questions to be asked, as well. I agree with Batson that e-portfolios can dramatically change how we assess students, and for the better; I also agree that e-portfolios have intrinsic value as a learning tool for the student. But we do need to provide answers to these questions as well, if e-portfolios are to be accepted as an alternative form of assessment, which is what I would like to see. Or will that wreck their value?

This paper answers two important questions: Is it taking students longer these days to graduate at the bachelor’s level? And if so, why?

The answer: yes, more students today are taking longer than [...]]]> Bound, J., Lovenheim, M. and Turner, S. (2010) Increasing Time to Baccalaureate Degree in the United States Cambridge MA: National Bureau of Economic Research

This paper answers two important questions: Is it taking students longer these days to graduate at the bachelor’s level? And if so, why?

The answer: yes, more students today are taking longer than in 1972, and the reason is that in the non-elite public universities, instructor:student ratios have deteriorated and students have to work more as well as study to pay tuition fees and other costs.

To obtain a copy of the full report, go to: http://papers.nber.org/papers/w15892

For a full description of the report see:

Jaschik, S. (2010) Why they take so long Inside Higher Education, April 14

The reason I find this interesting is that the elite schools still have ‘regular’ completion rates and have in fact improved their instructor:student ratio, and their students, presumably because they come from wealthier families, do not need to work so much. This suggests to me that the mass of state and public institutions need to provide even more flexibility in their delivery and need to change their teaching methods to increase interaction with students in larger classes: all of which can be assisted by technology, if this is accompanied by other strategies as well.

This article describes how three small colleges migrated to either Sakai or Moodle from WebCT.

I liked this article very much. Trent Batson lays out a whole host of compelling reasons why it is so difficult to get faculty to change and use technology more and better. His main argument is that the whole ecology of higher educational institutions [...]]]> Batson, T. (2010) Let faculty off the hook Campus Technology, March 17

I liked this article very much. Trent Batson lays out a whole host of compelling reasons why it is so difficult to get faculty to change and use technology more and better. His main argument is that the whole ecology of higher educational institutions reinforces the status quo.

My only quibble is that these conditions are only there because faculty actually make most of the decisions that continue these practices – particularly peer review – so they can’t escape the hook. Faculty are a major part of the problem.

The point is that all members of post-secondary institutions need to take responsibility for change, to make the institution fit for purpose in the 21st century. But faculty need help, in terms of proper training, time to adjust, and vision and leadership from the administration.