January 20, 2017

The success or otherwise of online students in the California Community College system

Listen with webReader

 Online offerings vary widely across subject

Johnson, H. and Mejia, M. (2014) Online learning and student outcomes in California’s community colleges San Francisco CA: Public Policy Institute of California, 20 pp

I’m not a great fan of studies into completion rates in online learning, because most studies fail to take into account a whole range of factors outside of the mode of delivery that influence student outcomes. However, this study is an exception. Conducted by researchers at the highly influential PPIC, it takes a very careful look at how well students across the whole California community college system (CCCS) do in online learning, and there are some very interesting findings that may not come as a surprise to experienced observers of online learning, but will certainly provide fodder for both supporters and skeptics of online learning.

Why the study is important

Several reasons:

  • California’s community colleges offer more online credit courses than any other public higher education institution in the country. By 2012, online course enrollment in the state’s community colleges totaled almost one million, representing about 11 percent of total enrollment
  • Over the past ten years, online course enrollment has increased by almost 850,000, while traditional course enrollment has declined by almost 285,000.
  • Community colleges are more likely than other institutions of higher education [in the USA] to serve nontraditional students. These students often have employment and family obligations and therefore may potentially benefit the most from online learning.
  • The state of California is investing $57 million over the next five to six years for online learning initiatives within the California Community College system
  • The California Community Colleges Chancellor’s Office (CCCCO) provided … access to unique longitudinal student- and course-level data from all of the state’s 112 community colleges

Main findings

  • Between 2008–09 and 2011–12, total credit enrollment at California’s community colleges declined by almost a million. The scarcity of traditional courses has been a factor in the huge increase in online enrollments. With the state cutting support to community colleges by more than $1.5 billion between 2007–08 and 2011–12, community colleges experienced an unprecedented falloff in enrollment 
  • online course success rates are between 11 and 14 percentage points lower than traditional course success rates.
  • in the long term, students who take online classes tend to be more successful than those who enroll only in traditional courses…students who take at least some online courses are more likely than those who take only traditional courses to earn an associate’s degree or to transfer to a four-year institution.
  • for students juggling school, family and work obligations, the ability to maintain a full-time load by mixing in one or two online courses per term may outweigh the lower chances of succeeding in each particular online course.
  • if a student’s choice is between taking an online course or waiting for the course to be offered in a classroom at a convenient time, taking the online course can help expedite completion or transfer
  • participation in online courses has increased for each of the state’s largest ethnic groups—and online enrollment rates for African American students, an underrepresented group in higher education in California, are particularly high. However, these rates are much lower among Latino students.

Main recommendations

  • move from ad hoc offerings to more strategic planning of online courses
  • improve the ability to transfer credits between community colleges and between colleges and the state’s universities
  • improve the design and provide more consistency in the quality of online courses between institutions
  • adopt a standardized learning management system across all colleges
  • collect systematic information on the cost of developing and maintaining online courses

My comments

This is another excellent and succinct research report on online learning, with a very strong methodology and important results, even if I am not at all surprised by the outcomes. I would expect online completion rates for individual courses to be lower than for traditional courses as students taking online courses often have a wider range of other commitments to manage than full-time, on campus students.

Similarly, I’m not surprised that online course success is lightly lower for community colleges than for universities (if we take both the figures from Ontario and my own experience as a DE director) and for certain ethnic groups who suffer from a range of socio-economic disadvantages. Online learning is more demanding and requires more experience in studying. Post-graduate students tend to do better at online learning than undergraduate students, and final year undergraduate students tend to do better than first year undergraduate students. Nevertheless, as the study clearly indicates, over the long term online learning provides not only increased access but also a greater chance of success for certain kinds of students.

I am worried though that online learning in California has ‘succeeded’ because of the massive cuts to campus-based education. It is better than nothing, but online learning deserves to be considered in its own right, not as a cheaper alternative to campus-based education. Online learning is not a panacea. Different students have different needs, and a successful public post secondary education system should cater to all needs. In the meantime, this is one of the most useful studies on online completion rates.

 

 

 

A comprehensive review of the literature on digital natives

Listen with webReader

Image retrieved from Hastac.org (Doug Beg's blog)

Image retrieved from Hastac.org (Stephen Berg’s blog)

Jones, C. and Shao, B. (2011) The Net Generation and Digital Natives: Implications for Higher Education Milton Keynes: Open University/Higher Education Academy

This paper is required reading for graduate students studying online learning or educational technology. The paper is little old (by Internet standards) but I just came across it looking for something else.

The discussion about ‘digital natives’ has gone quiet recently, and this paper might be one reason why. The authors have made a thorough review of the literature on this topic, with over 200 appropriate references, including surveys of relevant publications from countries in Europe, Asia, North America, Australia and South Africa. Here are some of their main conclusions, although the report is best read in full:

  • there is no evidence that there is a single new generation of young students entering Higher Education and the terms Net Generation and Digital Native do not capture the processes of change that are taking place;
  • demographic factors interact with age to pattern students’ responses to new technologies;
  • the gap between students and their teachers is not fixed, nor is the gulf so large that it cannot be bridged. In many ways the relationship is determined by the requirements teachers place upon their students to make use of new technologies and the way teachers integrate new technologies in their courses. There is little evidence that students enter university with demands for new technologies that teachers and universities cannot meet;
  • students do not naturally make extensive use of many of the most discussed new technologies such as Blogs, Wikis and 3D Virtual Worlds….Students who are required to use these technologies in their courses are unlikely to reject them and low use does not imply that they are inappropriate for educational use. The key point being made is that there is not a natural demand amongst students that teaching staff and universities should feel obliged to satisfy;
  • students will respond positively to changes in teaching and learning strategies that are well conceived, well explained and properly embedded in courses and degree programmes. However there is no evidence of a pent-up demand amongst students for changes in pedagogy or of a demand for greater collaboration;
  • the development of university infrastructures, such as new kinds of learning environments (for example Personal Learning Environments) should be choices about the kinds of provision that the university wishes to make and not a response to general statements about what a new generation of students are demanding; 
  • the evidence indicates that young students do not form a generational cohort and they do not express consistent or generationally organised demands. A key finding of this review is that political choices should be made explicit and not disguised by arguments about generational change.

Comment

This paper is a timely correction to the hype around digital natives, especially the claims made by Tapscott and Prensky. It is so easy to find a buzz-word or phrase and through constant repetition and media hype present a gross over-simplification of what are often subtle and complex changes.

It is also important to pay attention to what Jones and Shao are not saying. They are not saying that social media, personal learning environments, or collaborative learning are inappropriate, nor that the needs of students and the workforce are unchanging or unimportant, but the use of these tools or approaches should be driven by a holistic look at the needs of all students, the subject area and society, and not by an erroneous view of what a particular generation of students are demanding.

 

Who benefits from online learning?

Listen with webReader

Di Xu & Shanna Smith Jaggars (2013) Adaptability to Online Learning: Differences Across Types of Students and Academic Subject Areas New York: Community College Research Center, Columbia University

The study

Using a dataset containing nearly 500,000 courses taken by over 40,000 community and technical college students in Washington State, this study examines how well students adapt to the online environment in terms of their ability to persist and earn strong grades in online courses relative to their ability to do so in face-to-face courses. 

The hypothesis

Some populations of students—for example, those with more extensive exposure to technology or those who have been taught skills in terms of time-management and self-directed learning—may adapt more readily to online learning than others. In addition, some academic subject areas may lend themselves to highquality online learning experiences more readily than others

The methodology

Primary analyses were performed on a dataset containing 51,017 degree-seeking students who initially enrolled in one of Washington State’s 34 community or technical colleges during the fall term of 2004. These first-time college students were tracked through the spring of 2009 for 19 quarters of enrollment, or approximately five years.  The dataset, provided by the Washington State Board of Community and Technical Colleges (SBCTC), included information on student demographics, institutions attended, and transcript data on course enrollments and performance.

The results

  • In descriptive terms, students’ average persistence rate across courses was 94.12 percent, with a noticeable gap between online courses (91.19 percent) and face-to-face courses (94.45 percent). For courses in which students persisted through to the end of the term (N = 469,287), the average grade was 2.95 (on a 4.0-point scale), also with a gap between online courses (2.77) and face-to-face courses (2.98).
  • While all types of students in the study suffered decrements in performance in online courses, some struggled more than others to adapt: males, younger students, Black students, and students with lower grade point averages. 
  • Regardless of a particular student’s own adaptability to the online environment, her performance in an online course may suffer if her classmates adapt poorly. English and social science were two academic subjects that seemed to attract a high proportion of less-adaptable students, thereby introducing negative peer effects.
  • Older students adapted more readily to online courses than did younger students.
  • [Also] students who were more disposed to take online course also tended to have stronger overall academic performance than their peers  

 

Comments

First, it is encouraging to see a detailed quantitative assessment of the types of students taking online courses, and their relative performance. This report needs to be read in full, and carefully. It is good that it is based on a significantly large enough sample that one can have confidence in the generalizability of the results (at least in the U.S. two-year college sector). The study was very well carried out and is a model for quantitative analysis of student differences.

Furthermore, I am not surprised or even concerned about these findings. For instance, from my own experience of online teaching, I would agree that ‘students are not homogeneous in their adaptability to the online delivery format and may therefore have substantially different outcomes for online learning.’ Online learning doesn’t suit everyone, and it is valuable to have some research that helps identify the more ‘at-risk’ online learners.

One can put forward a number of reasons why online students, on average, are likely to struggle compared with face-to-face students. Students who choose an online course are likely on average to have less time for study that those attending regularly on campus. Second, for many online students, the mode of study will be unfamiliar, which means making more adaptation to a different way of learning.

My one quibble is that, although the results are clearly significant statistically (as is almost inevitable in large samples), the differences are quite small (96% vs 91% completion rates, for instance.) Thus I do challenge the authors’ conclusion that ‘most students had difficulty adapting to the online context.’ If 91% complete the course, then most students did not have difficulties sufficient to deter them from completing successfully their courses. That seems a pretty good adaptation level to me.

I also have a concern that these results will be misinterpreted. This should not mean that men, Blacks and young people should be discouraged from taking online courses, but that we should be taking more care to ensure that students who do take online courses are better prepared, with particular attention being paid to those likely to be at most risk. This may mean, for instance, gradually introducing students to online learning in a deliberate way throughout a program. This study does suggest those most likely to be at risk.

Further reading

Lederman, D. (2013) Who benefits from online Ed? Inside Higher Education, February 25

 

When MOOCs crash and burn

Listen with webReader

Jaschick, S. (2013) MOOC Mess, Inside Higher Education, February 4

Krause, S. (2013) Two Thoughts on the crash of the “Fundamentals of Online Education” MOOC Stevendkrause.com, February 4

Strauss, V. (2013) How online class about online learning failed miserably Washington Post, February 5

Well, it had to happen. You all probably know the story of the course on how to plan and manage online courses that wasn’t planned or managed. After two weeks, with 40,000 students online, this course from an instructor at the Georgia Institute of Technology really crashed and burned. The tool selected for group-work couldn’t handle the numbers, the faculty member was overwhelmed, so cancelled the course with barely an explanation.

Now I do feel sympathy not only with the students, but also with the instructor. I’ve made mistakes on my first online courses, but here are some questions that someone should be asking:

  1. Where is the quality control? Surely Coursera should accept some responsibility for this. They are getting paid by the institutions to host these courses. Shouldn’t they at least be asking some questions about what tools people are planning to use, and whether or not they will work with very large numbers? Are they doing due diligence before accepting and advertising their MOOCs? Apparently not. Nor did Georgia Institute of Technology. What has this done to its reputation?
  2. Are questions being asked about the qualifications or experience of the people who are offering MOOCs? Just a brief glance at this particular course suggests that the instrutor had little experience herself in planning and managing online courses. Georgia Institute of Technology is not at the top of my list of institutions with experience in online learning. But then, anyone can teach an online course about online learning, can’t they?

What really makes me angry is that badly designed MOOCs do such damage, both to their students and to the image of online learning. You just cannot go on ignoring best practice and hope to get away with it, especially with 40,000 students in the class.

Again, this is a classic example of the clash between computer scientists and educators. If you are trying out an app and it fails, no big deal, you’ve lost a bit of money – try another one. But in education you are playing with people’s hopes and dreams. Do the job properly.

Who has the richest professors? Canada!?

Listen with webReader

© Higher Ed Morning, 2011

Jaschik, S. (2012) Faculty Pay, Around the World, Inside Higher Education, March 22

Philip Altbach, Liz Reisberg, Maria Yudkevich, Gregory Androushchak, Iván Pacheco (in press) Paying the Professoriate: A Global Comparison of Compensation and Contracts London/New York: Routledge

The Inside Higher Education reports on a fascinating study conducted by researchers from the USA and Russia that compares academic salaries around the world, after adjusting for the cost of living in each country (but not for taxes, for some reason).

Canada has the highest paid faculty, both at entry and as an overall average, followed by Italy, South Africa, India and the United States, in that order.

The web site for the project has several very interesting maps showing relative salaries by country.

Comment

Such comparative studies are always open to criticism (see the comments after the Inside Higher Education article), but having travelled to many of the countries, the results make sense to me. It’s how you interpret them and what they may result in that matters.

First, one result of globalization is that there is an ‘arms race’ in salaries to attract the best faculty. However there are other factors too, such as the availability of research grants, and working conditions, especially teaching load, and other extras, such as the availability of consultancies and other extra work, and the proportion of much lower paid contract instructors. Nevertheless, the need for knowledge workers and the importance given by governments in many countries to their universities in producing such graduates means that there will be continued pressure to keep salaries increasing in a ‘free’ global labour market. In this sense, it’s probably good for Canada that it’s ‘top’.

Second, the report points out that despite the trends in salaries, most university faculty are well outside the top 1% of the wealthy.

Third, the USA is such a diverse system, with many ‘universities’ that barely deserve the name, as well as top rank elite universities, that an average for that country doesn’t mean a lot. Canada’s high salaries generally result from making comparisons with the elite universities just south of the border.

Fourth, expect Canadian salaries to come under pressure (or rather, not to continue to increase at a rate compared with other of the top five countries) over the next few years as some provincial governments grapple with budget deficits, and students (and politicians) try to limit increases to tuition fees. Canadian universities slid relatively unscathed through the 2008 economic recession compared to the state universities in the USA, who are currently undergoing massive cuts to their budgets, but that comparative advantage is not going to continue for ever as the US economy slowly recovers.

Fifth, India and China make interesting comparisons. Despite a massive increase in student numbers in both countries, India has managed to protect the salaries of its academics across the board, while China has had to rely on paying high salaries to a small elite of professors, but the general faculty salaries are still low in China even when the relatively low cost of living is taken into account. I leave it to you to speculate on what that ‘gap’ in salaries means for the future.

Lastly, what does all this mean in terms of value for money? Faculty salaries are by far the biggest single item in university budgets, accounting for at least 70% of all teaching costs. They have increased at least in North America at a much faster rate than inflation over the last 20 years, while teaching loads have actually dropped. Do higher salaries though lead to better teaching? I doubt it. What universities are looking for are top researchers rather than top teachers. Is there any way to tie increased salaries to teaching performance? Probably not, except perhaps for internal promotions. It looks like students and/or tax payers in Canada will continue to pay more without any expectation that the teaching will get any better or more productive. How depressing.