September 2, 2014

Resources and the design of teaching and learning

Listen with webReader

 Steelcase Node Classroom

Steelcase Node Classroom

The story so far

Chapter 5 of my open textbook, ‘Teaching in a Digital Age’ is about the design of teaching and learning, which I am currently writing and publishing as I go.

I started Chapter 5 by suggesting that instructors should think about design through the lens of constructing a comprehensive learning environment in which teaching and learning will take place. I have started to work through the various components of a learning environment, focusing particularly on how the digital age affects the way we need to look at some of these components.

I started by looking at how the characteristics of our learners are changing, and followed that by examining how our perspectives on content are being influenced by the digital age, how both intellectual and practical skills can be developed to meet the needs of a digital age, and the importance of learner support within the learning environment. In subsequent posts, I will cover resources and assessment respectively.

In this post, I examine how resources can and do influence the learning environment and ultimately the design of teaching and learning.

Resources

Resources 2Resources available to teachers/instructors and the learners are a critical component of an effective learning environment. As in the case of learner characteristics, an instructor may not have a lot of control over the resources available to him or her, but resources (or the lack of them) will impact a great deal on the design of teaching. Fighting for appropriate resources is often one of the most challenging tasks for many teachers and instructors. At the same time, of all the components, resources reflect some of the greatest changes resulting from a digital age.

Teaching assistance

I define teaching assistance as people such as adjunct or sessional instructors, teaching assistants, librarians, and technical support staff, including instructional designers, media producers and IT technical support. An institution may have policies or guidelines about how many support staff an instructor can have for a set number of students.

It is important to think about the best way to use supporting staff. In universities, the tendency is to chop a large class into sections, with each section with its own sessional instructor or teaching assistant, which then operate relatively independently, with often large differences in the quality of the teaching in different sections, depending on the experience of the instructor.

However, new technologies enable the teaching to be organised differently and more consistently. For instance, a senior professor may determine the overall curriculum and assessment strategy, and working with an instructional designer, provide the overall design of a course. Sessionals and/or teaching assistants then are hired to deliver the course either face-to-face or online or more often a mix of both, under the supervision of the senior professor (see the National Center for Academic Transformation for examples). Flipped classrooms are another way to organise resources differently (see Blended Learning in Introductory Psychology as an example.)

Furthermore, online learning may bring in more revenues through government grants for extra students and/or direct tuition revenue, so there may be economies of scale which would enable the institution to hire more sessionals from the extra revenues generated by the additional online students. Indeed, there are now examples of fully online masters programs more than covering the full cost, including the hiring of research professors to teach the program, from tuition revenues alone (the University of British Columbia’s online Master in Educational Technology is one example.) Thus design can influence resources, as well as the other way round.

Facilities

This refers primarily to physical facilities available to an instructor and students, such as classrooms, labs, and the library. These may provide constraints on the teaching, because for example the physical set-up of a lecture hall or classroom may limit opportunities for discussion or project work, or an instructor may be forced to organise the teaching around three hours of lecturing and six hours of labs per week, to ‘fit’ with broader institutional requirements for classroom allocations (see How Online Learning is Going to Affect Classroom Design regarding attempts to re-design classrooms for the digital age.)

Online learning can free instructors and students from such rigid physical constraints, but there is still a need for structure and organization of units or modules of teaching, even or especially when teaching online (see Is content still important in a digital age?).

Technology

The development of new technologies, and especially learning management systems, lecture capture, and social media,have radical implications for the design of teaching and learning. This will be discussed in much more depth in Chapter 7, but for the purpose of describing an effective learning environment, the technologies available to an instructor can contribute immensely to creating interactive and engaging learning environments for students. However, it is important to emphasise that technology is just one component within any effective learning environment, and needs to be balanced and integrated with all the other components.

The instructor’s time

The greatest and most precious resource of all! Building an effective learning environment is an iterative process, but in the end, the teaching design, and to some extent the learning environment as a whole, will be dependent on the time available from the instructor (and his or her team) for teaching. The less time available, the more restrictive the learning environment is likely to be, unless the instructor’s time is very carefully managed. Again, though, we shall see in Chapter 7 that good design takes into account the time available for teaching.

Resources, class size and control

Nothing drives an instructor to distraction more than trying to juggle with what are perceived as inadequate resources. Certainly, if a teacher or instructor is allocated a class of 200 students, with no additional teaching support, and an expectation that the class will be taught as a unit with six one hour lectures a week allocated to a large lecture hall, then the instructor is going to have difficulty creating a rich and effective learning environment, because the lack of resources limits the options. On the other hand, an instructor with 30 students, access to a wide range of technology, freedom to organise and structure the curriculum, and with support from an instructional designer and a web designer, has the luxury of exploring a range of different designs and possible learning environments.

Nevertheless it is probably when resources are most scarce that the most creativity is needed to break out of traditional teaching models. New technology, if properly used and available, does enable even large classes with otherwise few resources to be designed with a relatively rich learning environment. This will be explored in more depth in the next chapter. At the same time, expectations need to be realistic. Providing adequate learner support with an instructor:student ratio of 1:200 will always be a challenge. Improvements are possible through re-design – but not miracles. (For more on increasing productivity through online teaching, see Productivity and Online Learning Redux.)

Over to you

Your views, comments and criticisms are always welcome. In particular:

  • are there other resources that influence the design of an effective learning environment that I should have included?
  • Winston Churchill once said ‘We shape our buildings and in turn our buildings shape us.’ To what extent do you think online learning can free us of some of the constraints that buildings impose on the design of teaching and learning? What new constraints does online learning bring in terms of design?
  • how do you feel about the whole issue of teaching assistance? I have grave reservations myself about the use of students as teaching assistants in universities, in terms of the quality of the teaching. I also believe that sessionals and adjunct instructors are badly treated in terms of how they are managed. In British Columbia we have had two Supreme Court cases and a major teachers’ strike over class size and composition, and in particular how much help school teachers should receive for coping with students with learning disabilities. But by bringing in less qualified (and cheaper) support for instructors, do we strengthen or weaken the learning environment for students?

Or any other comments on resources as a critical component of a learning environment, please!

Next up

Assessment as a key component of an effective learning environment.

 

 

Developing intellectual and practical skills in a digital age

Listen with webReader

 

Skills 2

The story so far

Chapter 5 of my open textbook, ‘Teaching in a Digital Age’ is about the design of teaching and learning, which I am currently writing and publishing as I go.

I started Chapter 5 by suggesting that instructors should think about design through the lens of constructing a comprehensive learning environment in which teaching and learning will take place. I have started to work through the various components of a learning environment, focusing particularly on how the digital age affects the way we need to look at some of these components.

I started by looking at how the characteristics of our learners are changing, and followed that by examining how our perspectives on content are being influenced by the digital age. In this post, I look at how both intellectual and practical skills can be developed to meet the needs of a digital age. The following posts will do the same for learner support, resources and assessment respectively.

This will then lead to a discussion of different models for designing teaching and learning. These models aim to provide a structure for and integration of these various components of a learning environment.

Scenario: Developing historical thinking

© Wenxue City: China During the Early Days of the Reform

© Wenxue City: China During the Early Days of the Reform

Ralph Goodyear is a professor of history in a public Tier 1 research university in the central United States. He has a class of 120 undergraduate students taking HIST 305, ‘Historiography’.

For the first three weeks of the course, Goodyear had recorded a series of short 15 minute video lectures that covered the following topics/content:

  • the various sources used by historians (e.g. earlier writings, empirical records including registries of birth, marriage and death, eye witness accounts, artifacts such as paintings, photographs, and physical evidence such as ruins.)
  • the themes around which historical analysis tend to be written,
  • some of the techniques used by historians, such as narrative, analysis and interpretation
  • three different positions or theories about history (objectivist, marxist, post modernist).

Students downloaded the videos according to a schedule suggested by Goodyear. Students attended two one hour classes a week, where specific topics covered in the videos were discussed. Students also had an online discussion forum in the course space on the university’s learning management system, where Goodyear had posted similar topics for discussion. Students were expected to make at least one substantive contribution to each online topic for which they received a grade that went towards their final grade.

Students also had to read a major textbook on historiography over this three week period.

In the fourth week, he divided the class into twelve groups of six, and asked each group to research the history of any city outside the United States over the last 50 years or so. They could use whatever sources they could find, including online sources such as newspaper reports, images, research publications, and so on, as well as the university’s own library collection. In writing their report, they had to do the following:

  • pick a particular theme that covered the 50 years and write a narrative based around the theme
  • identify the sources they finally used in their report, and discuss why they selected some sources and dismissed others
  • compare their approach to the three positions covered in the lectures
  • post their report in the form of an online e-portfolio in the course space on the university’s learning management system

They had five weeks to do this.

The last three weeks of the course were devoted to presentations by each of the groups, with comments, discussion and questions, both in class and online (the in class presentations were recorded and made available online). At the end of the course, students assigned grades to each of the other groups’ work. Goodyear took these student gradings into consideration, but reserved the right to adjust the grades, with an explanation of why he did the adjustment. Goodyear also gave each student an individual grade, based on both their group’s grade, and their personal contribution to the online and class discussions.

Goodyear commented that he was surprised and delighted at the quality of the students’ work. He said: ‘What I liked was that the students weren’t learning about history; they were doing it.’

Based on an actual case, but with some embellishments.

Skills in a digital age

In Chapter 1, Section 1.4, I listed some of the skills that graduates need in a digital age, and argued that this requires a greater focus on developing such skills, at all levels of education, but particularly at a post-secondary level, where the focus is often on specialised content. Although skills such as critical thinking, problem solving and creative thinking have always been valued in higher education, the identification and development of such skills is often implicit and almost accidental, as if students will somehow pick up these skills from observing faculty themselves demonstrating such skills or through some form of osmosis resulting from the study of content. I also pointed out in the same section, though, that there is substantial research on skills development but the knowledge deriving from such research is at best applied haphazardly, if at all, to the development of intellectual skills.

Furthermore the skills required in a digital age are broader and more wide ranging than the abstract academic skills traditionally developed in higher education. For instance, they need to be grounded just as much in digital communications media as in traditional writing or lecturing, and include the development of digital competence and expertise within a subject domain, as well as skills such as independent learning and knowledge management. These are not so much new skills as a different emphasis, focus or direction.

It is somewhat artificial to separate content from skills, because content is the fuel that drives the development of intellectual skills. At the same time, in more traditionally vocational training, we see the reverse trend in a digital age, with much more focus on developing high level conceptual thinking as well as manual skills development. My aim here is not to downplay the importance of content, but to ensure that skills development receives as much focus and attention from instructors, and that we approach intellectual skills development in the same rigorous and explicit way as apprentices are trained in manual skills.

Setting goals for skills development

Thus a critical step is to be explicit about what skills a particular course or program is trying to develop, and to define these goals in such a way that they can be implemented and assessed. In other words it is not enough to say that a course aims to develop critical thinking, but to state clearly what this would look like in the context of the particular course or content area, in ways that are clear to students. In particular the ‘skills’ goals should be capable of assessment and students should be aware of the criteria or rubrics that will be used for assessment.

Thinking activities

A skill is not binary, in the sense that you either have it or you don’t. There is a tendency to talk about skills and competencies in terms of novice, intermediate, expert, and master, but in reality skills require constant practice and application and there is, at least with regard to intellectual skills, no final destination. So it is critically important when designing a course or program to design activities that require students to develop, practice and apply thinking skills on a continuous basis, preferably in a way that starts with small steps and leads eventually to larger ones. There are many ways in which this can be done, such as written assignments, project work, and focused discussion, but these thinking activities need to be thought about, planned and implemented on a consistent basis by the instructor.

Practical activities

It is a given in vocational programs that students need lots of practical activities to develop their manual skills. This though is equally true for intellectual skills. Students need to be able to demonstrate where they are along the road to mastery, get feedback on it, and retry as a result. This means doing work that enables them to practice specific skills.

In the scenario above, students had to cover and understand the essential content in the first three weeks, do research in a group, develop an agreed project report, in the form of an e-portfolio, share it with other students and the instructor for comments, feedback and assessment, and present their report orally and online. Ideally, they will have the opportunity to carry over many of these skills into other courses where the skills can be further refined and developed. Thus, with skills development, a longer term horizon than a single course will be necessary, so integrated program as well as course planning is important.

Discussion as a tool for developing intellectual skills

Discussion is a very important tool for developing thinking skills. However, not any kind of discussion. It was argued in Chapter 2 that academic knowledge requires a different kind of thinking to everyday thinking. It usually requires students to see the world differently, in terms of underlying principles, abstractions and ideas. Thus discussion needs to be carefully managed by the instructor, so that it focuses on the development of skills in thinking that are integral to the area of study. This requires the instructor to plan, structure and support discussion within the class, keeping the discussions in focus, and providing opportunities to demonstrate how experts in the field approach topics under discussion, and comparing students’ efforts.

Figure 5.3: Online threaded discussion forums provide students with opportunities for developing intellectual skills, but the instructor needs to design and manage such forums carefully for this to happen

Figure 5.3: Online threaded discussion forums provide students with opportunities for developing intellectual skills, but the instructor needs to design and manage such forums carefully for this to happen

In conclusion

There are many opportunities in even the most academic courses to develop intellectual and practical skills that will carry over into work and life activities in a digital age, without corrupting the values or standards of academia. Even in vocational courses, students need opportunities to practice intellectual or conceptual skills such as problem-solving, communication skills, and collaborative learning. However, this won’t happen merely through the delivery of content. Instructors need to:

  • think carefully about exactly what skills their students need,
  • how this fits with the nature of the subject matter,
  • the kind of activities that will allow students to develop and improve their intellectual skills, and
  • how to give feedback and to assess those skills, within the time and resources available.

This is a very brief discussion of how and why skills development should be an integral part of any learning environment. We will be discussing skills and skill development in more depth in later chapters.

Over to you

Your views, comments and criticisms are always welcome. In particular:

  • how does the history scenario work for you? Does it demonstrate adequately the points I’m making about skills development?
  • are the skills being developed by students in the history scenario relevant to a digital age?
  • is this post likely to change the way you think about teaching your subject, or do you already cover skills development adequately? If you feel you do cover skills development well, does your approach differ from mine?

Love to hear from you.

Next up

Learner support in a digital age

 

Kuali Foundation goes commercial

Listen with webReader
"No, you idiot, Kuali, not Koalas" 'But isn't kuali a Malaysian way of cooking?"

“No, you idiot, Kuali, not Koalas” ‘But isn’t kuali a Malaysian way of cooking?”

Straumsheim, C. (2014) Kuali Foundation: If you can’t beat them….., Inside Higher Education, August 25

While there are several providers of open source learning management systems for education, Kuali is the only provider of free, open source administrative software specifically built for higher education. In a blog post on August 22, it announced that while its software will still continue to be developed, open source and freely available, it will be creating a commercial company to provide for profit commercial services, such as hosting and contracted software development.

What is Kuali?

Kuali started as a consortium of mainly U.S. research universities which paid to join the Kuali Foundation, with the aim of developing free administrative software software systems designed specifically to meet the needs of higher education/post-secondary institutions.

What does Kuali do?

So far it has developed the following software systems:

How is it doing?

So far nearly 60 HE institutions are using Kuali products. However,  each product is at a different stage of development/usefulness. The financial system is the most advanced and most stabilized.

Why does it matter?

Although the days when Peoplesoft nearly bankrupted several major HE institutions are now long gone, commercial administrative systems such as Oracle and SAS are extremely expensive, designed primarily for a business rather than an educational environment, and as a consequence are often financially risky when it comes to adaptation and implementation within a higher education context. The development of administrative systems for higher education by higher education is a worthy goal, if it can be accomplished.

The ‘if’ though is still in some doubt. The financial system seems to be a success, the Student system is described as a ‘monster’ development project, and the HR system lacks enough investment. So Kuali as a whole is still very much a work in progress.

What are the changes? How is Kuali 2.0 different from the Kuali Foundation?

Kuali is now essentially a for-profit company, rather than a community consortium, although its governance is actually more complex than that. Universities and colleges paid to join the Foundation and contributed investment towards product development. The Foundation will continue to exist but members will not have votes or shares in the new company, although members can continue to contribute to projects that they want done. Other sources of revenue will come from charging for software as a service for cloud-based services.

Comment

I’m not in anyway involved with Kuali, so it is difficult to give an informed comment. I thought it was a good idea when it started, but making a consortium approach to sustainable software development and services work is a major challenge. It requires dedication, goodwill, and continuity from a large number of institutions. In these circumstances, any benefits for the participating organizations need to direct and substantive.

Changing it to a commercial organization is a major disruption to this model. In particular, even if the same people are involved in the investment in product development, governance and operation, it radically changes the culture of the organization. I’m not a governance expert, but I don’t understand why full members who invest substantially in product development don’t have shares or voting rights in the board.

I do hope it succeeds in its goal of providing reliable, sustainable open source solutions for administrative software for HE institutions. I wouldn’t bet my own money on it now, though.

For more on Kuali, see:

A student information system monopoly?

Open source software for research administration

Open source software for administrative systems

 

Learning theories and online learning

Listen with webReader

Figure 3.3. Adults learning in groups in a constructivist manner - and assisted by technology

Figure 3.3. Adults learning in groups in a constructivist manner – and assisted by technology

Introduction

Chapter 3 of my open textbook on ‘Teaching in a Digital Age‘ is about theory and practice in teaching for a digital age, which I am still in the process of writing. I have to admit that I approached writing about learning theories with some dread. In particular I was concerned (in order of dread) that:

  • this will appear incredibly boring/lack originality, because it has been done so many times before by other, more qualified authors (but then those that already know this stuff can easily skip it)
  • I’m not sure that theories of learning actually drive teaching (although surely an understanding of how students learn should do so)
  • I would have to deal with connectivism somehow, and I am certainly not an expert on that topic – but maybe that might be an advantage in bringing it to the attention of people who have previously shown no interest in it, and how it differs from previous theories
  • it could be argued that past learning theories are made irrelevant by digital technologies (and I certainly don’t agree with that point of view.)

In the end, I can’t see how a discussion of learning theories can be avoided. Unless readers of the book have this basic understanding of the different views of learning, they will not be in a good position to make choices, especially regarding the use of technology for teaching and learning. In particular, I see a danger of becoming dogmatic and blinkered by unchallenged assumptions about the nature of learning that results from not exploring alternative theories. But lastly, as Kurt Lewin said, there is nothing more practical than a good theory. A good theory helps us make informed decisions in areas of uncertainty. So, I am sharing here my first draft with you. Please note this is just part of the whole chapter, which also includes the following:

  • Teaching and learning styles
  • Deep vs surface learning.
  • Learner-centered teaching, learner engagement.
  • What we know about skills development
  • Competency based learning.
  • Learning design models
  • learner characteristics: digital natives and digital literacy
  • are we right to fear the use of computers for teaching?
  • Summary of research on teaching.

Also, Chapter 2 discusses the nature of knowledge, and in particular different epistemologies that underpin different theories of learning. However, theories of learning are more than enough to chew on for the moment.

Theories of learning

…there is an impressive body of evidence on how teaching methods and curriculum design affect deep, autonomous, and reflective learning. Yet most faculty are largely ignorant of this scholarship, and instructional practices and curriculum planning are dominated by tradition rather than research evidence. As a result, teaching remains largely didactic, assessment of student work is often trivial, and curricula are more likely to emphasize content coverage than acquisition of lifelong and life-wide learning skills.”

Knapper, 2010, p. 229

“There is nothing so practical as a good theory.” Kurt Lewin, 1951, p. 169

Why an understanding of theories of learning is important

Most teachers in the k-12 sector will be familiar with the main theories of learning, but because instructors in post-secondary education are hired primarily for their subject experience, or research or vocational skills, it is essential to introduce and discuss, if only briefly, these main theories. In practice, even without formal training or knowledge of different theories of learning, all teachers and instructors will approach teaching within one of these main theoretical approaches, whether or not they are aware of the educational jargon surrounding these approaches. Also, as online learning, technology-based teaching, and informal digital networks of learners have evolved, new theories of learning are emerging.

With a knowledge of alternative theoretical approaches, teachers and instructors are in a better position to make choices about how to approach their teaching in ways that will best fit the perceived needs of their students, within the very many different learning contexts that teachers and instructors face. This is particularly important when addressing many of the requirements of learners in a digital age. Furthermore, the choice of or preference for one particular theoretical approach will have major implications for the way that technology is used to support teaching.

In fact, there is a huge amount of literature on theories of learning, and I am aware that the treatment here is cursory, to say the least. Those who would prefer a more detailed introduction to theories of learning could, for an obscene price, purchase Schunk (2011), or for a more reasonable price Harasim (2012). The aim of my book though is not to be comprehensive in terms of in-depth coverage of all learning theories, but to provide a basis on which to suggest and evaluate different ways of teaching to meet the diverse needs of learners in a digital age.

Behaviourism

Although initially developed in the 1920s, behaviourism still dominates approaches to teaching and learning in many places, particularly in the USA.

Behaviourist psychology is an attempt to model the study of human behaviour on the methods of the physical sciences, and therefore concentrates attention on those aspects of behaviour that are capable of direct observation and measurement. At the heart of behaviourism is the idea that certain behavioural responses become associated in a mechanistic and invariant way with specific stimuli. Thus a certain stimulus will evoke a particular response. At its simplest, it may be a purely physiological reflex action, like the contraction of an iris in the eye when stimulated by bright light.

However, most human behaviour is more complex. Nevertheless behaviourists have demonstrated in labs that it is possible to reinforce through reward or punishment the association between any particular stimulus or event and a particular behavioural response. The bond formed between a stimulus and response will depend on the existence of an appropriate means of reinforcement at the time of association between stimulus and response.  This depends on random behaviour (trial and error) being appropriately reinforced as it occurs.

This is essentially the concept of operant conditioning, a principle most clearly developed by Skinner (1968). He showed that pigeons could be trained in quite complex behaviour by rewarding particular, desired responses that might initially occur at random, with appropriate stimuli, such as the provision of food pellets. He also found that a chain of responses could be developed, without the need for intervening stimuli to be present, thus linking an initially remote stimulus with a more complex behaviour. Furthermore, inappropriate or previously learned behaviour could be extinguished by withdrawing reinforcement. Reinforcement in humans can be quite simple, such as immediate feedback for an activity or getting a correct answer to a multiple-choice test.

Skinner and his machine 2

Figure 3.1 YouTube video/film of B.F. Skinner demonstrating his teaching machine, 1954

You can see a fascinating five minute film of B.F. Skinner describing his teaching machine in a 1954 YouTube video, either by clicking on the picture above or at: http://www.youtube.com/watch?v=jTH3ob1IRFo

Underlying a behaviourist approach to teaching is the belief that learning is governed by invariant principles, and these principles are independent of conscious control on the part of the learner. Behaviourists attempt to maintain a high degree of objectivity in the way they view human activity, and they generally reject reference to unmeasurable states, such as feelings, attitudes, and consciousness. Human behaviour is above all seen as predictable and controllable. Behaviourism thus stems from a strongly objectivist epistemological position.

Skinner’s theory of learning provides the underlying theoretical basis for the development of teaching machines, measurable learning objectives, computer-assisted instruction, and multiple choice tests. Behaviourism’s influence is still strong in corporate and military training, and in some areas of science, engineering, and medical training. It can be of particular value for rote learning of facts or standard procedures such as multiplication tables, for dealing with children or adults with limited cognitive ability due to brain disorders, or for compliance with industrial or business standards or processes that are invariant and do not require individual judgement.

Finally, it should be noted that behaviourism, with its emphasis on rewards and punishment as drivers of learning, and on pre-defined and measurable outcomes, is the basis of populist conceptions of learning among many parents, politicians, and, it should be noted, computer scientists interested in automating learning. It is not surprising then that there has also been a tendency until recently to see technology, and in particular computer-aided instruction, as being closely associated with behaviourist approaches to learning, although we shall see that this does not necessarily follow.

Cognitivism

An obvious criticism of behaviourism is that it treats humans as a black box, where inputs into the black box, and outputs from the black box, are known and measurable, but what goes on inside the black box is ignored or not considered of interest. However, humans have the ability for conscious thought, decision-making, emotions, and the ability to express ideas through social discourse, all of which may be highly significant for learning. Thus we will likely get a better understanding of learning if we try to find out what goes on inside the black box. Cognitivists therefore have focused on identifying mental processes – internal and conscious representations of the world – that they consider are essential for human learning. Fontana (1981) summarises the cognitive approach to learning as follows:

The cognitive approach … holds that if we are to understand learning we cannot confine ourselves to observable behaviour, but must also concern ourselves with the learner’s ability mentally to re-organize his psychological field (i.e. his inner world of concepts, memories, etc.) in response to experience. This latter approach therefore lays stress not only on the environment, but upon the way in which the individual interprets and tries to make sense of the environment. It sees the individual not as the somewhat mechanical product of his environment, but as an active agent in the learning process, deliberately trying to process and categorize the stream of information fed into him by the external world.’ (p. 148)

Thus the search for rules, principles or relationships in processing new information, and the search for meaning and consistency in reconciling new information with previous knowledge, are key concepts in cognitive psychology. Cognitive psychology is concerned with identifying and describing mental processes that affect learning, thinking and behaviour, and the conditions that influence those mental processes.

© Agile Development Blog, 2013

© Agile Development Blog, 2013

Figure 3.2: Some of the areas covered by cognitivism, based on Bloom’s taxonomy (1956). Note that this becomes a reductionist exercise, as psychologists delve deeper into each of these cognitive activities to understand the underlying mental processes.

Cognitive approaches to learning cover a very wide range. At one end, the objectivist end, cognitivists consider basic mental processes to be genetic or hard-wired, but can be programmed or modified by external factors, such as new experiences. Early cognitivists in particular were interested in the concept of mind as computer, and more recently brain research has led to a search for linking learning to the development and reinforcement of neural networks in the brain. In terms of practice this concept of mind as computer has led to several technology-based developments in teaching, including:

  • intelligent tutoring systems, a more refined version of teaching machines, based on analysing student responses to questions and redirecting them to the appropriate next steps in learning. Adaptive learning is the latest extension of such developments;
  • artificial intelligence, which seeks to represent in computer software the mental processes used in human learning (which of course if successful would result in computers replacing many human activities – such as teaching, if learning is considered in an objectivist framework.)
  • pre-determined learning outcomes, based on an analysis and development of different kinds of cognitive activities, such as comprehension, analysis, synthesis, and evaluation
  • certain instructional design approaches that attempt to manage the design of teaching to ensure successful achievement of pre-determined learning outcomes or objectives.

On the other hand, many other cognitivists, coming from a more constructivist epistemological perspective, would argue that mental states or even processes are not fixed but constantly evolving as new information is integrated with prior knowledge, and new strategies for seeking meaning are developed by the individual. Thus teachers who place a strong emphasis on learners developing personal meaning through reflection, analysis and construction of knowledge through conscious mental processing would represent much more of a constructivist epistemological position. It is here that the boundaries between cognitivist and constructivist learning begin to break down.

Cognitive approaches to learning, with a focus on comprehension, abstraction, analysis, synthesis, generalization, evaluation, decision-making and creative thinking, seem to fit much better with higher education than behaviourism,  but even in k-12 education, a cognitivist approach would mean for instance focusing on teaching learners how to learn, on developing stronger or new mental processes for future learning, and on developing deeper and constantly changing understanding of concepts and ideas.

Put simply, brains have more plasticity, adaptability and complexity than current computer software programs, and other factors, such as emotion, motivation, self-determination, values, and a wider range of senses, make human learning very different from the way computers operate, at least at the moment. Education would be much better served if computer scientists tried to make software to support learning more reflective of the way human learning operates, rather than trying to fit human learning into the current restrictions of behaviourist computer programming.

Nevertheless, cognitivists have increased our understanding of how humans process and make sense of new information, how we access, interpret, integrate, process, organize and manage knowledge, and have given us a better understanding of the conditions that affect learners’ mental states.

Constructivism

Both behaviourist and some elements of cognitive theories of learning are deterministic, in the sense that behaviour and learning are believed to be rule-based and operate under predictable and constant conditions over which the individual learner has no or little control. However, constructivists emphasise the importance of consciousness, free will and social influences on learning. Carl Rogers (1969) stated that: ‘every individual exists in a continually changing world of experience in which he is the center.’ The external world is interpreted within the context of that private world. The belief that humans are essentially active, free and strive for meaning in personal terms has been around for a long time.

Constructivists argue that individuals consciously strive for meaning to make sense of their environment in terms of past experience and their present state. It is an attempt to create order in their minds out of disorder, to resolve incongruities, and to reconcile external realities with prior experience. The means by which this is done are complex and multi-faceted, from personal reflection, seeking new information, to testing ideas through social contact with others. Problems are resolved, and incongruities sorted out, through strategies such as seeking relationships between what was known and what is new, identifying similarities and differences, and testing hypotheses or assumptions. Reality is always tentative and dynamic.

For many educators, the social context of learning is critical. Ideas are tested not just on the teacher, but with fellow students, friends and colleagues. Furthermore, knowledge is mainly acquired through social processes or institutions that are socially constructed: schools, universities, and increasingly these days, online communities. Thus what is taken to be ‘valued’ knowledge is also socially constructed. Thus knowledge is not just about content, but also values. One set of values are those around the concept of a liberal education. According to this ideology, one of the principal aims of education is that it should develop a critical awareness of the values and ideologies that shape the form of received knowledge. This then suggests a constant probing and criticism of received knowledge.

One consequence of constructivist theory is that each individual is unique, because the interaction of their different experiences, and their search for personal meaning, results in each person being different from anyone else. Thus behaviour is not predictable or deterministic, at least not at the individual level. The key point here is that learning is seen as essentially a social process, requiring communication between learner, teacher and others. This social process cannot effectively be replaced by technology, although technology may facilitate it.

It can be seen that although constructivist approaches can be and have been applied to all fields of knowledge, it is more commonly found in approaches to teaching in the humanities, social sciences, education, and other less quantitative subject areas.

Online collaborative learning

The concurrence of both constructivist approaches to learning and the development of the Internet has led to the development of a particular form of constructivist teaching, originally called computer-mediated communication (CMC), but which has developed into what Harasim (2012) now calls online collaborative learning theory (OCL). She describes OCL as follows (p. 90):

OCL theory provides a model of learning in which students are encouraged and supported to work together to create knowledge: to invent, to explore ways to innovate, and, by so doing, to seek the conceptual knowledge needed to solve problems rather than recite what they think is the right answer. While OCL theory does encourage the learner to be active and engaged, this is not considered to be sufficient for learning or knowledge construction……In the OCL theory, the teacher plays a key role not as a fellow-learner, but as the link to the knowledge community, or state of the art in that discipline. Learning is defined as conceptual change and is key to building knowledge. Learning activity needs to be informed and guided by the norms of the discipline and a discourse process that emphasises conceptual learning and builds knowledge.

This approach to the use of technology for teaching is very different from the more objectivist approaches found in computer-assisted learning, teaching machines, and artificial intelligence applications to education, which primarily aim to use computing to replace at least some of the activities traditionally done by human teachers. With online collaborative learning, the aim is not to replace the teacher, but to use the technology primarily to increase and improve communication between teacher and learners, with a particular approach to the development of learning based on knowledge construction assisted and developed through social discourse. This social discourse furthermore is not random in OCL, but managed in such a way as to ‘scaffold’ learning, by assisting with the construction of knowledge in ways that are guided by the instructor, that reflect the norms or values of the discipline, and that also respect or take into consideration the prior knowledge within the discipline.

Connectivism

Connectivism is a relatively new theory of learning or epistemology (there’s not even agreement about which it is), it is still being refined and developed, and it is currently highly controversial, with many critics. Siemens, Downes and Cormier constructed the first massive open online course (MOOC), Connectivism and Connective Knowledge 2011, partly to explain and partly to model a connectivist approach to learning. More recently, Downes (2014) has spelled out, in a presentation called The MOOC of One, some of the relationships between individual learning, the contribution of individuals to knowledge and its flow, and networks of learners, within a broad interpretation of connectivist theory. In this presentation Downes sets out some design principles for  connectivist ‘courses’ or cMOOCs, such as:

  • learner autonomy, in terms of choice of content and how they choose to learn
  • openness, in terms of  access to the course, content, activities and methods of assessment
  • diversity: varied content, individual perspectives and multiple tools, especially for networking learners and creating opportunities for dialogue and discussio
  • interactivity: ‘massive’ communication between learners and co-operative learning, resulting in emergent knowledge
Figure 2.1: A map of connectivism, © Stephen Downes, 2011 (accessed via pkab.wordpress.com)

Figure 2.1: A map of connectivism, © Stephen Downes, 2011 (accessed via pkab.wordpress.com)

Connectivists such as Siemens and Downes tend to be somewhat vague about the role of teachers or instructors, as the focus of connectivism is more on individual participants, networks and the flow of information and the new forms of knowledge that result.. The main purpose of a teacher appears to be to provide the initial learning environment and context that brings learners together, and  to help learners construct their own personal learning environments to enable them to connect to ‘successful’ networks, with the assumption that learning will automatically occur as a result, through exposure to the flow of information and the individual’s autonomous reflection on its meaning. There is no need for formal institutions to support this kind of learning, especially since such learning often depends heavily on social media readily available to all participants.

There are numerous criticisms of the connectivist approach to teaching and learning, which include:

  • there is no control on the quality of content, or on contributions from participants;
  • assessment strategies, such as peer assessment, are primitive and unreliable, thus making reliable or valid recognition of achievement more difficult;
  • the kinds of learning that take place in connectivist MOOCs or courses are not necessarily academic, in the sense of meeting the requirements for academic knowledge, as defined in Chapter 2;
  • many participants struggle with the lack of structure and are overwhelmed by the volume of content generated by other learners;
  • most students need a high level of explicit support in learning from an ‘expert’ teacher and this is lacking in connectivist courses
  • this kind of learning requires learners already to have at least some level of more formal or traditional education before they participate if they are to fully benefit from this kind of learning experience (and there is substantial evidence that MOOC participants tend to have an already high level of post-secondary education).
  • thus this kind of learning is more appropriate for non-formal learning or communities of practice than for formal education.

Some of these criticisms may be overcome as practice improves, as new tools for assessment, and for organizing co-operative and collaborative work with massive numbers, are developed, and as more experience is gained. More importantly, connectivism is really the first theoretical attempt to radically re-examine the implications for learning of the Internet and the explosion of new communications technologies.

Conclusion

Different theories of learning reflect different positions on the nature of knowledge. With the possible exception of connectivism, there is some form of empirical evidence to support each of the theories of learning outlined here.

However, while the theories suggest different ways in which all people learn, they do not automatically tell teachers or instructors how to teach. Indeed, theories of behaviourism, cognitivism and constructivism were all developed outside of education, in experimental labs, psychology , neuroscience, and psychotherapy respectively. Educators have had to work out how to move from the theoretical position to the practical one of applying these theories within an educational experience. In other words, they have had to develop teaching methods that build on such learning theories. The next section of the book examines a range of teaching methods that have been developed, their epistemological roots, and their implications for teaching in a digital age.

Over to you

Your feedback on this will be invaluable. In particular:

  • are theories of learning still relevant in a digital age? Is it important to discuss these?
  • is the description of the various theories accurate and useful; if not, what should be changed?
  • are there important theories or theoretical positions that have been missed?

References

Bloom, B., Englehart, M., Furst, E., Hill, W. and Krathwohl, D. (1956) Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain, Longmans Green, New York, 1956

Downes, S. (2014) The MOOC of One, Stephen’s Web, March 10

Fontana, D. (1981) Psychology for Teachers London: Macmillan/British Psychological Society

Harasim, L. (2012) Learning Theory and Online Technologies New York/London: Routledge

Knapper, C. (2010) ‘Changing Teaching Practice: Barriers and Strategies’ in Christensen Hughes, J. and Mighty, J. eds. Taking Stock: Research on Teaching and Learning in Higher Education Toronto ON: McGill-Queen’s University Press

Lewin, K. (1951) Field theory in social science; selected theoretical papers. D. Cartwright (ed.). New York: Harper & Row.

Rogers, C. (1969) Freedom to Learn Columbus, OH: Charles E. Merrill Publishing Co.

Schunk, D. (2011) Learning Theories: An Educational Perspective (6th edition) New York: Pearson

Why lectures are dead (or soon will be)

Listen with webReader
Artist: Laurentius de Voltolina; Liber ethicorum des Henricus de Alemannia; Kupferstichkabinett SMPK, Berlin/Staatliche Museen Preussiischer Kulturbesitz, Min. 1233

Artist: Laurentius de Voltolina;
Liber ethicorum des Henricus de Alemannia; Kupferstichkabinett SMPK, Berlin/Staatliche Museen Preussiischer Kulturbesitz, Min. 1233

As part of my open textbook on Teaching in a Digital Age, I am working my way through theories of learning and methods of teaching. I will post shortly my initial draft on theories of learning and their relevance for a digital age. In this post I want to discuss the lecture and its relevance for a digital age. Comments as always are more than welcome.

Definition:

‘[Lectures] are more or less continuous expositions by a speaker who wants the audience to learn something.’

Bligh, 2000

History

Lectures go back as far as ancient Greece and Roman times, and certainly from at least the start of the European university, in the 13th century. The term ‘lecture’ comes from the Latin to read. This was because in the 13th century, most books were extremely rare. They were painstakingly handcrafted and illustrated by monks, often from fragments or collections of earlier and exceedingly rare and valuable scrolls remaining from more than 1,000 years earlier from ancient Greek or Roman times, or were translated from Arabic sources, as much documentation was destroyed in Europe during the Dark Ages following the fall of the Roman empire. As a result, a university would often have only one copy of a book, and it may have been the only copy available in the world. The library and its collection therefore became critical to the reputation of a university, and professors had to borrow the only text from the library and literally read from it to the students, who dutifully wrote down their own version of the lecture.

The illustration at the head of this post from a thirteenth-century manuscript shows Henry of Germany delivering a lecture to university students in Bologna, Italy, in 1233. What is striking is how similar the whole context is to lectures today, with students taking notes, some talking at the back, and one clearly asleep. Certainly, if Rip Van Winkle awoke in a modern lecture theatre from his 800 years of sleeping, he would know exactly where he was and what was happening.

Lectures themselves belong to an even longer oral tradition of learning, where knowledge is passed on by word of mouth from one generation to the next. In such contexts, accuracy and authority (or power in controlling access to knowledge) are critical for ‘accepted’ knowledge to be successfully transmitted. Thus accurate memory, repetition and a reference to authoritative sources become exceedingly important in terms of validating the information transmitted. The great sagas of the ancient Greeks and much later, of the Vikings, and even today,the oral myths and legends of many indigenous communities, are examples of the power of the oral transmission of knowledge.

Nevertheless, the lecture format has been questioned for many years. Samuel Johnson (1709-1784) long ago produced his own straightforward critique of lectures:

People have nowadays…got a strange opinion that everything should be taught by lectures. Now, I cannot see that lectures can do as much good as reading the books from which the lectures are taken…Lectures were once useful, but now, when all can read, and books are so numerous, lectures are unnecessary.’

What is remarkable is that even after the invention of the printing press, radio, television, and the Internet, the lecture, characterised by the authoritative instructor talking to a group of students, still remains the dominant methodology for teaching in many institutions, even in a digital age, where information is available at a click of a button.

It could be argued that anything that has lasted this long must have something going for it. On the other hand, we need to question whether the lecture is still the most appropriate means of teaching, given all the changes that have taken place in recent years, and in particular given the kinds of knowledge and skills needed in a digital age.

What does research tell us about the effectiveness of lectures?

Whatever you may think of Samuel Johnson’s opinion, there has indeed been a great deal of research into the effectiveness of lectures, going back to the 1960s, and continued through until today. The most authoritative analysis of the research on the effectiveness of lectures remains Bligh’s (2000). He summarized a wide range of meta-analyses and studies of the effectiveness of lectures compared with other teaching methods and found consistent results:

  1. The lecture is as effective as other methods for transmitting information (the corollary of course is that other methods – such as video, reading, independent study - are just as effective as lecturing for transmitting information)
  2. Most lectures are not as effective as discussion for promoting thought
  3. Lectures are generally ineffective for changing attitudes or values or for inspiring interest in a subject
  4. Lectures are relatively ineffective for teaching behavioural skills.

It should be noted that are are also many studies that suggest that it makes little difference to the learning effectiveness of a lecture if it is live (with the lecturer and the audience together at the same place and time), if it is transmitted in real time across distance (such as via a webcast or video-conference) or is viewed once on a recording as a continuous event. Thus merely by transmitting a MOOC in the form of a video lecture makes it no more or less effective in terms of an individual’s learning than if it was delivered in a classroom (although of course the MOOC will reach a lot more learners). Thus the medium of transmission makes no difference to an individual’s learning if the form of the lecture remains the same.

However, my research colleagues and I at the U.K. Open University, as early as 1984, established that making a lecture available in a recorded format (either on video or audio) increased the learning effectiveness, because it increased students’ time on task, by enabling them to review and repeat the material. We also found that recorded video or audio was even more effective than a recorded lecture if the program was re-designed to break the transmission of information into small chunks, and if the stop-start facility of recordings was used to build in student activities and feedback following each chunk of information. Proponents of Coursera-style MOOCs are just beginning to rediscover this thirty years later.

Bligh also examined research on student attention, on memorizing, and on motivation, and concluded (p.56):

We see evidence… once again to suppose that lectures should not be longer than twenty to thirty minutes – at least without techniques to vary stimulation.’

These research studies have shown that in order to understand, analyze, apply, and commit information to long-term memory, the learner must actively engage with the material. In order for a lecture to be effective, it must include activities that compel the student to mentally manipulate the information. Many lecturers of course do this, by stopping and asking for comments or questions throughout the lecture – but many do not.

Again, although these findings have been available for a long time, and You Tube videos now last approximately eight minutes and TED talks 20 minutes at a maximum, teaching in many educational institutions is still organized around a standard 50 minute lecture session, with, if students are lucky, a few minutes at the end for questions or discussion. Indeed in some institutions it is not uncommon to find even longer lecture sessions.

There are two important conclusions from the research:

1. Even for the sole purpose for which lectures may be effective – the transmission of information – the 50 minute lecture needs to be well organized, with frequent opportunities for student questions and discussion. (Bligh provides excellent suggestions on how to do this in his book.)

2. For all other important learning activities, such as developing critical thinking, deep understanding, and application of knowledge – the kind of skills needed in a digital age – lectures are ineffective. Other forms of teaching and learning – such as opportunities for discussion and student activities – are necessary.

Does new technology make lectures more relevant?

Over the years, institutions have made massive investments in ‘technologising’ the lecture. Powerpoint presentations, multiple projectors and screens, clickers for recording student responses, even ‘back-chat’ channels on Twitter, enabling students to comment on a lecture – or more often, the lecturer – in real time (surely the worse form of torture), have all been tried. Students have been asked to bring tablets or lap-tops to class, and universities in particular have invested millions of dollars in state of the art lecture theatres.

Nevertheless, all this is just lipstick on a pig. The essence of a lecture remains the transmission of information, all of which is now readily and, in most cases, freely available in other media and in more learner-friendly formats.

I worked in a college where in one program all students had to bring laptops to class. At least in these classes, there were some activities to do related to the lecture that required the students to use the laptops during class time. However, in most classes this took less than 25 per cent of the lesson time. Most of the other time, students were talked at, and as a result used their laptops for other, mainly non-academic activities, especially playing online poker.

Faculty often complain about students use of technology such as mobile phones or tablets, for ‘non-relevant’ multitasking in class, but this misses the point. If most students have mobile phones or laptops, why are they still having physically to come to a lecture hall? Why can’t they get a podcast of the lecture? Second, if they are coming, why are the lecturers not requiring them to use their mobile phones, tablets, or laptops for study? Why not break them into small groups and get them to do some online research then come back with group answers to share with the rest of the class? If lectures are to be offered, the aim should be to make the lecture engaging in its own right, so the students are not distracted by their online activity. If lecturers can’t do this, perhaps they should give up lecturing and find more interactive ways of engaging students.

Is there then no role for lectures in a digital age?

I do believe that lectures have their uses. As an example, I have attended an inaugural lecture for a newly appointed research professor. In this lecture, he summarised all the research he and his team had done, resulting in treatments for several cancers and other diseases. This was a public lecture, so he had to satisfy not only other leading researchers in the area, but also a lay public with often no science background. He did this by using excellent visuals and analogies. The lecture was followed by a small wine and cheese reception for the audience.

The lecture worked for several reasons:

  • first of all, it was a celebratory occasion bring together family, colleagues and friends.
  • second, it was an opportunity to pull together nearly 20 years of research into a single, coherent narrative or story.
  • third, the lecture was well supported by an appropriate use of graphics and video.
  • lastly, he put a great deal of work into preparing this lecture and thinking about who would be in the audience – much more preparation than would be the case if this was just one of many lectures in a course.

More importantly, though, that lecture is now publicly available via You Tube for anyone to view.

McKeachie and Svinicki (2006, p. 58) believe that lecturing is best used for:

  • providing up-to-date material that can’t be found in one source
  • summarizing material found in a variety of sources.
  • adapting material to the interests of a particular group.
  • initially helping students discover key concepts, principles or ideas
  • modelling expert thinking.

The last point is important. Faculty often argue that the real value of a lecture is to model for students how the faculty member, as an expert, approaches a topic or problem. Thus the important point of the lecture is not the transmission of content (facts, principles, ideas), which the students could get from just reading, but an expert way of thinking about the topic. The trouble with this argument for lectures is three-fold:

  • students are rarely aware that this is the purpose of the lecture, and therefore focus on memorizing the content, rather than the ‘modelling’ of expert thinking
  • faculty themselves are not explicit about how they are doing the modelling (or fail to offer other ways in which modelling could be used, so students can compare and contrast)
  • students get no practice themselves in modelling this skill, even if they are aware of the modelling.

So, yes, there are a few occasions when lectures work very well. But they should not be the default model for regular teaching. There are much better ways to teach that will result in better learning over the length of a course or program, and that lectures, whether live, or on MOOCs, YouTube videos or TED talks, are a poor way to prepare learners for a digital age.

Why are lectures still the main form of educational delivery?

Given all of the above, some explanation needs to be offered for the persistence of the lecture into the 21st century. Here are my suggestions

  1. in fact, in many areas of education, the lecture has been replaced, particularly in many elementary or primary schools (although parents often are unhappy about this, because a lecture represents their understanding of what teaching is all about); online learning (usually avoiding recorded lectures) and open education is also increasing more rapidly than classroom based learning
  2. a false assumption that the lecture is economically efficient. It is true it is efficient in the use of an instructor’s time, in that increasing student numbers in a lecture is no more work for the instructor for each student added to a class. However, efficiency needs to take into account output. If the output is to transfer information both less effectively per individual learner and in terms of providing the knowledge and skills in demand, then large lecture classes are a false efficiency;
  3. architectural inertia: a huge investment has been made by institutions in facilities that support the lecture model. What is to happen to all that real estate if it is not used?
  4. the Carnegie unit of teaching, which is based on a notion of one hour per week of classroom time per credit over a 13 week period. It is easy then to divide a three credit course into 39 one hour lectures over which the curriculum for the course must be covered. It is on this basis that teaching load and resources are decided.
  5. faculty in post-secondary education have no other model for teaching. This is the model they are used to, and because appointment is based on training in research or work experience, and not on qualifications in teaching, they have no knowledge of how students learn or confidence or experience in other methods of teaching.
  6. many experts prefer the oral tradition of teaching and learning, because it enhances their status as an expert and source of knowledge: being allowed an hour of other people’s time to hear your ideas without major interruption is very satisfying on a personal level (at least for me).

Is there a future for lectures in a digital age?

That depends on how far into the future one wants to look. Given the inertia in the system, I suspect that lectures will still predominate for another ten years, but after that, in most institutions, courses based on three lectures a week over 13 weeks will have disappeared. There are several reasons for this.

  • the first is that all content can be easily digitalized and made available on demand at very low cost.
  • second, institutions will be making greater use of dynamic video (not talking heads) for demonstration, simulations, animations, etc. Thus most content modules will be multi-media.
  • third, open textbooks incorporating multi media components and student activities will provide the content, organization and interpretation that are the rationale for most lectures.
  • lastly, and most significantly, the priority for teaching will have changed from information transmission and organization to knowledge management, where students have the responsibility for finding, analyzing, evaluating, sharing and applying knowledge, under the direction of a skilled subject expert. Project-based learning, collaborative learning and situated or experiential learning will become much more widely prevalent. Also many instructors will prefer to use the time they would have spent on a series of  lectures in providing more direct, individual and group learner support, thus bringing them into closer contact with learners.

This does not mean that lectures will disappear altogether, but they will be special events, and probably multi-media, synchronously and asynchronously delivered. Special events might include a professor’s summary of his latest research, the introduction to a course, a point mid-way through a course for taking stock and dealing with common difficulties, or the wrap-up to a course. A lecture will provide a chance for an instructor to makes themselves known, to impart their interests and enthusiasm, and to motivate learners, but this will be just one, relatively small, but important component of a much broader learning experience for students.

 In the meantime, institutions should be looking at their building plans, and deciding if the money they are thinking of in terms of classrooms and lecture theatres may not be better spent on digitizing the curriculum and making it openly available.

References

Bates, A. (1985) Broadcasting in Education: An Evaluation. London: Constable

Bligh, D. (2000) What’s the Use of Lectures? San Francisco: Jossey-Bass

Boswell, James (1986), Hibbert, Christopher, ed., The Life of Samuel Johnson, New York: Penguin Classics, ISBN 0-14-043116-0.

McKeachie, W. and Svinicki, M. (2006) McKeachie’s Teaching Tips: Strategies, Research, and Theory for College and University Teachers, 13th Edition Independence KY: Cengage