Reynard, R. (2009) Can We Gauge Technology’s Impact on Learning Outcomes? Campus Technology, March 25
A thoughtful essay in a long line that discusses the reasons why it is so difficult to show improved learning outcomes from the use of technology. For more academic research-based studies on this topic, see the following:
Clark, R. (1983) ‘Reconsidering research on learning from media’ Review of Educational Research, Vol. 53, pp. 445-459
Kozma, R. (1994) ‘Will Media Influence Learning? Reframing the Debate’, Educational Technology Research and Development, Vol. 42, No. 2, pp. 7-19
Russell, T. L. (1999) The No Significant Difference Phenomenon Raleigh, NC: North Carolina State University, Office of Instructional Telecommunication
I have written several times about this issue: see for example, pp. 70-73 in Effective Teaching with Technology in Higher Education: Foundations for Success and Chapter 3 of Technology, E-Learning and Distance Education
In general, we tend to set assessment questions that place a heavy emphasis on understanding and remembering facts, theories, arguments, and descriptive processes (content), and less on measuring critical thinking, creative thinking, application of knowledge to specific tasks, and ability to create in different media (skills). The use of media (text, audio, video, speech, graphics) within different technologies can help facilitate both the presentation of content and particularly the development of skills. If we do not assess the learning outcomes that we are often using technologies to develop, and focus only on those that are specific to text or speech-based learning, then we will find it difficult to see the impact of using learning technologies.
The important point is that when we decide to use a technology, whether it is a laptop in class or an online discussion forum, we need also to ask the question: what learning outcomes are we trying to achieve and is our choice of technology appropriate for helping learners achieve these outcomes? To get the answer to this question, though, you need to read particularly Technology, E-Learning and Distance Education