Tamim, R. et al. (2011) What Forty Years of Research Says About the Impact of Technology on Learning: A Second-Order Meta-Analysis and Validation Study Review of Educational Research, Vol. 81, No. 1
This study found that looking at studies over 40 years, there is a slight tendency for students who study with technology to do better than students who study without technology.
Don’t get too excited about this. This is a ‘second-order meta-analysis.’ A meta-analysis collects a wide range of research publications on the topic, aggregates the data then runs statistical tests to see if across all the studies there are consistent statistically valid results, even if individual studies didn’t find clear differences or produced mixed or contradictory results. Meta-analyses sometimes find differences ‘missed’ in the original studies, because the larger the sample, the smaller the differences needed to be statistically significant. The results of the meta-analysis depend heavily on the criteria used to chose the original studies.
Note though that this paper is a meta-analysis of previous meta-analyses, so that although the aggregated sample size of participants is very large, the study is now two levels of analysis away from the original research. You begin to wonder what this really means, especially since that even at the second-order level of analysis, the measured difference between ‘effect’ (studying with technology) and ‘control’ (studying without technology) is quite weak (0.35 on a range of .00 to 1.0). As the researchers themselves conclude:
‘It is important to note that these average effects must be interpreted cautiously because of the wide variability that surrounds them. We interpret this to mean that other factors, not identified in previous meta-analyses or in this summary, may account for this variability….Thus, it is arguable that it is aspects of the goals of instruction, pedagogy, teacher effectiveness, subject matter, age level, fidelity of technology implementation, and possibly other factors that may represent more powerful influences on effect sizes than the nature of the technology intervention.’
Right on, but, as always, I recommend you read the article in full if you believe the results could be important.