Drachsler, H. et al. (2016) Is Privacy a Show-stopper for Learning Analytics? A Review of Current Issues and Their Solutions Learning Analytics Review, no. 6, January 2016, ISSN: 2057-7494
One of the most interesting sessions for me at last week’s EDEN conference in Budapest was a workshop run by Sally Reynolds of ATiT in Brussels and Dai Griffiths of the University of Bolton, UK. They are both participants in a European Commission project called LACE (Learning Analytics Community Exchange).
The LACE web site states:
LACE partners are passionate about the opportunities afforded by current and future views of learning analytics (LA) and educational data mining (EDM) but we were concerned about missed opportunities and failing to realise value. The project aimed to integrate communities working on LA and EDM from schools, workplace and universities by sharing effective solutions to real problems.
There are a number of reviews and case studies of the use of learning analytics available from the web site, which, if you are interested in (or concerned) about the use of learning analytics, are well worth reading.
The EDEN workshop
The EDEN workshop focused on one of the reviews concerned with issues around ethics and privacy in the use of learning analytics, and in particular the use of big data.
I am reasonably familiar with the use of ‘small’ data for learning analytics, such as the use of institutional student data regarding the students in the courses I am teaching, or the analysis of participation in online discussions, both in quantitative and qualitative terms. I am less familiar with the large-scale use of data and especially how data collected via learning management or MOOC registration systems are or could be used to guide teaching and learning.
However, the focus of the workshop was specifically on ethical and privacy issues, based on the review quoted above, but nevertheless I learned a great deal about learning analytics in general through the workshop.
What is the concern?
This is best stated in the review article:
Once the Pandora’s Box of data availability has been opened, then individuals lose control of the data about them that have been harvested. They are unable to specify who has access to the data, and for what purpose, and may not be confident that the changes to the education system which result from learning analytics will be desirable. More generally, the lack of transparency in data collection and analysis exacerbates the fear of undermining privacy and personal information rights in society beyond the confines of education. The transport of data from one context to another can result in an unfair and unjustified discrimination against an individual.
In the review article, these concerns are exemplified by case studies covering schools, universities and the workplace. These concerns are summarized under the following headings:
- informed consent and transparency in data collection
- location and interpretation of data
- data management and security
- data ownership
- possibility of error
- role of knowing and obligation to act
There are in fact a number of guidelines regarding data collection and use that could be applied to learning analytics, such as the Nuremberg Code on research ethics, the OECD Privacy Framework, (both of which are general), or the JISC code of practice for learning analytics. However, the main challenge is that some proponents of learning analytics want to approach the issue in ways that are radically different from past data collection methods (like my ‘small’ data analysis). In particular they propose using random data collection then subsequently analysing it through data analysis algorithms to identify possible post-hoc applications and interpretations.
It could be argued that educational organizations have always collected data about students, such as registers of attendance, age, address and student grades. However, new technology, such as data trawling and the ability to combine data from completely different sources, as well as automated analysis, completely changes the game, raising the following questions:
- who determines what data is collected and used within a learning management system?
- who ensures the security of student (or instructor) data?
- who controls access to student data?
- who controls how the data is used?
- who owns the data?
In particular, increasingly student (and instructor) data is being accessed, stored and used not just outside an institution, but even outside a particular country, and hence subject to laws (such as the U.S. Patriot Act) that do not apply in the country from which the data was collected.
Recommendations from the LACE working group
The LACE working group has developed an eight point checklist called DELICATE, ‘to support a new learner contract, as the basis for a trusted implementation of Learning Analytics.’
For more on DELICATE see:
Drachsler, H. and Greller, W. (2016) Privacy and Learning Analytics – its a DELICATE issue Heerlen NL: The Open University of the Netherlands
Issues raised in the workshop
First it was pointed out that by today’s standards, most institutional data doesn’t qualify as ‘big data’. In education, what would constitute big data would for example be student information from the whole education system. The strategy would be to collect data about or from all students, then apply analysis that may well result in by-passing or even replacing institutions with alternative services. MOOC platforms are possibly the closest that come to this model, hence their potential for disruption. Nevertheless, even within an institution, it is important to develop policies and practices that take into account ethics and privacy when collecting and using data.
As in many workshops, we were divided into small groups to discuss some of these issues, with a small set of questions to guide the discussion. In my small group of five conference participants, none of the participants was in an institution that had a policy regarding ethics and privacy in the use of learning analytics (or if it existed, they were unaware of it).
There was a concern on our table that increasing amounts of student data around learning was accessible to external organizations (such as LMS software companies and social media organizations such as Facebook). In particular, there was a concern that in reality, many technology decisions, such as choice of an institutional learning platform, were influenced strongly by the CIO, who may not take into sufficient account ethical and privacy concerns when negotiating agreements, or even by students themselves, who are often unaware of the implications of data collection and use by technology providers.
Our table ended by suggesting that every post-secondary institution should establish a small data ethics/privacy committee that would include, if available, someone who is a specialist in data ethics and privacy, and representatives of faculty and students, as well as the CIO, to implement and oversee policy in this area.
This was an excellent workshop that tried to find solutions that combine a balance between the need to track learner behaviour and privacy and ethical issues.
Over to you
Some questions for you:
- is your institution using learning analytics – or considering it
- if so, does your institution have a policy or process for monitoring data ethics and privacy issues?
- is this really a lot of fuss over nothing?
I’d love to hear from you on this.
Centre for Extended Learning (2016) How do we create useful online learning experiences? Waterloo ON: University of Waterloo
The University of Waterloo’s Centre for Extended Learning has combined Peter Morville’s user experience (UX) honeycomb and Richard Mayer’s theory and research on the use of multimedia for learning, to create a well-designed set of guidelines for online course design. This should be compulsory reading/viewing for all instructors moving into online learning.
Thanks to Naza Djafarova, Director, Digital Education Strategies, The Chang School of Continuing Education, Ryerson University, for directing me to this invaluable resource.
Kirby, P. (2016) Degrees of Debt: Funding and finance for undergraduates in Anglophone countries London, UK: The Sutton Trust
The answer, in a comparison between the major anglophone nations of England, Scotland, Wales, Northern Ireland (NI), USA, Canada, Australia and New Zealand (NZ) is clearly: England. Graduates from English universities will leave with twice the debt of even private universities in the USA (£44,500 compared with the equivalent of £29,000 for for-profit universities in the USA and around £20,000 for public and private non-profits). Scotland will have the lowest average student debt (around £10,000), followed by Canada.
A good degree of caution is needed in interpreting these results. Tuition fees can vary considerably, both within countries (e.g. Canada) and between different kinds of institution in the same country (e.g. the USA). Student debt is influenced not just by the level of tuition fees but also by the availability of grants to students, parental contributions, and the availability of part-time work while studying. There are always problems with converting from several different currencies into one standard currency (in this case the U.K £). Debt also is influenced by the economic benefits following from graduation; debt is much more serious if there are few well-paying jobs after graduation. Canadian students may not feel this way, but they are fortunate in that within the first 10 years of graduating their annual income will average twice their total student debt, making repayment more manageable than in all the other countries except Scotland.
Even allowing for understandable methodological difficulties, the differences are stark, and the consequences significant. These are best described by the report’s own main conclusions:
- the average English student faces the highest levels of graduate debt within the major anglophone countries;
- however, the vast majority of English students’ study-related debt is held by the state, which has relatively clear repayment conditions compared to other Anglophone countries;
- as a result [of the high tuition fees], the number of part-time and mature students enrolling at UK institutions across recent years has dropped precipitously
- while full-time undergraduate university enrolment [in England] has recovered since the imposition of £9,000 fees in 2012, university needs to remain a viable option for everyone, especially those from poorer backgrounds, who are disproportionately under-represented across the UK professional landscape.
I am coming to the end of ten days spent in England, talking to friends who include an experienced primary school headmistress, and family who include two professors at English universities, two grandsons about to go to university, and two nieces who have recently completed their university studies. This is not a representative sample, but all week I have been hearing a tale of woe about public education in England.
The current Conservative government seems to be ideologically driven towards the privatisation of public education in England. Government funding for universities has been replaced by tuition fees, and the government wants to introduce market competition between schools and also between universities in the belief that this will drive up ‘quality’. Nevertheless there is no empirical evidence in the UK that shows that students from academies (which are replacing local government-run schools) or institutional competition through tuition pricing in universities is leading to better learning outcomes.
The Conservatives seem to have a completely wrong concept of education, based on set curricula, repeated testing of content, highly selective ‘weeding out’ of students who do not fit this paradigm, and governance by unelected trusts or corporations, a model of education that is clearly influenced by the British public boarding school system from which most of the Conservative government ministers have graduated. The current English education system is in a time warp that seems to belong more to the 1920s than the 2020s.
The results of these government policies have been high levels of stress and anxiety for school children in particular, a fundamental weakening (as intended) of the concept of public education, accompanied by a stagnant economy which is barely above the level following the economic recession of 2008. Could it be that English productivity and innovation are suffering as a result of these misguided educational policies?