The lack of generalisability in learning analytics research: why, how does it matter, and where to?

Concerns about the lack of impact of learning analytics (LA) research has been part of the evolution of the field since its emergence as a research focus and practice in 2011. The preponderance of small-scale and exploratory nature of much of LA research are well-documented as contributing factors to the lack of generalisability, transferability, replicability and scalability. Through an analysis of 144 full research papers published in the conference proceedings of the Learning Analytics & Knowledge (LAK) Conference '22, 23 and 24, this paper provides an overview of the extent and contours of the lack of generalisability in LA research and pointers for making LA research more generalisable. The inductive and deductive analysis of the recent three LAK conferences provide evidence that a significant percentage (46%) of the corpus papers do not refer at all to generalisability or transferability, while few papers report on the scalability of their research findings. While the crisis of replicability/reproducibility is a wider concern in the broader context of research, considering and reporting on generalisability and transferability is integral to the scientific rigour. We conclude our paper with a range of pointers for addressing the lack of generalisability in LA research including, but not limited to expanding data, methodological adaptation and the potential of open science.