Heterogeneous Treatment Effects of Learning Analytics Dashboards: Do All Learners Benefit Equally?


ÖZTÜRK A., Schmucker R., Mitchell T., KUMTEPE A. T.

18th International Conference on Educational Data Mining, EDM 2025, Palermo, İtalya, 20 - 23 Temmuz 2025, ss.418-426, (Tam Metin Bildiri) identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.5281/zenodo.15870272
  • Basıldığı Şehir: Palermo
  • Basıldığı Ülke: İtalya
  • Sayfa Sayıları: ss.418-426
  • Anahtar Kelimeler: heterogeneity of treatment effect, Learning analytics dashboards (LADs), online learning
  • Anadolu Üniversitesi Adresli: Evet

Özet

This study investigates the heterogeneity in the effects of a Learning Analytics Dashboard (LAD) intervention, which provides personalized feedback messages, across a diverse population of learners. Specifically, it evaluates the impact of the LAD on learners’ total material usage and final grades, considering variables such as age, sex, prior material access level, prior academic performance, and firstyear status. The intervention was implemented through a controlled experimental design with 26,753 learners over 17 weeks in a fully online course. Participants were randomly assigned to either the experimental (n = 13,377) or control (n = 13,376) groups. Results indicate significant heterogeneity in the effects of the LAD intervention within the online learning environment. The LAD improved learning material usage and academic performance, with factors such as sex, age, prior course material access, and prior academic performance moderating these effects. Female learners, older learners, and those with low prior academic performance benefited more from the intervention. This suggests that personalized learning analytics tools can enhance engagement and outcomes for specific learner subgroups, reinforcing the need for targeted interventions in online education.