lnu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Use of Learning Analytics in K-12 Mathematics Education: Systematic Scoping Review of Impact on Teaching and Learning
Linnaeus University, Faculty of Social Sciences, Department of Pedagogy and Learning.ORCID iD: 0000-0002-1914-1626
Linnaeus University, Faculty of Social Sciences, Department of Education and Teacher's Practice.ORCID iD: 0000-0002-2924-4100
Linnaeus University, Faculty of Social Sciences, Department of Pedagogy and Learning.ORCID iD: 0000-0001-7525-6180
Linnaeus University, Faculty of Technology, Department of computer science and media technology (CM).ORCID iD: 0000-0002-3297-0189
Show others and affiliations
2024 (English)Conference paper, Oral presentation with published abstract (Refereed)
Sustainable development
SDG 4: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all
Abstract [en]

Introduction

The generation and use of digital data and analyses in education comes with promises and opportunities, especially where digital materials allow use of Learning Analytics (LA) as a tool in Data-Based Decision-Making (DBDM). LA implies, analysing educational data to understand and optimise learning and learning environments (Siemens & Baker, 2012). In this paper we discuss LA as “a sophisticated form of data driven decision making” (Mandinach & Abrams, 2022, p. 196) as we explore how LA is used to support mathematics teaching and learning with digital materials in classroom practice. Data driven decision making or DBDM has been defined by Schildkamp and Kuiper (2010) as “systematically analyzing existing data sources within the school, applying outcomes of analyses to innovate teaching, curricula, and school performance, and, implementing (e.g., genuine improvement actions) and evaluating these innovations” (p. 482). DBDM is a key for the interpretation of LA, and can use any form of data, but in this review, the term DBDM is restricted to digital data. Using LA as a tool for DBDM could streamline data, making it more readily interpretable. However, questions remain about how usage can translate into practice (Mandinach & Abrams, 2022). 

Quality of technology integration is not merely about technology use, but also about pedagogical use (Ottestad & Guðmundsdottir, 2018), about transformation and amplification of teaching as well as learning through use of technology (Consoli, Desiron & Cattaneo, 2023). LA within Digital Learning Material (DLM) can offer learners adaptive functions seamlessly embedded in DLMs or, provide learners (and teachers) compiled student assessments in relation to learning goals extracted from learning activities (Wise, Zhao & Hausknecht, 2014). The role of the teacher in student learning is clearly of central importance (Hattie & Yates, 2013; Yackel & Cobb, 1996), and teachers have a key responsibility to make digital technology a recourse in teaching to support student learning (Scherer, Siddiq & Tondeur, 2019). 

This paper present findings from an exploratory systematic scoping review which was conducted regarding the use and impact of LA and DBDM in classroom practice to outline aspects related to Digital Learning Material (DLM), teacher usage, and student learning in the context of K-12 mathematics education. 

A scoping review was deemed most appropriate since it can be performed even if there is limited number of published primary research (Gough, Oliver & Thomas, 2017), fitting new research areas such as LA, as it provides “a technique to ‘map’ relevant literature in the field of interest” (Arksey & O’Malley, 2005, p. 20), as well as combine different kinds of evidence (Gough, et al., 2017).

Method

The methodology used the five-stage framework (Arksey & O’Malley, 2005), identifying the research question, identifying relevant studies, study selection, charting the data, collating, summarizing, and reporting the results. The databases ACM Digital Library, ERIC, PsycINFO, Scopus and Web of Science were chosen as they cover a wide range of topics within both technology and educational science to answer:

RQ1: How are analyses of digital data from DLM used in mathematics education?

RQ2: How do analyses of digital data from DLM impact teaching and learning?

The key elements of the research questions, Participants, Phenomena of Interest, Outcome, Context, Type of Source of Evidence (Arksey & O’Malley, 2005) were used to create the eligibility criteria. Publications that were included reported qualitative and/or quantitative data and were connected to the use of DLM and LA based on digital data involving students (between 6–19 years old) and teachers in mathematics K-12 education. The search was limited to papers published from 2000 up-to-date (March 2023) in English, Swedish or Norwegian. Exclusion criteria were developed to ensure consistency within the selection process.

Each record was screened by two reviewers and the relevance were coded according to the inclusion criteria. An independent researcher outside of the review group was consulted to design and validate the results of an inter-rater reliability test. The calculated inter-rater reliability score was 0.822, greater than 0.8, indicating a strong level of agreement (McHugh, 2012). After further screening 57 records were assessed to be eligible. At this stage the review pairs swapped batches and preformed data extraction showing, authors, year, title, location, aim, population, digital technology, method, intervention, outcomes, and key findings was performed for each record. 

The final selection of 15 articles was made by group discussion and consensus. Discussions mainly centred around four components (use, analysis, learning and teaching). The heterogeneity in our sample demanded a configurative approach to the synthesis to combine different types of evidence (Gough et al., 2017). A thematic summary provided the analysis with a narrative approach to answer RQ1. To explore RQ2 more deeply, a thematic synthesis was performed (Gough et al., 2017). The analysis focused on LA-usage based on digital data for student learning, for teaching, and for teachers’ DBDM. PRISMA Extension for Scoping Reviews (PRISMAScR) (Tricco, Lillie, Zarin, O'Brien, Colquhoun, Levac et al., 2018) was used as guidelines for reporting the results.

Preliminary results

3653 records were identified whereof 15 studies were included. Results show that LA-research is an emerging field, where LA-applications is used across many contents and curricula standards of K-12 mathematics education. LA were mainly based on continuously collected individual student log data concerning student activity in relation to mathematical content. Eight of the studies included embedded analytics and all 15 studies included extracted analytics, but accessibility varied for students and teachers. Overall, extracted analytics were mainly mentioned as a function for teacher-usage, available as tools for formative assessment, where analytics need to be translated by teachers into some kind of pedagogical action (i.e., into teaching).

LA-usage supports a wide variety of teachers’ data use, and while mathematics teachers seemed to have a positive attitude towards LA-usage, some teachers were unsure of how to apply it into their practice. The thematic synthesis yielded two themes regarding teaching, which showed that teaching by DBDM focused on Supervision and Guidance. Results indicate extracted analytics is more commonly used for Supervision than guidance. 

Results regarding learning suggest that LA-usage have a positive effect on student learning, where high-performing students benefit most. The included studies examine students’ digital learning behaviour, by describing sequences of actions related to LA, learning outcomes and student feelings. Hereby, through the thematic synthesis, we capture parts of students’ studying-learning process and how it can be affected by LA usage. Finally, we suggest a definition of an additional class of LA, which we introduce as Guiding analytics for learners.

Going forward, research on using LA and DBDM is essential to support teachers and school leaders to meet today’s demands of utilising data, to be aware of possible unwanted consequences, and to use technology to enhance active learners and students’ ownership of learning.

Place, publisher, year, edition, pages
2024. article id 629
Keywords [en]
Learning Analytics, K-12 education, teaching, learning, data-based decision-making
National Category
Educational Sciences Computer and Information Sciences
Research subject
Pedagogics and Educational Sciences; Computer and Information Sciences Computer Science
Identifiers
URN: urn:nbn:se:lnu:diva-133161OAI: oai:DiVA.org:lnu-133161DiVA, id: diva2:1917734
Conference
The European Conference on Educational Research (ECER), Nicosia, Cyprus, 27-30 August, 2024
Note

This is a shorter and preliminary paper based on an accepted paper soon to be published.

Available from: 2024-12-03 Created: 2024-12-03 Last updated: 2025-01-14Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Rundquist 2024 ECER

Authority records

Rundquist, RebeckaHolmberg, KristinaRack, JohnMohseni, ZeynabMasiello, Italo

Search in DiVA

By author/editor
Rundquist, RebeckaHolmberg, KristinaRack, JohnMohseni, ZeynabMasiello, Italo
By organisation
Department of Pedagogy and LearningDepartment of Education and Teacher's PracticeDepartment of computer science and media technology (CM)
Educational SciencesComputer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 66 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf