Tracing Mathematical Proficiency Through Problem-Solving Processes
cs.LG, cs.AI, cs.CY
Submitted: 2025-11-29
Updated: 2026-09-07
Comments: Accepted to ACL 2026 Findings
DOI: 10.18653/v1/2026.findings-acl.961
License: http://creativecommons.org/licenses/by/4.0/
The gist: Knowledge Tracing (KT) aims to model student's knowledge state and predict future performance to enable personalized learning in Intelligent Tutoring Systems.
Terminology
Abstract
Knowledge Tracing (KT) aims to model student's knowledge state and predict future performance to enable personalized learning in Intelligent Tutoring Systems. However, traditional KT methods face fundamental limitations in explainability, as they rely solely on the response correctness, neglecting the rich information embedded in students' problem-solving processes. To address this gap, we propose Knowledge Tracing Leveraging Problem-Solving Process (KT-PSP), which incorporates students' problem-solving processes to capture the multidimensional aspects of mathematical proficiency. We also introduce KT-PSP-25, a new dataset specifically designed for KT-PSP. Building on this, we present StatusKT, a KT framework that employs a teacher-student-teacher three-stage LLM pipeline to extract students' Mathematical Proficiency (MP) as intermediate representation. In this pipeline, the teacher LLM first extracts problem-specific proficiency indicators, then a student LLM generates responses based on the student's solution process, and a teacher LLM evaluates these responses to determine mastery of each indicator. The experimental results on KT-PSP-25 demonstrate that StatusKT improves the prediction performance of existing KT methods. Moreover, StatusKT provides interpretable explanations for its predictions by explicitly modeling students' mathematical proficiency. Code is available here.
Sources
- DBE-KT22: A Knowledge Tracing Dataset Based on Online Student Evaluation
- Adam: A Method for Stochastic Optimization
- Cold Start Problem: An Experimental Study of Knowledge Tracing Models with New Students
- A Self-Attentive model for Knowledge Tracing
- simpleKT: A Simple But Tough-to-Beat Baseline for Knowledge Tracing
- GIKT: A Graph-based Interaction Model for Knowledge Tracing
- Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory
- DKT2: Revisiting Applicable and Comprehensive Knowledge Tracing in Large-Scale Data
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks