- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the challenge of distinguishing short-term response fluctuations from genuine mastery changes in knowledge tracing and proposes a framework called BddKT. The method uses an LLM to encode item identifiers and response statistics into item profiles, and combines exponential moving average (EMA)-based temporal state updates with attention-based historical profile replay to calibrate student knowledge states. Experiments are conducted on four public datasets against 15 baseline models, supplemented by ablation studies, hyperparameter analysis, temporal robustness evaluation, and case studies. The reported results show that BddKT outperforms the compared baselines in terms of AUC and ACC.
Critical strengths
1.The paper addresses the ambiguity between transient response fluctuations and genuine mastery changes in knowledge tracing, a meaningful challenge for reliable student knowledge-state estimation. 2.The paper evaluates BddKT against 15 baselines on four public datasets, supplemented by ablation and robustness analyses. The reported AUC and ACC results consistently outperform the compared baselines, suggesting promising predictive performance.
Critical weaknesses
1.Eq. (3) incorporates the current response (r_t) into the input representation, which is subsequently used in Algorithm 1 to predict the same response. This appears inconsistent with the prediction objective defined in Eq. (1). Since the paper does not clarify whether temporal shifting is applied, this raises concerns about the validity of the reported experimental results. 2.The experiments primarily demonstrate predictive performance and temporal robustness under varying history lengths, but do not rule out general temporal smoothing or historical information fusion as alternative explanations for the improvements. In particular, the paper does not adequately establish whether BddKT can suppress short-term response fluctuations while remaining sensitive to genuine mastery changes, leaving its central discrimination claim insufficiently supported.
Questions for authors
1.Eq. (3) incorporates the current response (r_t) into the input representation, which is subsequently used to predict (\hat y_t) in Algorithm 1. Could the authors clarify how the input sequences and prediction targets are aligned in the implementation, and confirm that (r_t) is not accessible when predicting the same response? 2.The current experiments mainly evaluate predictive performance and temporal robustness. Could the authors clarify what evidence supports the claimed discrimination between transient behavioral drift and genuine mastery changes, beyond improvements potentially attributable to temporal smoothing or historical information fusion? 3.Since the item profiles are constructed from item identifiers and aggregate response statistics, could the authors clarify the benefit of LLM-based encoding over directly embedding these features? In particular, how does BddKT compare with a simple MLP-based profile encoder using the same inputs? 4.Table 2 reports AUC values of 0.8843 and 0.9362 for BddKT on AS2009 and Junyi, respectively, whereas Figure 5 reports 0.9196 and 0.9237 under the full-history setting. The authors should clarify the reasons for these discrepancies. 5.Eq. (5) produces non-finite values when item accuracy equals 0 or 1. The authors should clarify whether smoothing or clipping is applied. 6.Section 3.4 does not clearly explain how historical item profiles are selected for the replay candidate pool or how sequences shorter than the replay depth are handled. 7.Figure 7 introduces a risk-related gating expression involving (Risk_t), which is not defined in the methodology. Its relationship to the calibration gate in Eq. (20) should be clarified. 8.Eq. (15) appears to repeat Eq. (10) and contains text unrelated to the replay retrieval attention mechanism. Additionally, Sections 4.5 and 4.6 share the same heading, "Temporal Robustness Analysis," despite addressing different analyses.
I look forward to the authors’ response and am open to further discussion.