- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduced GRACE, a knowledge tracing framework that treats cognitive-state evolution as a learned optimization process. GRACE recasts state evolution as a learned gradient step on the learner's own BCE loss: the state-gradient in concept coordinates supplies the update direction, and a graph-conditioned meta-optimizer learns the update rule from interaction context — emitting a per-concept retention gate, step size, graph-diffusion coefficient, and a diagonal-plus-low-rank preconditioner — so the same concept graph both represents concepts and propagates gradients. Experiments on multiple public datasets confirm the effectiveness of the proposed method.
Critical strengths
1.This paper turns the state transition from an architectural by-product into an explicit, learnable modeling component. 2.This paper supplies the update direction directly via the gradient, while a context-dependent meta-optimizer learns retention and magnitude. 3.This paper explicitly controls how updates propagate across concepts. 4.Experiments on three public datasets confirm the effectiveness of the proposed method.
Critical weaknesses
1.The model is over-engineered: a large stack of learned control quantities without a compelling account of why each is needed. 2.For a knowledge-tracking paper, the design is not grounded in educational or learning theory. 3.The central claim is never tested against the most obvious alternative. Every ablation retains the gradient path, so the paper never compares its gradient-guided update against a parametric transition of matched parameter count within the same framework. Without that comparison, the claim that "using the gradient" — rather than the added gating/preconditioning capacity — is what matters remains unverified, and this is exactly the comparison that would settle the paper's thesis. 4.Reported results lack variance and significance testing, and the reported protocol is not fully consistent. Results are single runs without standard deviations or significance tests, while several margins are only 0.1–0.45 AUC points and thus within plausible run-to-run variation. Separately, the preprocessing folds items into "number of items = number of concepts" (117/117, 715/715) and reports baseline AUCs far below values common in the public literature, contradicting the stated pyKT-protocol comparability and making cross-paper comparison unreliable. No parameter-count or training-cost comparison is given either, weakening the "better than wider representations" argument.
Questions for authors
1.All ablations retain the gradient path, so the paper never compares the gradient-guided update against a parametric transition of matched parameter count under the same framework. Can you provide this comparison — replacing g_t with a learned parametric map of equal capacity but keeping gating, diffusion, and preconditioning fixed? 2.In the claimed "learned optimizer" setting, an update is required at every interaction, yet no parameter-count or training-cost comparison is reported. What are the parameter counts and training/inference time per epoch relative to the strongest baseline, and how much of the gain survives when the baseline is given an equal parameter and compute budget? 3.Results appear to be single runs with no standard deviations or significance tests, while several margins are only 0.1–0.45 AUC points. Can you report variance across multiple seeds and significance tests against the strongest baseline, and clarify the inconsistency between the reported preprocessing and the claimed pyKT-protocol comparability?