- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper aims to address the challenges existing KT methods face in distinguishing short-term fluctuations from changes in students’ underlying knowledge states and handling cold-start scenarios with limited question's practice histories. The authors propose DSCA-KT, which jointly models student and question states using context-aware recurrent aggregators with frequency-based feature gating and a context modulator that incorporates response history, temporal information, and prior structure knowledge about questions and knowledge concepts.
Critical strengths
The paper addresses practically relevant challenges in knowledge tracing. The evaluation covers plenty of datasets under both transductive and inductive settings and includes extensive ablation studies to assess the contributions of the proposed components.
Critical weaknesses
-
The connection between the proposed frequency-based operations and the claimed ability to distinguish meaningful cognitive changes remains insufficiently established. While the results support improved predictive performance, the evidence for this stronger mechanistic interpretation is less convincing.
-
Although the paper evaluates inductive performance, it provides limited discussion of and comparison with KT methods specifically designed for cold-start scenarios.
Questions for authors
-
Regarding the first weakness, Eq. 10–11 aggregates high-frequency magnitudes to generate a feature-wise gate g, which is then applied to X^Emb_s across all time steps. Could the authors clarify how this operation helps distinguish meaningful cognitive changes from transient noise? I appreciate that the method may work empirically, but further theoretical clarification would help support this interpretation.
-
Regarding the second weakness, do any of the evaluated baselines specifically address cold-start scenarios? There are some existing works (e.g. csKT) explicitly studies cold-start KT. It would be better to include some comparisons with cold-start methods if the current experiments have not.
-
Several confusions about Eq.2:
- Eq. 2 appears to assign the indicators to the opposite sequences. In the student’s history, s_i = s always holds, while in the question’s history, q_i = q always holds. Should the student-side features use m_{q_i} and vice versa for the question-side?
- In Eq. 2, the right-hand side uses a scalar indicator for a single step i, so the projected result x_ctx is d-dimensional. However, Eq. 3 adds this result to x_ans, whose dimension is N × d. The dimensions are mismatched here.
-
Could the authors clarify the motivation for treating a question’s response history across different students as an order-sensitive sequence? While new responses provide additional evidence about a question, it is less clear what temporal changes the question-side state is intended to capture, or how high-frequency variations in this sequence should be interpreted. Such variations may reflect differences between the capabilities of students who practice, rather than changes in question state.
-
As a minor presentation suggestion, please check the use of narrative and parenthetical citations throughout the paper. For example, “such as DKT Piech et al. (2015)” should read “such as DKT (Piech et al., 2015),” typically using \citep rather than \citet.
2 comments
Q4: Sorry for the confusion. The question-side state represents the observed interaction context of a question up to the prediction time, rather than a cognitive state of the question itself. Its representation can evolve as evidence accumulates even when the question’s content and intrinsic properties remain unchanged.
Chronological aggregation captures recency and temporal patterns: histories with identical overall correctness rates can exhibit different recent response patterns. Chronological ordering, together with encoded time intervals, preserves each observation’s temporal proximity to the prediction time, allowing the model to distinguish recent patterns from older evidence. These features inform prediction jointly with the target student’s representation.
Variations across a question’s responses can indeed reflect differences in participating students’ abilities. Accordingly, question-side high-frequency statistics describe rapid variation in contextualized interaction features, without uniquely attributing it to intrinsic question difficulty. The frequency-conditioned gate adjusts channel contributions based on this variation; it does not identify its causal source. The motivation is to learn a temporally informed question representation complementary to student-side history, rather than assume that every response fluctuation reflects a change in the question itself.
Q5: Thanks for your suggestion. We will standardize citation formatting throughout the manuscript, using \citep for parenthetical references and \citet when author names form part of the sentence. The example should read “such as DKT (Piech et al., 2015).”
Thank you very much for your further comments and advice. We will answer your questions and weaknesses point by point:
W1 & Q1: We clarify that FE uses high-frequency information as a conditioning signal for channel selection, rather than treating high-frequency content itself as evidence of meaningful cognitive change.
The spectral summary is computed from embeddings incorporating response-related and temporal context. Since the gate is shared across time steps, it does not perform event-level filtering. Instead, it adjusts each feature channel’s contribution based on the high-frequency magnitude statistics of the contextualized history. The reweighted embeddings retain their temporal ordering for subsequent GRU aggregation.
End-to-end predictive supervision learns these channel weights, providing the capacity to retain useful variation and attenuate less predictive features. Frequency alone cannot uniquely distinguish slips, insights, and noise; the gate does not independently identify such events.
Appendix A.5 illustrates responses to isolated errors, abrupt transitions, and alternating outcomes, rather than directly measuring cognitive processes. Our same-backbone experiments further show that FE exceeds Temporal-Std Gate by 1.12 and 1.13 AUC points on ASSIST2012 and ASSIST2017 under the transductive setting. This provides empirical evidence that spectral conditioning captures predictive information not fully reflected by the temporal-standard-deviation summary used in this control.
W2 & Q2: Thanks for the suggestion. csKT provides a relevant comparison specifically targeting cold-start KT. It uses kernel bias for generalization from short training sequences to longer test sequences and cone attention to capture hierarchical knowledge-component relationships. DSCA-KT differs in jointly modeling student- and question-side interaction histories while using a static knowledge graph to generate concept-derived priors for questions, particularly when their interaction histories are sparse.
We conducted an additional comparison on ASSIST2017 under a shared new-student evaluation protocol. Both models use the same data split and test interactions, including students’ first interactions. Predictions are grouped by the number of student interactions observed strictly before each prediction timestamp. This evaluates performance across student-history lengths rather than reproducing csKT’s original train-short/test-long protocol.
DSCA-KT achieves higher AUC and AP than csKT across all student-history groups, with the largest gains in the zero-history group.
Q3: Sorry for the confusion. The indicators are relative to the target student–question pair. In the student’s history, the relevant indicator is mqi=I(qi=q); in the question’s history, it is msi=I(si=s). The implemented correspondence is therefore
\qquad x_{\mathrm{ctx},i}^{q}=m_{s_i}W_{\mathrm{st}}+b_{\mathrm{st}}.$$ These encode whether each historical interaction involves the target question or target student, respectively. The indicator subscripts in Eq. (2) are interchanged; the implementation already uses the correspondence above. Regarding dimensions, each projected $x_{\mathrm{ctx},i}$ is a $d$-dimensional row vector. Stacking the projections over the $N$ historical positions yields $$x_{\mathrm{ctx}}^{s},x_{\mathrm{ctx}}^{q}\in\mathbb{R}^{N\times d},$$ matching the answer embeddings in Eq. (3). Consequently, $CC_s$ and $CC_q$ are also $N\times d$ matrices before recurrent aggregation. These are notation and dimension-labeling issues; the implemented projections and additions operate consistently over the sequence dimension.