- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents a framework for multimodal knowledge tracing. It uses a vision–language model to convert exercise images into structured descriptions offline. These descriptions are encoded to represent exercises and build a thresholded semantic graph. It then retrieves relevant past interactions and processes them with a selective state-space model. The authors report better performance than recurrent, attention-based, and sparse KT baselines on NIPS34, MOOC-Radar, and XES3G5M. They also provide ablation studies, sensitivity analyses, runtime comparisons, and qualitative visualizations to further assess the method.
Critical strengths
-
This paper jointly addresses visual exercise content and irrelevant historical interactions, which makes the proposed direction useful for educational platforms.
-
Potentially practical deployment design. Separating offline content understanding from online tracing allows expensive VLM outputs to be reused across learners, which potentially reduces inference overhead.
Critical weaknesses
-
Although selective state-space modeling is central to the framework, Table 2 omits Mamba4KT, which the paper itself cites. There is also no controlled comparison replacing the SSM with an LSTM or Transformer while retaining the same semantic inputs and retrieval module. Consequently, the experiments do not adequately distinguish gains from the proposed retrieval mechanism, additional pretrained semantic information, and the backbone architecture.
-
Removing structured semantic rendering reduces the model to ID/concept embeddings, eliminating content information altogether. This demonstrates the potential value of additional content, but does not establish that the proposed three-stage rendering is superior to OCR text, original exercise text, or direct image features. The paper also does not assess the accuracy of generated visual relations or performance specifically on exercises requiring visual information.
-
Equation (5) defines using field projections , while lines 172–174 state that all are computed once and cached, and that the field projections are optimized with KT supervision. Updating these projections would generally change both and the graph derived from it. Conversely, keeping the final representations fixed would prevent downstream gradients from updating the projections through that path. The paper needs to specify what is cached, when the graph is updated, and how gradients propagate. Without these details, the training procedure is not unambiguously defined.
-
A linear-time SSM scan does not imply linear complexity for the full model, which also performs history retrieval. A fixed similarity threshold does not bound the number of retained neighbors. When historical exercises are highly similar, the operator in Equation (8) can still contain nonzero entries. The paper does not report how retained-edge counts scale with sequence length, and Figure 7 lacks sufficient information about hardware, batch size, and the scope of timing. These omissions limit the claimed efficiency advantage.
Questions for authors
-
Table 2 contains mathematically inconsistent probability-error metrics.
For LefoKT on MOOC-Radar, the reported RMSE and MAE are and , respectively. For binary labels and probability predictions, , so . Therefore,
However,
These values cannot arise from the same probability predictions evaluated over the same samples with consistent weighting.
-
The matrix products in Equations (8) and (14) are dimensionally invalid as written.
The paper defines and , while Equation (7) implies a feature projection . Thus, Equation (8),
is not a valid matrix expression for general . A dimensionally consistent formulation matching the per-position feature projection would be
Equation (14) repeats the same ordering problem.