- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper identifies two limitations in cognitive diagnosis (CD): existing methods mostly rely only on response outcomes and interaction relations, without fully exploring learning dynamics encoded in time intervals; explicit ability estimates derived from limited historical interactions are susceptible to response fluctuations and insufficient concept coverage, resulting in unreliable cognitive state estimation. The authors propose DBSCD, which builds dual-behavior sequences via joint modeling of response outcomes and time intervals, takes explicit concept accuracy as a statistical prior, refines ability representations through conditional diffusion, and adopts content-enhanced item representations and an IRT-style prediction head. The method is validated on two online datasets and a newly constructed offline exam dataset named Computer. The core merit of this work is the real-world offline exam dataset, which expands the evaluation scope from online tutoring logs to high-stakes exam scenarios. However, four major issues remain:
- Insufficient novelty: A number of prior CD studies have already integrated temporal information, but none of these relevant works are cited. The claim that "the temporal dimension remains unexplored in CD" is unsupported.
- Weak evidence for the core contributions.
- The specific mechanism by which learning dynamics influence diagnostic performance is not clarified.
- Lack of mechanistic case analysis. For the reasons above, this paper is not recommended for acceptance.
Critical strengths
- This work introduces a new real-world offline examination dataset, dubbed Computer, that extends the evaluation scope of cognitive diagnosis from timestamped online tutoring logs to high-stakes exam scenarios.
Critical weaknesses
Insufficient novelty and missing discussion of closely related work. To the best of my knowledge, UTIRT, KPT, CT-NCM, TC-MIRT and several other studies have already incorporated temporal information into cognitive diagnosis, yet none of these direct prior works are cited or discussed in this paper. Furthermore, the paper fails to clarify the essential distinctions between the proposed DBSCD and these existing methods. As a result, the central claim that "the temporal dimension has not been explored in cognitive diagnosis" is factually untenable and not well supported.
-
Weak empirical evidence for the core contribution. The empirical support for the core contribution — conditional diffusion-based ability refinement — is notably weak. In the ablation study, removing the diffusion module leads to an AUC drop of only 0.0009 on XES3G5M (from 0.8029 to 0.8020) and 0.0022 on MoocRadar (from 0.8716 to 0.8694). These effect sizes are comparable to, or even entirely fall within, the reported standard deviations across random seeds (0.0007 and 0.0022, respectively). The observed change in accuracy on XES3G5M also falls within the range of experimental noise, meaning the diffusion module does not deliver statistically meaningful performance gains.
-
Unelaborated functional impacts of learning dynamics. The specific impacts of learning dynamics remain largely unillustrated. The paper only states that learning dynamics reflected by time intervals are insufficiently exploited, but does not elaborate on their concrete functional effects — for example, how time intervals map to forgetting mechanisms, knowledge retention patterns, or other well-established learning theories.
-
Lack of interpretability and conditional analysis in experiments. The experimental design is deficient in in-depth case analysis and interpretability evidence. Across the entire paper, there is no empirical analysis or case study to explain why the introduction of time intervals and the diffusion module is effective, nor under what conditions (e.g., for specific student groups, exercise types, or interaction patterns) the proposed approach yields consistent benefits.
Questions for authors
-
In the ablation study of the "w/o diffusion" variant, what specific component or operation replaces the proposed diffusion module? Is it substituted by an identity mapping, a linear projection layer, or another baseline operation?
-
The abstract claims that the work models temporal learning dynamics. Does simply treating time intervals as embedding features provide sufficient justification for this claim? Furthermore, is there empirical evidence demonstrating that learned time interval embeddings outperform explicit forgetting curve formulations or classical stochastic process models in capturing temporal learning patterns?
-
The baseline methods NCD and ACD exhibit notably poor AUC performance relative to other approaches. Have the authors analyzed the underlying reasons for this underperformance? For instance, does it stem from the inherent limitations of the methods, suboptimal implementation details, or misalignment with the experimental settings?
-
To enhance the generality and cross-method comparability of the experimental results, the authors are requested to include at least one widely adopted benchmark dataset that is commonly used across mainstream cognitive diagnosis methods for experimental comparison.