- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SPAR, a state-conditioned framework for exercise recommendation that addresses three related problems: unreliable learner representations under sparse interaction histories, difficulty-aware candidate retrieval, and adaptive balancing of multiple recommendation objectives.
SPAR consists of three stages. First, prototype-guided alignment uses representations of active learners to improve those of learners with sparse histories. Second, ZPD-inspired retrieval adjusts the target difficulty according to recent response accuracy and estimated concept mastery. Third, a state-conditioned routing network combines relevance, difficulty, and diversity scores to rank candidate exercises.
The authors evaluate SPAR on three public datasets against eight baselines. The experiments include overall performance comparisons, component ablations, cold-start analysis, parameter sensitivity, and controlled comparisons of retrieval and re-ranking modules. SPAR achieves consistent improvements in offline recommendation metrics, particularly for learners with sparse interaction histories.
Critical strengths
-
A reasonable approach to coordinating different stages of exercise recommendation. The paper recognizes the dependencies between learner representation, candidate retrieval, and re-ranking. Inaccurate learner representations may lead to inappropriate candidate sets, which cannot be fully corrected by downstream re-ranking. Incorporating learner-state information across these stages is a meaningful approach to improving adaptive exercise recommendation.
-
Consistent performance improvements with useful component-level evidence. SPAR improves NDCG@10 over the strongest baselines on all three datasets (Table 2). The ablation studies (Table 3) show that each module contributes to performance. More importantly, the retrieval/re-ranking comparisons in Appendix A.3 (Table 4) show that both proposed components can improve existing recommendation pipelines. The cold-start analysis further supports the benefit of prototype alignment for learners with sparse histories.
These experiments provide credible evidence for the effectiveness of the proposed system in offline recommendation.
Critical weaknesses
-
The training signal for the diversity component is unclear and potentially inconsistent with its definition. In Section 3.5, Eqs. (9)–(11), diversity gain is defined as additional concept coverage relative to a learner's historical exercises. However, Section 4.1 states that candidates containing unseen concepts are excluded from the BCE loss. Under these definitions, candidates used for BCE training would have zero marginal historical concept coverage, leaving the diversity scorer without a candidate-dependent coverage signal from these examples. The paper does not explain whether additional supervision or implementation details resolve this issue. This concern directly affects the proposed multi-objective routing mechanism, which is a central contribution.
-
The experiments do not sufficiently isolate the benefits of state-conditioned adaptation. The meso ablation replaces ZPD-inspired retrieval with global retrieval, rather than comparing dynamic and fixed difficulty thresholds within the same retrieval framework. Similarly, the micro ablation compares state-conditioned routing with uniform weights, but not with learned state-independent weights. Furthermore, Figure 4 demonstrates changes in routing weights over time, but does not establish that these changes reflect appropriate pedagogical priorities. Given that existing methods already employ multi-stage exercise recommendation, stronger evidence is needed to establish the independent contribution of the proposed state-conditioned mechanisms.
Questions for authors
-
In the implementation, how does the diversity scorer receive informative training signals for nonzero concept coverage gains, given the exclusion of candidates containing unseen concepts from the BCE loss? Are there additional losses or implementation details not described in the paper?
-
Could the authors provide controlled comparisons of (a) dynamic versus fixed difficulty thresholds and (b) state-conditioned versus learned global routing weights, while keeping the remaining components unchanged? These results would help isolate the contribution of state adaptation.
Thank you very much for your further comments and advice. We will answer your questions and weaknesses point by point:
W1 & Q1: Thanks for your suggestion. In the original submission, Eq. (10) defined marginal coverage relative to the learner’s history. Under that definition, every BCE-eligible exercise has zero marginal coverage gain. Eqs. (9)–(11) in the revised manuscript define coverage over the retrieved candidate pool Ec, as used in our experiments. For an exercise ej, the contribution is
Δj=Coverage(Ec)−Coverage(Ec∖{ej}).
Under this definition, excluding candidates involving unseen concepts from BCE supervision does not necessarily make their diversity gains zero. For example, an exercise that uniquely covers a previously practiced concept can pass the BCE mask if all its concepts have observed responses. Removing that exercise reduces candidate-list coverage.
We multiply the coverage feature elementwise by the learner’s concept preference vector and pass the result to the diversity scorer. The diversity score contributes to the fused recommendation score used by BCE on eligible candidates. A nonzero coverage feature can therefore carry candidate-dependent gradients to the diversity scorer and the corresponding preference parameters. This learning signal comes from the existing BCE objective; no additional diversity-specific loss is required for this mechanism.
Section 4.1 now specifies that coverage is computed over the full retrieved pool before the BCE mask is applied.
W2 & Q2: Thanks for your suggestion. Table 3 evaluates module-level changes: the meso ablation replaces ZPD-inspired retrieval with global retrieval, while the micro ablation replaces state-conditioned routing with uniform weights. Figure 4 illustrates how routing weights evolve. To isolate the contributions of state adaptation to offline recommendation quality, we compare two sets of matched variants.
(1) Adaptive versus fixed difficulty targets. We compare the adaptive target in Eq. (4) with a single fixed difficulty target selected on the validation set. We use the same mastery estimates, exercise search space, retrieval scoring rule, candidate budget, and downstream re-ranking protocol. The candidate lists can differ because the target determines which exercises are retrieved.
Supplementary comparison of adaptive and fixed difficulty targets
(2) State-conditioned versus learned global routing weights. We compare the proposed state-conditioned gate with a learned global softmax weight vector shared across learners. Both variants are trained independently under the same training and validation protocols, using identical candidate lists, scorer architectures, and training objectives.
Supplementary comparison of state-conditioned and learned global routing weights
On XES3G5M, Assist2009, and Assist2012, the adaptive difficulty target improves downstream NDCG@10 over the fixed target by 0.94, 1.83, and 0.84 points, respectively. State-conditioned routing improves NDCG@10 over learned global weights by 0.89, 2.33, and 0.54 points, respectively. The matched comparisons quantify the contribution of each adaptation to offline recommendation performance under the corresponding protocol.