- Rating
- 2
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
As a researcher in AI, this work is very difficult for me to understand. I would not attribute this case to heavy AI writing, but because very strong background mismatch.
In a more natural presentation: this work tries to optimize daily learning schedule for a human learner. To do this, one need to fit a forgetting curve (HLR), and then predict a better schedule (ACRI) given the forgetting dynamics.
I am not sure whether OSR is a standard word for education, but definitely not a common technical term in AI.
Critical strengths
This work has extensive baselines for comparing the accuracy of different algorithms.
Critical weaknesses
First, I am not sure who the intended audience is, but I am definitely not one of them. From the perspective as an AI researcher, the method design in section 4 is very specific to the dataset, and quite arbitrary. There is no scientific insight or generalizable technique.
Second, it seems half of the claim is built around the ACRI method, which is unverified in practice, but only simulated for comparison. The simulation uses the ground truth model and hence the improvement is expected.
Questions for authors
What is the key contribution from the perspective of AI researchers?
5 comments
Response Part 2/2
W2 — Practical verification and the use of simulation
"Half of the claim is built around ACRI, which is only simulated"
(a) The core claims are verified on real learners' outcomes.
Most of the paper's evidence does not rely on simulation. OSR-HLR's predictions are evaluated on held-out real reviews from three platforms, under 15 dataset–protocol settings and on separately held-out Anki users. The decisions these predictions induce are also scored on real data: in the next-review evaluation, each interval chosen by a shared threshold scheduler is evaluated against a half-life independently fitted from real held-out reviews, and OSR-HLR reduces target-recall error by 19.7% and late-review shortfall by 23.1%. Simulation is used only for the 365-day analyses, where it is the only way to follow many predictors and schedulers on the same learners over a year.
(b) The simulation is grounded in real-world data and enables comparisons a live trial cannot.
Every component of the simulation comes from real learners: the predictors are trained on real review logs, the initial memory states are taken from real held-out learners, and the transition environments are fitted on real review transitions. Simulation then offers three advantages. First, every policy is evaluated on exactly the same memory states and random outcomes, a paired counterfactual comparison that a live trial cannot provide, because a real learner can follow only one schedule. Second, it makes large sensitivity analyses feasible, such as the four transition environments and the 105-setting scan below. Third, it exposes no real learner to an untested policy.
(c) Future work: verification in a deployed system.
To further address the reviewer's concern, we will explore real-world deployment and seek collaboration with a learning platform for a prospective study. This is feasible: related memory models already run in Duolingo (Settles & Meeder, 2016) and Anki (FSRS, one of our baselines), DHP/SSP-MMC were developed with the MaiMemo app (Ye et al., 2022; Su et al., 2023), and our methods use the same kind of review logs.
"The simulation uses the ground-truth model, hence the improvement is expected"
(a) No policy reads the simulator's ground truth.
In the scheduling comparison, every scheduler receives the same predictor, not the simulator's latent half-life. The latent half-life is hidden from all policies, which observe only forecasts, completed outcomes and queue age. All policies also share the same initial states, daily capacity, transition environment and random streams, so the simulator treats every method identically and the only difference is the allocation rule. OSR-ACRI's advantage therefore cannot come from knowing the answer.
(b) The advantage is robust across environments and predictors.
The simulator is fitted independently from Train transitions, with four environments (ridge and gradient-boosting growth models, each in a standard and a pessimistic variant). In the headline 365-day comparison over all four environments at matched workload, OSR-ACRI's paired gain over Threshold is +0.056 retention AUC (95% CI [+0.051, +0.060]; 14/16 paired decks improved). In a further 105-setting scan over three predictors unrelated to the simulator (DHP, GRU-HLR, FIFAKT), five daily capacities and seven target-recall levels, using two of these environments, OSR-ACRI ranks first among seven schedulers in 101/105 settings for retention AUC, 102/105 for time below target and 87/105 for final recall.
Thanks
I thank the authors for the feedback.
I think this work is very applied, and more like a field study for data science in education / cognition. The AI and math aspect in this work is not convincing, but I am not qualified to judge this work in its particular applied field. I still recommend rejection as the topic is a bit too focused, but I would also hope the AC to seek feedback from experts in the field.
Response Part 1/2
We sincerely thank the reviewer for their careful evaluation, constructive suggestions, and recognition of the strengths of our work. Below, we address your concerns point by point.
W1 / Q1 — Key contribution from an AI perspective
To address your concern, we clarify four points: we develop novel AI methods for this problem
(a); our experiments yield scientific insights into how memory can be modeled and acted upon(b); the methods transfer across models, datasets and evaluation protocols(c); and the work belongs to an established line of research at major AI venues(d).(a) What the work contributes [W1, Q1].The reviewer's plain-language summary is accurate: we fit a forgetting curve (half-life) from review histories and use it to decide what to review each day under a fixed budget. "OSR" simply abbreviates Open-Space Retrieval, our name for the retrieval-based correction layer. We develop novel AI methods for this problem. From an AI perspective, the work contributes three techniques that are not tied to a particular dataset:
(b) Scientific insight [W1].Our experiments reveal three findings about how human memory can be modeled and acted upon:
(c) The design is neither dataset-specific nor model-specific [W1].The concern that the method is tailored to one dataset is directly testable, and three results show that it transfers across models, data sources and evaluation protocols:
(d) Relevance to the AI community [W1, Q1].Memory modeling and review scheduling are an established, widely cited topic at major AI venues. Half-life regression was introduced at ACL (Settles & Meeder, 2016), deep knowledge tracing at NeurIPS (Piech et al., 2015; over 1,800 citations on Semantic Scholar) and optimized spaced-repetition scheduling at KDD (Ye et al., 2022), and recent memory and forgetting models continue to appear at AAAI and KDD (Bai et al., 2025; Sun et al., 2025).