- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes OnePred, which predicts a user's next query using a compact memory updated after each conversational turn. It trains the prediction first with the full history and then with recursive memory. Experiments show improved prediction scores and lower token use, although gains over learned-summary baselines are small.
Critical strengths
The method reduces token use substantially, even after counting the cost of generating memory. This is useful for prediction over longer conversations.
Critical weaknesses
The improvement over learned-summary baselines is small, and the paper does not report paired significance tests or variation across training runs. It is unclear whether tracking intent provides a reliable advantage over ordinary recursive summarization.
Questions for authors
Are the gains over Summary-memory RL consistent across training runs, and can you provide paired confidence intervals?
How does OnePred compare with the closest next-query prediction and learned-memory methods discussed in the paper?
Reliability across runs and comparison with the closest methods
Thank you for these questions. On NQP-Wild, OnePred scores higher than both learned-summary controls in each of three training runs, and the paired 95% intervals for the reported run lie above zero. The additional comparisons below also address the closest learned-memory and next-utterance methods.
Gains over learned-summary controls. On NQP-Wild, OnePred improves over Summary-memory RL by +0.87 points (paired 95% CI [0.28, 1.48]) and over Full-history Summary RL by +0.66 ([0.06, 1.24]). These are pointwise percentile intervals from 10,000 paired bootstrap resamples of conversations drawn with replacement, retaining all prediction points within each sampled conversation and preserving prediction-point-weighted averaging. Differences use the 0-100 alignment scale.
Across three training runs, the paired gains are:
SD is the sample standard deviation across runs; the bootstrap intervals concern Run A. The controls share the memory budget, candidate count, prediction reward, and training-step budget. These comparisons test the added predictive value of OnePred's representation and training procedure under matched conditions.
On the deployment logs, the gains over Summary-memory RL are +0.40 for Best-of-3 (95% CI [-0.28, 1.07]) and +0.45 for Top-1 ([-0.18, 1.06]); these intervals leave the differences uncertain.
Closest-method comparisons. We evaluated NQP adaptations of MemAgent and MEM1, which learn recurrent memory from task outcomes, and ProUtt, which uses intent trajectories for next-user-utterance prediction.
All rows use three candidates. ProUtt's published configuration generates four, so the result here concerns its three-candidate adaptation. The intervals favor OnePred over the MemAgent and MEM1 adaptations, while the ProUtt comparison remains uncertain.
OnePred directs its bounded state toward active topics, unresolved needs, and changes in user intent. Its two-stage schedule first learns full-history prediction, then learns to retain information useful for that prediction. The matched-summary results support a gain in prediction quality at the same memory and training-step budgets. This complements the paper's token-efficiency results against full-history prediction.