- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies interference caused by accumulated reasoning text in multi-step multimodal agents and proposes SPARE for managing reasoning history. After generating a temporary task-state summary, the model replays historical reasoning with and without summary conditioning and uses a truncated reverse-KL score to measure token-level sensitivity. Low-sensitivity segments are removed, while high-sensitivity segments are compressed through rule-based extraction of visual evidence. Original tool calls, observations, and images are preserved. The authors also investigate SFT for improving summarization and evaluate accuracy, reasoning-token pruning, and textual interference across three backbones and multiple benchmarks.
Critical strengths
SPARE preserves original visual and tool evidence while modifying only self-generated reasoning history. This focuses the investigation on managing potentially misleading textual context rather than further compressing visual inputs that may already receive insufficient attention.
For the 30B backbone, accuracy improves by 13.56 percentage points on GTA and 12.72 percentage points on MNMS. Comparisons with complete deletion, random deletion, evidence-only reconstruction, and KL-guided selection provide initial support for selective compression over indiscriminate removal.
Critical weaknesses
Insensitivity to adding a summary does not imply insensitivity to deleting historical reasoning. The implemented score additionally relies on top-20 truncation, a fixed fallback for unavailable teacher log probabilities, and event-level normalization. Although the appendix appropriately acknowledges that the theoretical bound does not justify the actual deletion rule, the experiments do not sufficiently establish whether the score reliably predicts the consequences of removing a segment.
In Table 1, average accuracy improves by only 0.32 percentage points for the 8B backbone and decreases for Nemotron; the 30B backbone also shows a substantial decline on V*. Meanwhile, each pruning event requires summary generation and two forward replays, whose costs are excluded from the reported persistent-history pruning rate. It therefore remains unclear under which tasks and trajectory lengths SPARE provides a reliable accuracy–compute advantage.
Questions for authors
Have you measured the relationship between the KL score and the actual change in answer quality after deleting a segment? At matched retained-context lengths, does the proposed selection rule consistently outperform random selection, conventional summarization, or simpler importance heuristics?
What are the changes in end-to-end latency, total processed tokens, or FLOPs when summary generation and both replay passes are included? Could you also provide confidence intervals for the main accuracy differences to establish whether the smaller gains are robust?