- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ForeCache is a training-free method for reducing repeated processing of shared history in LLM-based multi-agent systems. It combines consumer-conditioned saliency (CCS), which keeps history relevant to the next agent’s task, with divergence-guided recomputation (DGR), which reuses earlier KV-cache states and selectively updates them when the new agent’s context changes.
The authors compare ForeCache with full prefilling and two cache-reuse baselines on GSM8K, MMLU, and HumanEval using two to six agents. ForeCache generally maintains accuracy close to full prefilling. In a six-agent AIME-24 efficiency test, it reports a 3.4× aggregate speedup and a 6.6× speedup in the final answer agent’s time to first token. Ablations examine the contributions of history selection and cache recomputation.
Critical strengths
- The paper provides useful evidence for both parts of ForeCache. In the Swap experiment, changing only whose instructions guide history selection lowers evidence recall from 87% with the intended consumer’s prompt to 66% with another consumer’s prompt and 61% with producer attention. This directly supports selecting history for the agent that will use it. The ablation also shows why KV repair matters: direct cache transfer reduces accuracy by 3.8–9.2 percentage points across the three tasks, while DGR recovers performance close to full prefilling. These are the paper’s strongest results because they show that both components have a substantive effect, even though the broader accuracy and speed claims need stronger validation.
Critical weaknesses
-
The evaluation is too narrow to support the paper’s generality claims. Task accuracy is measured on three benchmarks using two 7–8B models, but no benchmark is run on both models, so model and task effects cannot be separated. The experiments also use one fixed sequential agent setup. The headline 6.6× speedup comes from a 30-problem AIME-24 test without task accuracy; speedups are smaller where accuracy is measured. A stronger test would include a task where multi-agent communication clearly helps and report both accuracy and latency.
-
The comparisons are also difficult to interpret because the evaluation sets are small and most results have no reported variance. MMLU has only 157 questions and HumanEval 164, so a one-point difference can represent just one or two cases. Only one table reports repeated runs, and its statistical tests compare ForeCache with Full Prefill rather than with the cache-reuse baselines. Larger evaluation sets, repeated runs for the baselines, and paired comparisons would make the reported gains more convincing.
Questions for authors
-
On AIME-24, what is ForeCache’s final-answer accuracy compared with Full Prefill? Does the reported 6.6× TTFT speedup include the cost of probing, pruning, and cache repair?
-
How much does retaining older agent outputs help compared with keeping only the immediately preceding output?