- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Mutual Recursive Improvement (MRI), a framework for adapting cooperation among frozen LLM agents through an interaction-reflection-revision cycle. The framework exposes agents to five human-inspired interventions: Counterparty Review (CR), Self-Reflection (SR), Public Contracting (PC), Case-Based Reasoning (CBR), and Experience Discounting (ED). The authors evaluate cumulative configurations of these interventions across eight LLMs, three market-conditioned economic games, and 50 rounds derived from quarterly SPY returns. The evaluation considers allocation quality, collective surplus, regime-aligned cooperation, and payoff dispersion.
Critical strengths
-
The paper studies a meaningful problem: how LLM agents with different interests adjust their cooperation as external conditions change. This matters for multi-agent deployment because strong individual agents do not necessarily coordinate well as a group.
-
The paper is clearly written. The framework, intervention levels, metrics, and experiments are easy to follow, and the authors report negative results such as the degradation caused by routine self-reflection rather than presenting every intervention as beneficial.
Critical weaknesses
-
MRI combines familiar ideas such as reflection, memory, case support, communication, and contracting, but the experiments do not isolate what the full recursive loop adds beyond these components. This concern is important because CBR is the highest-scoring standalone intervention in 22 of 24 model-game rows, while standalone and composed CBR are applied at different points in the decision process. Without matched ablations and budget-controlled baselines, the claimed method-level novelty remains unclear.
-
The empirical scope is limited. Each configuration is reported on one fixed 50-quarter market sequence without uncertainty across repeated executions, and the heterogeneous evaluation is confined to one Trust-game setting. It is therefore unclear whether the gains are stable across runs and market paths, or whether they support the broader claim of co-evolving agent societies.
Questions for authors
-
Do the authors have results that separate the contribution of the full recursive loop from CBR, such as L5 without CBR, L4 with ED while retaining SR, or L4 without SR? If not, what existing evidence supports attributing the gains to the recursive loop rather than the fixed case library?
-
Were the main B0/L1/L4/L5 comparisons repeated under the same settings? If so, please report run-level variation or uncertainty. If not, how sensitive might the reported 23/24 and 22/24 counts be to repeated execution or a different market path?
-
How do model-call count, context length, and token usage differ across B0-L5? Have the authors tested a budget-matched deliberation baseline or a simple regime-rule prompt baseline that could rule out additional prompting budget as the main explanation?
For revision, the paper would also benefit from reporting the construction criteria and full contents of the CBR case library, distinguishing standalone CBR from composed CBR whenever their results are discussed, using “allocation quality,” “collective surplus,” and “mutual benefit” consistently, and defining B0-L5 once in a compact table with consistent notation across the main text, captions, and appendix. These are presentation and reproducibility suggestions rather than separate reasons for rejection.