- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CRISP, a decentralized online prompt optimization framework for multi-agent LLM social simulations. The method combines behavioral strategies with cooperation mechanisms, employs an opponent-aware surrogate for reward and action prediction, and introduces a physics-inspired controller to adapt exploration based on collective cooperation dynamics. The paper further provides regret and coarse-correlated equilibrium analyses and outlines evaluations on Sotopia and CoopEval.
Critical strengths
-
The paper addresses the challenges of online strategy optimization in multi-agent settings where agents continuously adapt to each other's behavior.
-
Combining behavioral strategies with explicit cooperation mechanisms provides a meaningful extension to conventional prompt optimization.
Critical weaknesses
-
The main experimental results, including comparisons on Sotopia and CoopEval and component ablations, are explicitly marked as provisional placeholders. Without completed experiments, the claimed improvements over existing methods cannot be substantiated.
-
The proposed exploration schedule relies on a Curie--Weiss mean-field approximation, but the paper provides limited evidence that real LLM interactions exhibit the assumed phase-transition dynamics. It remains unclear whether this complex controller offers meaningful advantages over simpler adaptive exploration schedules.
-
The regret and CCE guarantees rely on assumptions about surrogate approximation errors and specific update schedules that do not fully match the practical algorithm. In particular, the theoretical analysis assumes no forgetting, whereas CRISP introduces a time-varying recency factor, making the applicability of the stated guarantees unclear.
Questions for authors
-
Can the authors provide completed experimental results and ablations demonstrating that CRISP consistently outperforms existing methods, particularly isolating the contribution of the critical-manifold controller against simpler adaptive schedules?
-
Can the authors clarify how the theoretical regret and CCE guarantees extend to the actual CRISP algorithm with surrogate prediction errors, time-varying learning rates, and a non-unit recency factor?