acceptodds
Under review as a conference paper at ICLR 2027

CRISP: Critical-Manifold Informed Strategic Play for Multi-Agent LLM Social Simulation

Abstract

Cooperation among self-interested learning agents is a central question for LLM-based social simulation, yet the online prompt-optimization tools that steer such agents are built for single learners against stationary environments. When the same online learner is deployed at every seat of a multi-agent conversation, opponent policies co-evolve, cooperation-inducing mechanisms lie outside the arm space, and hand-tuned exploration schedules leave agents locked into defect-dominant equilibria. We introduce CRISP (Critical-manifold Informed Strategic Play), a decentralized online learner that lets every LLM agent jointly search over behavioral strategies and textual mechanism proposals (repetition, reputation, mediation, contract), and that drives the sampling temperature, step size, and recency factor from the empirical distance to a phase-transition manifold estimated online. Under this construction, the exponential-weights potential admits a formal analysis: individual regret is up to a surrogate approximation term, and the empirical joint play of all agents converges to an -coarse-correlated equilibrium, without any communication among agents. A physics-inspired controller derived from a Curie–Weiss free energy concentrates exploration precisely where a per-turn reward has non-trivial influence on the collective norm, preserving the convergence rate while eliminating manual scheduling. We specify an evaluation on Sotopia and CoopEval, spanning seven social dimensions and four social dilemmas, with component ablations designed to test whether CRISP improves cooperation under co-adaptive partners.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_6YcqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CRISP, a decentralized online prompt optimization framework for multi-agent LLM social simulations. The method combines behavioral strategies with cooperation mechanisms, employs an opponent-aware surrogate for reward and action prediction, and introduces a physics-inspired controller to adapt exploration based on collective cooperation dynamics. The paper further provides regret and coarse-correlated equilibrium analyses and outlines evaluations on Sotopia and CoopEval.

Critical strengths
  1. The paper addresses the challenges of online strategy optimization in multi-agent settings where agents continuously adapt to each other's behavior.

  2. Combining behavioral strategies with explicit cooperation mechanisms provides a meaningful extension to conventional prompt optimization.

Critical weaknesses
  1. The main experimental results, including comparisons on Sotopia and CoopEval and component ablations, are explicitly marked as provisional placeholders. Without completed experiments, the claimed improvements over existing methods cannot be substantiated.

  2. The proposed exploration schedule relies on a Curie--Weiss mean-field approximation, but the paper provides limited evidence that real LLM interactions exhibit the assumed phase-transition dynamics. It remains unclear whether this complex controller offers meaningful advantages over simpler adaptive exploration schedules.

  3. The regret and CCE guarantees rely on assumptions about surrogate approximation errors and specific update schedules that do not fully match the practical algorithm. In particular, the theoretical analysis assumes no forgetting, whereas CRISP introduces a time-varying recency factor, making the applicability of the stated guarantees unclear.

Questions for authors
  1. Can the authors provide completed experimental results and ablations demonstrating that CRISP consistently outperforms existing methods, particularly isolating the contribution of the critical-manifold controller against simpler adaptive schedules?

  2. Can the authors clarify how the theoretical regret and CCE guarantees extend to the actual CRISP algorithm with surrogate prediction errors, time-varying learning rates, and a non-unit recency factor?

Reviewer_j61xOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Summary not provided.

Critical strengths

Not provided.

Critical weaknesses

The paper should be desk-rejected. I have alerted PC, SAC, and AC but received no response, so I am also posting the review to flag this.

The paper doesn't contain an AI Use statement section and should be desk-rejected.

Additionally, I would also like to flag this as a heavily negligent submission:

Lines 319 and 411 mark the current data in Tables 2 and 3 as "provisional templates awaiting completed runs". In my experience, this happens when AI writes the intermediary version, and authors forget to remove it. This means that authors didn't even proofread their work.

In addition, the paper seems to be a reproduction of the paper ALSO: Adversarial Online Strategy Optimization for Social Agents, but they don't cite it.

Questions for authors

Not provided.

Reviewer_3wTTOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents CRISP, an approach to prompt-tuning in multi-agent settings. In the CRISP architecture, each agent learns over a finite set of possible prompts, where at each iteration prompts are chosen using the Exp3 adversarial bandit algorithm. Because this algorithm can only guarantee convergence to coarse-correlated equilibria (CCEs), which may be non-cooperative, they modify Exp3 with a physics-inspired adaptive mechanism that adjusts the learning rate and exploration temperature in regions of the strategy space where greater exploration may help the agents find a more cooperative strategy.

Experimental results on two LLM cooperation benchmarks, SOTOPIA and CoopEval, demonstrate that CRISP outperforms other multi-agent prompt-tuning baselines, as well as the underlying untuned LLMs, across multiple measures of effective cooperation.

Critical strengths
  1. The empirical results show the CRISP architecture does outperform the base LLMs and prompt-tuning baselines on the two cooperation benchmarks, at least b a small margin. Thorough ablations also demonstrate that each element of the complex architecture contributes to this performance.
  2. The phase-transition based exploration mechanism is interesting on its own, and might be applied to other architectures, such as existing cooperative multi-agent RL algorithms.
Critical weaknesses
  1. My main concern is that the core theoretical result, Theorem 4.1, appears to be incorrect. While the regret bound (Equation 8) appears to hold for the learning rate schedule specified in the theorem itself, the reduction to a CCE does not seem to be well-defined, because it is unclear what game the CCE is referring to. The regret bound in Equation 8 refers to the reward as a function of the state and action, while the definition of the the CCE in Equation 3 drops the dependence on the state. The game being played at each time-step tt will change as the state changes, in a way that is correlated with the actions, and so there would seem to be no guarantee that the "average" strategy corresponds to a CCE of the "average" game. The proof does not resolve this issue. Furthermore, while the paper suggests that the adaptive learning rate (controlled by the phase-transition mechanism) should not break the regret guarantees of theorem 4.1, this is not proven, and it seems likely that there would be cases where the adaptation could cause excessive regret.
  2. The value of the algorithm appears to be limited in practice, as it can only perform "prompt tuning" over a finite number of hand-written prompts. Given that each prompt provided encodes a great deal of human knowledge about what will ensure cooperation, it seems that a more effective approach to tuning would be to simply test each prompt separately (giving all agents the same prompt). It is also unclear that a random mixture over prompts, which is what CRISP ultimately learns, would be better solution to the benchmark tasks than a deterministic prompt (thought it could conceivably be more exploitable in CoopEval).
Questions for authors
  1. Given that the surrogate function is (appears to be) trained to predict immediate reward, as opposed to total future return, what is the advantage of using the surrogate reward to estimate the value of an action, rather than using the observed immediate reward? It is possible that I have misinterpreted the reward signal used in the surrogate training loss in Equation 5.
  2. As I understand it, a complete run of CRISP in the benchmark tasks would only take 20 actions in total. As there are 60 (12 x 5) hybrid prompts defined, does this mean that the majority of the prompts would never be evaluated during learning?
  3. How many different runs (random seeds) were used for each experiment?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.