- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2(−1)
- Soundness
- 2(+1)
Read review
Paper summary
This paper explores how post-training amplifies the memory curse phenomenon in multi-agent LLM cooperation. The authors find that (1) post-training does amplifies the memory curse, i.e., longer memory results in lower cooperation rate, and (2) post-training makes the models more sensitive to changes in payoff.
Critical strengths
S1. This paper explores an interesting research question.
S2. The writing is good and easy to follow.
S3. The experiments are extensive: 21 checkpoints, 7 model lineages, and 7 social-dilemma games.
Critical weaknesses
W1. It has not been answered through what kind of mechanism post-training amplifies the memory curse.
W2. The authors present a testable explanation that post-trained models are less likely to recover cooperation. This implies post-trained models should respond to earlier defection more negatively than the Base checkpoints, which, however, is contrary to the results shown in Table 25.
W3. The paper does not provide any preliminary experiments on the instruction-following capabilities of the Base checkpoints. The authors acknowledge that swapping actions and testing payoff comprehension would be helpful to address this concern.
Questions for authors
See W2.
5 comments
Response to Reviewer ZfSj (2/3)
(continued)
W2 · Why Table 25 does not contradict the recovery hypothesis
Table 25 was presented as a mechanistic exploration, not as a test of a universal slower-recovery claim. Section 4 proposes a withdrawal–recovery feedback hypothesis: when one agent defects, others may reduce their cooperation in response, potentially reinforcing mutual defection. Appendix J examines a narrower question: whether earlier defections still influence the next decision after a scripted sequence of cooperative rounds. Because those cooperative rounds are provided by the experiment rather than generated through interaction, Table 25 measures sensitivity to older defections, not the process of recovering cooperation.
Importantly, being less sensitive to past defections does not necessarily mean recovering cooperation more easily. For example, a model might (i) withdraw cooperation more strongly when a partner first defects, even if it later becomes less sensitive to that event; (ii) hesitate to cooperate again because it expects the other agent to make the first move; (iii) enter a feedback loop where its own reduced cooperation leads to further unfavorable responses and outcomes, making cooperation increasingly difficult to restore; or various other possibilities. These processes unfold across multiple rounds and cannot be inferred from a single decision after a scripted cooperative history.
Therefore, Table 25 does not logically contradict the proposed withdrawal–recovery hypothesis. Moreover, our new Always-Cooperative Partner Control (Table R1 in our above rebuttal) shows that the amplified history effect can arise even without partner defection, pointing to broader history-dependent behavioral mechanisms. We will clarify the exploratory role of Table 25 in Appendix J and distinguish these potential mechanisms from the directly measured stage-by-history findings in Section 4.
Response to Reviewer ZfSj (3/3)
(continued)
W3 · Base model-compatible decision protocol and label-swap controls
Our evaluation is designed to make Base/SFT/Aligned comparisons less dependent on instruction-tuned answer formatting. Instead of asking a model to produce a long rationale and append an exactly formatted action in one response, we use the two-request rationale-to-action protocol (Appendix M.1). This separates generating an explanation from selecting a legal action and avoids extracting decisions from free-form prose.
Phase 1. Generate a rationale. The model receives the game rules, visible interaction history, and legal actions, then is asked to “Briefly state your strategic rationale in at most X words.” It generates ordinary text with a predefined token budget, while no final-answer is required in this phase.
Phase 2. Select an action. A new model request repeats the game and history, includes the generated rationale as an assistant turn, and asks, “Now commit to one action for this round: ” vLLM choice-constrained decoding restricts the final output to that game's legal action labels (e.g., A0/A1 in the PGG template), with an eight-token cap. Importantly, the decoder enforces only a valid response format: it leaves every legal action selectable and does not determine whether the model cooperates. Thus, the cooperation measure comes from the model's semantic action choice, not a parser's interpretation of unrestricted text.
We validated the actual final-action readout. In the audited repeated-CPR Base slice, 336,000 accepted action choices had zero logged final-action parse failures. Rationale retries are tracked separately, and the runner retains the first valid rationale/action rather than inserting a default action. These counts directly address whether Base decisions can be converted reliably into legal actions under the executed interface.
We also tested Base checkpoints on questions with objectively correct answers. Across all these checkpoints, we separated copying, selecting the larger explicitly stated payoff, and calculating a payoff from the rules. On the explicit-payoff comparison, Nemotron-Cascade Base achieved 94.58% and OLMo 3 Base 81.67% correct choices under the constrained readout. Both answer options were legal, so these scores reflect selecting the correct option, not simply satisfying the output constraint. These task-specific tests provide evidence of payoff comparison capability in Base models, complementing the interface audit.
Finally, we tested whether the stage-by-history effect depends on a particular action label. In the label-swap controls, the meanings assigned to legal labels were exchanged, and cooperation was scored by the underlying action, not the token string. For each checkpoint, define H = Coop(HL40) − Coop(HL2), and the within-lineage contrast K = H(post-trained) − H(Base). A negative K means longer histories have a more negative effect after post-training than at Base. Table R2 compares the original lineage-level K values and new label-swap estimates.
Table R2. Stage-relative history contrast K (mean ± reported SD; percentage points).
| Model lineage | Paper Original K (SFT − Base) | Paper Original K (Aligned − Base) | Label swap K (SFT − Base) | Label swap K (Aligned − Base) |
|---|---|---|---|---|
| SmolLM2 | −16.2 ± 2.4 | −18.1 ± 2.4 | −8.15 ± 2.27 | −8.10 ± 1.88 |
| Nemotron-Cascade | −4.9 ± 2.9 | −13.7 ± 3.8 | −6.73 ± 6.51 | −27.22 ± 6.02 |
| OLMo 3 | −15.2 ± 2.8 | −10.7 ± 2.0 | −15.92 ± 19.82 | −35.78 ± 15.97 |
All six new label-swap mean K contrasts are negative across SmolLM2, Nemotron-Cascade, and OLMo 3, matching the direction of their original lineage-level contrasts. Taken together, the two-stage interface, verified action readouts, task-specific Base capability checks, and action-label controls make the stage-by-history comparisons interpretable without assuming that Base checkpoints have instruction-tuned response-formatting behavior. We will make the executed two-request protocol and audit denominators explicit in Appendices M/N and add the capability and label-swap controls to the final version of the paper.
Thank you again for your thoughtful feedback. We hope our new results and clarifications address your concerns, and would greatly appreciate it if you could consider revisiting your rating.
Response to Reviewer ZfSj (1/3)
We thank the reviewer for recognizing the importance of the research question, the clarity of the writing, and the extensive evaluation across 21 checkpoints, seven lineages, and seven social dilemmas.
W1 · Behavioral pathway: history-conditioned cooperation beyond partner defection
CoopShift's central contribution is to identify how the effect of visible interaction history changes across post-training stages, rather than only whether longer histories reduce cooperation. We compare 21 checkpoints from seven model lineages across seven social dilemmas, with matched Base, SFT, and Aligned evaluations. In Table 3, the mean history effect H = Coop(HL40) − Coop(HL2) across six binary-action games is −1.17 percentage points (pp) at Base, −8.11 pp at SFT, and −10.93 pp at Aligned; the paired SFT-minus-Base and Aligned-minus-Base contrasts are −6.93 pp and −9.76 pp. Most of the average amplification appears by SFT, identifying a key training stage at which history-conditioned cooperation changes. Our payoff and controlled-partner analyses further characterize this effect under different incentives and partner behaviors.
These findings supply both a mechanistic research direction and practical evaluation principles. For agentic post-training, our results motivate assessing (i) whether agents sustain beneficial cooperation as their interaction histories grow, and (ii) whether contributions remain appropriate to payoff incentives and opportunities for reciprocity (Table 2; Appendix U). The concentration of amplification by SFT also motivates evaluating intermediate checkpoints alongside final Aligned ones, including as possible starting points for collaboration- or incentive-aware post-training. A targeted next step is to vary the focal agent's earlier actions, realized payoffs, and partner histories independently while holding the current game state fixed, to identify which history-feedback channels shape later contributions.
To probe the behavioral explanation, we ran a new Always-Cooperative Partner Control to test whether the amplification requires observing partner defection. The focal LLM controls one player in a three-player repeated game while two scripted partners cooperate throughout, including an initial 40-round warm-up. The focal agent still chooses its own actions, and those actions and its realized payoffs enter later histories. We compared HL2 and HL40 across SmolLM2 and DeepSeekMath at Base, SFT, and Aligned stages, six payoff settings spanning PGG, CPR, and Donation-PD, and ten fresh seeds; cooperation is measured in rounds 47–100.
Table R1. Always-Cooperative Partner Control. H = Coop(HL40) − Coop(HL2), averaged over rounds 47–100 and six payoff settings (mean ± sample SD over ten seeds; pp). Both programmed partners cooperate at every round.
The always-cooperative control sharpens the behavioral explanation. Across both lineages, the two Base history effects are near zero, whereas all four post-trained checkpoints show negative mean effects (−20.23 to −15.62 pp) despite partners never defecting. Thus, observed partner defection and retaliation against it are not necessary for the average amplification in these settings. Because the focal agent's own actions and payoffs still enter its growing interaction record, history-conditioned incentive adaptation is a concrete behavioral pathway to investigate. The finding directs attention to how post-trained agents use accumulated experience when deciding whether to continue contributing, even with reliable partners.
The payoff consequences make this behavior strategically important. In the always-cooperative PGG control (r = 1.2), we compare DeepSeekMath SFT's individual and total group payoffs per round under HL2 and HL40. The longer history coincides with a higher individual return but lower total group return, illustrating why both individual and collective incentives matter alongside cooperation rates.
We will revise Section 4 and the Discussion to foreground this stage-resolved contribution, the behavioral insight from cooperative-partner controls, and the resulting evaluation and future training directions.