- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The work proposes a skill evolution method through the MiniMax framework, where the agent tries to maximize the reward from the current state, and the same agent plays the opponent’s reply and picks the one that is worst for itself (adversarial simulation). Once the games are done, an evaluator agent audits the logs and improves the skill and the harness around the agent.
Critical strengths
- The work addresses a practical problem of harness evolution. Adversarial review, in general, seems to improve results across many domains.
- The paper reports clear improvements in chess with Rep-think, and it is intuitive how the methods help avoid clear blunders.
Critical weaknesses
- Frontier models are not evaluated. The work only evaluates DeepSeek v4 Flash and MiniMax 2.5. Ideally, we want the models to use test-time compute to simulate before taking actions. So, evaluating frontier models with adversarial reviews would help us identify whether separate simulation improves performance or not.
- Lack of novelty. Simulating a step before taking a decision is a well-known method. It is hard to pinpoint how the Mini-Max framework is different from adversarial simulation. Appendix E provides evaluator instructions, but the complete player and baseline prompts are not provided, which makes it difficult to understand how the authors are injecting the MiniMax narrative.
- Minimal information is provided about the evaluation protocol of baselines. In this line of work, it is very important to compare different methods with tuned hyperparameters. Also, trade-offs in inference compute between Rep-think and other variants are not discussed.
- The paper is hard to follow. Figure 2 has overlapping text and arrows.
Questions for authors
- How are MiniMax simulations different from normal adversarial simulations?
- I see adversarial simulations as a form of test-time scaling method. Instead of scaling depth, can we scale the width during each play? i .e., sample n times -> then do adversarial review for all the moves; this can be one agent given all moves to choose the best move or different agents doing adversarial simulation for each unique move, then choose the best possible move. Compare compute-matched depth vs. width methods.