- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper introduces ReGain, which is an adversarial learning approach to improve reasoning in multimodal large language models used for GUI agents. Instead of learning only a critic over the trace source, they also distinguish between Chain-of-Thought responses and responses without Chain-of-Thought. They report results on a single model for a GUI task, where it mostly outperforms the RARO and SFT baselines.
Critical strengths
S1) Motivation: Overall, I like the paper and the approach. It is well motivated by related work (e.g., DAPO) and its current failure modes, and it explains how to address them. I find their reward gain approach sensible.
S2) Scientific Reporting: I appreciate that the authors are reporting standard errors to demonstrate the uncertainty of their experiments. Unfortunately, this is often not the case in reviews today. I thank the authors for this, as it makes the results easier to interpret.
S3) Baselines: I appreciate the baselines the authors use. While there are obviously more adversarial reasoning approaches (e.g., rePIRL, GAD, R-AIRL, GAR, ...), it makes sense to compare against RARO, which has different tuning parameters. However, I would maybe discuss these works in the related work section.
Critical weaknesses
W1) Dataset / Task: My biggest concern with this paper is that it tests only a single model on a single GUI dataset. I almost misinterpreted the appendix results as ReGain being trained on different datasets, but I understand it is only tested after being trained on the GUI Agent dataset. I believe, however, that this method is not necessarily limited to GUI Agents, and it would be interesting to see whether it also generalises to other reasoning datasets, such as MATH, MMLU-Pro, etc., and what the gains are over SFT and RARO.
Questions for authors
Q1) Stability What stability methods were used to address the non-stationarity of the method (e.g. replay buffer, KL regularisation, etc.)
3 comments
Thank you to the authors for their response. Overall, I think the paper is in good shape, with a well-motivated, clear method and results to validate their claims.
I understand that the authors do not have time to test their method beyond GUI control, and they do not claim to do so in the paper. I would nevertheless encourage the authors to expand their experiments to more general use cases, as this could make the paper even stronger. Despite this point, they clarified my questions, and I increased my score.
I wish the authors good luck with their work.
Thank you very much for reviewing our response and revised manuscript, and for raising your score to 4. Your pointers on related adversarial and inverse-RL reasoning works, stabilization details, and broader evaluation helped us make the positioning and empirical section much clearer. Your clear acceptance with high confidence has indeed greatly encouraged us.
Response to Reviewer i5wz
Thank you for the positive evaluation and helpful references.
[S3] Related adversarial and inverse-RL reasoning works. We added these works in §2 (Related Work): rePIRL (Wu et al., 2026) and R-AIRL (Fanconi et al., 2025) use inverse RL to learn process-level reward models from expert trajectories or CoTs, GAD (Ye et al., 2025) trains a sequence-level discriminator as an on-policy reward for black-box distillation, and GAR (Liu et al., 2025) co-trains a reasoner with a slice-level adversarial discriminator. In all of these methods, the discriminator or reward model is scored only on its final classification, without comparing against a no-reasoning judgment.
[W1] Evaluation beyond a single dataset. During the rebuttal period we prioritized adding public evaluation and online RL continuation on MobileGym-Bench [1] (256 test tasks, 4 trials per task; new §5.5, Figure 3):
We leave extending the gain-from-reasoning formulation to general text-reasoning benchmarks (such as MATH and MMLU-Pro) as a direction for future work, as the present study focuses on demonstration cold starts for GUI agents ahead of sparse-reward online RL.
[Q1] Stabilization techniques. Across all adversarial runs, we address co-training non-stationarity through six mechanisms (detailed in Appendix J, Table 3):
Beyond these shared settings, the gain reward itself helps prevent critic collapse: at step 160, ReGain maintains higher critic advantage variance (0.11 vs. 0.05 for discrete c=0.75 and Brier-only) and avoids tie collapse (35% ties vs. 64% for Brier-only). Late-stage style overfitting (after step 320) is controlled via a fixed training budget.
[1] Wu et al. 2026. MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research. In Proceedings of EMNLP 2026, pages 23223-23247.