- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper proposes NoisyAgent, an agentic reinforcement learning framework designed to improve LLM agent robustness under noisy environments. It introduces user-side and tool-side perturbations during training and employs a hybrid training strategy that combines clean and noisy rollouts with group-wise advantage normalization. An adaptive scheduling strategy progressively increases noise difficulty and proportion based on the performance gap between clean and noisy environments. Experiments on AgentNoiseBench and standard agent benchmarks demonstrate improvements in both noisy and idealized environments.
Critical strengths
- Practical motivation: The paper addresses an important limitation of existing agentic RL training, which typically assumes idealized environments. Incorporating realistic interaction imperfections into training is a meaningful direction for improving agent robustness.
- Effective noise-aware training: The proposed approach consistently improves performance across different benchmarks and model scales, demonstrating the benefits of incorporating noise into agentic RL training.
Critical weaknesses
- Definition of noise severity: The definition of the “severity” of each noise type is unclear. Noise severity could be defined by how much a perturbation changes the original user instruction or tool outcome, or by how difficult the resulting task becomes. However, the paper lacks clear mechanisms for controlling noise severity during generation and validating the severity of generated perturbations. This makes it unclear how noise difficulty is progressively increased during scheduling.
- Realism of noise: The authors do not empirically evaluate how realistic the generated noise is. Without such validation, although the proposed method demonstrates strong performance in the specific experimental settings, it remains unclear whether the generated perturbations adequately reflect real-world interaction patterns and whether the resulting robustness improvements would generalize to real-world environments.
- Details of scheduling: Although the authors define noise scale and describe the criteria for increasing noise difficulty, the exact update rules are not sufficiently specified. In particular, the initial noise configurations, increment sizes, and update frequency remain unclear. Providing these details would improve the clarity and reproducibility of the proposed method.
- Effectiveness of scheduling: The results reported in Table 3 show that joint training without scheduling achieves relatively comparable performance to the full method. Although additional experiments in Appendix B.2 show consistent improvements, the incremental benefits of adaptive scheduling appear relatively modest compared with those of noise injection itself. Additionally, the fixed noise difficulty and noise proportion used in the ablation are not reported, making it difficult to assess the effectiveness of the proposed scheduling strategy.
- Ablation on noise types: Overall, training with noise improves final performance. However, the individual contributions of different noise types to the performance improvements remain unclear. In particular, the paper does not provide separate ablation studies for user-side and tool-side noise, making it difficult to assess the effectiveness of each component.
- Validation of task solvability: Although the authors claim that noise injection preserves task solvability, they do not explain how this property is ensured or verified. Providing details on the verification procedure would improve the clarity and reproducibility of the proposed method.
Questions for authors
- How is the severity of each noise type defined and quantified? How do the authors control and validate noise severity during generation?
- How do the authors ensure that the generated noise realistically reflects real-world user interactions and tool failures? Could the authors provide empirical validation, such as human evaluations or comparisons with real-world interaction data?
- Could the authors clarify the exact scheduling procedure, including how noise scale and difficulty are updated when the progression criterion is satisfied? What are the initial values, increment sizes, and update frequency?
- Given the relatively modest performance improvements from scheduling in Table 3 and Appendix B.2, could the authors provide further evidence of the effectiveness of adaptive scheduling? Also, what fixed noise difficulty and proportion were used in the w/o scheduling ablation?