acceptodds
Under review as a conference paper at ICLR 2027

Learning to Spray in Uncertain and Windy Environments

Abstract

Deploying robotic systems for precision spraying offers a sustainable alternative to blanket spraying in agriculture. However, existing systems typically assume deterministic kinematics and perfect observations, ignoring the inherent stochasticity of agricultural fields such as unpredictable wind patterns, sensor noise, and variable spray-dispersion dynamics. To bridge this gap, we introduce a novel stochastic multi-robot environment for precision spraying that explicitly models real-world uncertainties: stochastic wind, actuation noise, observation noise, and probabilistic spray decay. We design a spraying mechanism based on wind-dependent Gaussian falloff that models advective spray drift and anisotropic plume elongation caused by variable wind patterns, and calibrate its parameters against published field measurements of UAV spray deposition. We formulate the task as a constrained stochastic optimization problem for coordinating robots to treat infected locations under uncertainty while minimizing path lengths and avoiding collisions. We constructed both centralized (single-agent) and decentralized (multi-agent) POMDPs that share the same global state space, dynamics, and reward function for performing this task. Our environment is compatible with standard single-agent and multi-agent deep Reinforcement Learning (RL) methods and serves as a rigorous benchmark for robust agricultural coordination, which can be modified or extended for other similar applications with real-world uncertainties. We evaluated centralized and decentralized deep RL methods and a model-based controller across multiple environment variations, demonstrating the feasibility of learning effective policies under various uncertainties.

Primary area:
reinforcement learning
Keywords:
reinforcement learning, precision spraying, multi-agent systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Rd8MOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a stochastic multi-robot reinforcement-learning (RL) environment for selective (precision) agricultural spraying. It models four uncertainty sources — temporally correlated (AR(1)/Ornstein–Uhlenbeck) wind, actuation noise, observation noise, and stochastic spray deposition — plus a wind-dependent Gaussian deposition kernel that advects with the wind and elongates along the wind direction. The kernel parameters (still-air dispersion beta_s, mean fall time t_bar_w, fall-time std sigma_dr, wind attenuation eta_w) are calibrated by least squares against a published UAV field-deposition model (Martin et al., 2025). The spraying task is formalized as a chance-constrained stochastic optimization problem and instantiated as both a centralized single-agent POMDP and a decentralized Dec-POMDP that share the same state space, dynamics, and reward. The authors evaluate centralized PPO/TRPO, decentralized MAPPO/VDN, and a hand-designed model-based controller across team sizes N in {2,3,4,5} and 10 environment variations, with appendices adding a 10-learner benchmark and a scalability study to N=8.

Critical strengths
  1. Strong, well-controlled experimental design and statistics. The centralized and decentralized formulations share state/dynamics/reward by construction, so any performance gap isolates decentralized execution rather than a task change. Combined with held-out seeds, IQM with stratified-bootstrap CIs, paired comparisons, and explicit reporting of checkpoint-selection optimism, the results are trustworthy.
  2. A genuinely useful robustness study. The observation-gap analysis (Tables 3/19/20) shows decentralized MAPPO loses at most ~2% of return under inference-time sensor perturbations, while VDN degrades substantially and the model-based controller collapses under 5 m position noise. This is a concrete, transferable insight and the strongest empirical contribution.
Critical weaknesses
  1. The central premise is undercut by the paper's own calibration and ablations. Calibration leaves sigma_dr and eta_w unidentified and sets both to zero, so the heavily advertised "anisotropic plume elongation and wind attenuation" reduces to an isotropic Gaussian advected by t_bar_w*w. More seriously, the uncertainty ablations (Tables 24/26/27) show every uncertainty source — wind, actuation, observation, and even full determinism — yields statistically indistinguishable returns (MAPPO 3416–3429, VDN 3303–3370; AR(1) vs. i.i.d. wind identical). If training without any noise loses nothing, the environment does not actually stress-test robustness, and the motivating claim that ignoring these uncertainties produces brittle policies is not empirically demonstrated. This directly questions the paper's value as a "rigorous benchmark for robust coordination."
  2. Limited task difficulty and evidence of memorization. The infection map, sites, and intensities are fixed and known per variation (only initial positions and initial wind are randomized) across just 10 hand-designed fields. The observation ablation shows learners barely use the infection map because they "memorize the layout" (removing it costs ~0.2%). This suggests the benchmark may measure memorization of map geometry rather than generalizable coordination under uncertainty, limiting the strength of the coordination claims.
Questions for authors
  1. Given that all calibrated uncertainty sources are effectively inert, what noise magnitudes would make the task meaningfully stochastic, and can you quantify the environment's "robustness envelope," which the paper states is unknown?
  2. Since the kernel degenerates to isotropic advection under calibration, is the elongation/attenuation machinery ever exercised in any reported result?
  3. Do you expect the memorization behavior to persist under procedurally randomized maps and sites, and how could the environment be changed to prevent it?
Reviewer_guaeOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a new RL and MARL benchmark for UAV-based spraying of plants. The model includes wind, spraying plumes, and a position-velocity-acceleration model with noise. Reward engineering has been applied to make the environment solvable via standard RL algorithms. A variety of RL algorithms have been compared under the environment together with a few observation and reward ablations, showing the necessity of the reward formulations.

Critical strengths
  • The application area is in classical UAV swarms, and the synthesis of ideas makes sense.
  • The presentation of the work looks clean, figures are legible etc.
  • The experiment section is quite comprehensive, including various MARL algorithms and ablations.
Critical weaknesses
  • As an application or benchmark paper, the work seems to be based on public traces and has not been verified directly on a robotic platform, so the Sim-to-Real-World gap is not measured, which limits applicability in the real world. Moreover, the model assumes robots inherently know of the finite number of infected locations ℓ\ell, which also makes the decentralized problem less decentralized.
  • As a paper on RL on the other hand, I feel there is no significant novelty on the machine and representation learning side. A stochastic optimization problem is stated, but it appears it is not used for finding a solution.
  • Regarding the model, the v_min and v_max does not appear physical, as Eq. (3) is otherwise derived from drag and thrust. There should be a natural v_min and v_max derived from the drag. The overall modelling is complex and specific, and without justification why, it is not clear to me how this model is more useful in practice than a more simplified model.
  • I wonder about the relation to vehicle routing problems and other classical combinatorial optimization problems.
Questions for authors
  1. The scenario seems simple enough for heuristics, I wonder how would an approach compare such as UAVs moving to the nearest still-infected location?
  2. Is the stochastic constrained optimization problem solved or used as one of the baselines?
  3. Does the problem not separate into the big field of vehicle routing problems (VRPs) + a local spraying problem? In which case, this would be a good heuristic.
  4. How does the model compare to more simple models when applied in practice? What is the advantage of the model over more simple models?
Reviewer_oU1eOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a new multi-robot environment by combining stochastic dynamics, spray physics modelling, optimization problems, reinforcement learning environments, multi-agent reinforcement learning (MARL), and robotic simulation. The main contribution of this paper is to try design a new environment which considering a more realistic scenario. The authors propose a simulation environment that incorporates realistic factors, such as wind fields and geospatial information, to enhance coordination and collaboration among multiple robots under complex environmental conditions. Extensive experimental analysis on simulated environmental scenario are conducted to prove the effevtiveness of the proposed method.

Critical strengths
  1. The problem addressed by the authors is highly relevant and worthy of further investigation, as it has significant practical implications for addressing real-world challenges in agriculture.
  2. The extensive theoretical analysis provides a rigorous foundation for the proposed method and supports its logical soundness.
Critical weaknesses
  1. My main concern is the limited methodological novelty of this work. The proposed framework largely integrates existing components, including established physical models, Gaussian spray deposition models, POMDP formulations, and existing reinforcement learning algorithms. Therefore, the main contribution appears to lie more in engineering-oriented system integration and simulation design than in fundamental methodological advances in machine learning. While I acknowledge the practical value of such an integrated framework, I am not convinced that this level and type of novelty sufficiently align with the expectations of a machine learning venue such as ICLR.
  2. The whole experiments are conducted in the simulation scenarios. More realistic datasets should be used to prove the effectiveness of the proposed method.
  3. The manuscript contains an excessive number of mathematical formulations, many of which simply reiterate existing methods rather than highlighting the key contributions of this work. This makes the paper difficult to follow and obscures the central ideas and methodological novelty of the proposed framework. Furthermore, the lack of illustrative figures or architectural diagrams makes it challenging for readers to understand the overall design, the relationships between different components, and the workflow of the proposed framework. I strongly encourage the authors to streamline the mathematical descriptions and provide clear visualizations to better communicate the core contributions and overall framework design.
Questions for authors
  1. The authors should more clearly highlight and articulate the specific contributions of this work to the machine learning community, particularly what methodological innovations are introduced beyond the integration of existing models and reinforcement learning algorithms.
  2. Experimental results on realistic datasets are needed to prove the effectiveness of the proposed method.
  3. The authors should reorganize the manuscript to improve its clarity and better emphasize the key contributions. Many mathematical formulations appear to be derived directly from existing literature and do not require extensive elaboration. Instead, the authors should provide appropriate references for established methods and focus the technical discussion on the novel aspects of the proposed framework.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.