- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors incorporate planar laser-induced fluorescence (PLIF) recordings of turbulent odour plumes, as gathered in another study, in a benchmark framework. The PLIF measurements provide concentration over time, together with the air flow and geometry in the environment. The authors made a software framework that allows to read in these measurements to become the inputs to a simulated robot. They then continue on to compare three odor source localisation methods, Infotaxis, PMFS, and RNN-PPO. They show that one of the learned methods works on a robot in small 4x2 m arena, the robot arrives within 0.3m from the source in 29/30 trials.
Critical strengths
- Benchmarking is important, but challenging in robotics. This study tries to advance benchmarking for odor source localization by leveraging difficult-to-make measurements of an actual flow and gas field, in order to make a software package that others can use.
- The paper immediately benchmarks three methods, infotaxis, PMFS, and RNN-PPO (two variants).
- The validity of the simulation is verified by real robot experiments.
Critical weaknesses
- The measurements used, seem quite limited: "The evaluated split uses six recordings for training and three for testing", so 9 sequences in total. The geometry is not explicitly mentioned, but based on the figures seems to be a small open space. The plumes seem to have some turbulence, judging on the images, but it is not exactly clear which ones are from the real measurements? All, also in Fig 1 bottom right? Hence, the question is how well methods trained on this dataset will generalize. Also, if one wants to incorporate different measurements, they may be made with a very different sensing setup, etc., so it may not be compatible... this can severely limit the usefulness of the proposed modular framework for others.
- The real experiments are also quite limited: (a) the space is small (4x2m) and is completely open, making the odor probably spread mostly by diffusion, (b) it is not clear what kind of wind is present in the space, and in any case "wind is prescribed", meaning not measured on the robot - indeed, measuring the air flow would be challenging, especially for low air flows typically experienced indoors.
Questions for authors
- Eq. 2: What is the recording index if not the time? Shouldn't kt also be present on the left side?
- "The source concentration is set to 20ppm": What does this mean? I thought this was measured? Or did you normalize with respect to this value?
- What is the size of the environment that was measured with PLIF?
- What form does the wind-conditioning take for the belief update?
- "we connect a learned controller to a mobile robot searching ..." -> Which learned controller? Belief-RNN-PPO?
- How does the work relate to other benchmarks? For example, GSL-bench uses simulated flow and odor source spreading to simulate larger scale spaces with obstacles: Erwich, H. H., Duisterhof, B. P., & de Croon, G. C. (2024, May). Gsl-bench: High fidelity gas source localization benchmarking tool. In 2024 IEEE International Conference on Robotics and Automation (ICRA) (pp. 17672-17678). IEEE. There is a clear difference, as it is not based on real measurements. However, it was also validated with preliminary work with small drones.
- "Gas measurements acquired near 1 kHz are filtered and decimated to 20 Hz, then converted to a baseline-relative proxy." What are the dynamics of the sensor? The sampling frequency may be high, but how fast does the measurement go up when concentration goes up, and how fast does it go down when the specific gas is gone? So what are the dynamics there?
4 comments
Acknowledgement of rebuttal and revision
I thank the authors for their reply. They have performed substantial additional work, further improving the value of the article, e.g., adding more methods to the benchmarking, increasing the complexity of the task (comparing to a gas-blind up-wind movement), characterizing the dynamics of the e-nose, and testing the generalizability of the results with held-out environments.
Personally, I am a bit suprised by the bad performance of the no-belief versions of RNN-PPO. They perform worse than the reactive baselines, such as cast and surge and gas-blind up-wind motion. Small feedforward neural networks should be able to at least match those performances, and an RNN should in principle in terms of capacity be able to surpass them. So, in my eyes, there must be a problem with the training setup (reward function, curriculum, etc.). In the very least, I would recommend the authors to make a remark on a potential sub-optimal setup for training. Of course, if the authors do believe that RNN-PPO is fundamentally unable to at least match reactive behaviors' performance, I would love to hear it.
I asked whether all images are from real measurements. The authors say: "All reference-benchmark fields are physical PLIF measurements" --> so is the image in the bottom right of figure 1 really a real measurement field? It seems to have (1) way more detail, and (2) way more turbulence than all the other measurements. Please explain.
Reply to vuAn
We are very happy that the reviewer appreciates the improvements, and we thank them for pushing on the remaining points. The question whether something in our setup is wrong, or whether we believe recurrent PPO is fundamentally unable to match the reactive rules, has motivated us to run additional experiments, which we report below.
Is something wrong with the training? We went through reward terms, observation, wind geometry, optimiser, and whether the input supports a good policy. We found no bug; retraining reproduces the checkpoints. The evidence points to the reward, as you suspected.
All learned rows share one reward: a small per-step cost, a wall-proximity penalty, a collision penalty, and a progress term that pays for moving towards the source inside the plume. We designed it once, with the belief input in mind, and held it fixed for the ablations so that rows differ only in their inputs. For an agent without a belief, the reward is nearly silent: the source is upwind of only 41 % of starts, so before the first detection an upwind step gains on average a tenth of its length towards the source and a crosswind step nothing. Any move risks the wall penalty, standing still does not, so before the first detection nothing beats stopping. The trained no-belief policies are near-uniform, with "stop" as their most likely action.
Did we try to fix it? Yes, within this reward structure: we experimented with ungated progress, softer penalties, entropy bonuses, curricula, a thresholded detection input, a minibatch/BPTT optimiser, and combinations (sixteen). None moves the policy off uniform; the belief policy learns under all of them. We also gave the policy cast-and-surge's own ingredients, the time since the last detection and four wind-relative actions (8.1 +- 2.2 %), and a bonus per plume entry (5.4 +- 4.6 %): still uniform. The missing piece is not representation: we behaviour-cloned the unchanged GRU, on the identical 15-D input, from cast-and-surge in the training environment. Without RL it reaches 48.2 +- 9.0 % on the reference track and 31.6 +- 4.3 % held-out (cast-and-surge 84.6 / 25.8 %). PPO fine-tuning under our reward does not improve it (30.6 and 51.2 % with the two optimisers), again due to our reward: the clone earns about five times less per episode than standing still. Casting in these small arenas keeps the agent near the walls, and the wall terms cost more than the terminal reward returns. PPO converged to what the reward prefers. The reward was designed for the belief agent, which moves directly and rarely nears a wall; for cue-only search in this geometry it is ill-posed, as the new appendix now explains.
So is RNN-PPO fundamentally unable to match the reactive rules without a belief? We do not believe that, and will not claim it. The clone shows that the network can hold a cast-and-surge-class policy from this input; Position-only RNN-PPO (61.5 %) shows that the same learner and reward solve the task once the input has a spatial reference. Our no-belief row measures what this learner finds under a reward chosen for the belief agent. We expect that a reward designed for cue-only search (tolerating casting near walls, paying before the first contact), longer training and a proper sweep over seeds and hyper-parameters would raise it; the toolbox is built for such a study. Doing so would require re-tuning every learned row for a fair comparison, an interesting study of its own, that unfortunately does not fit the time left. However, we would like to clarify that the paper's claim does not rest on this row: the belief policy is compared with the reactive rules on the same local inputs, with position-only PPO under the same reward, and on the held-out campaign (see updated Section 5.2).
Changes to the paper. We add the remark you ask for: Section 4 and Appendix B state that the no-belief row measures what this learner finds under the shared reward and budget, not what the input supports. Section 5.2 gives the argument and the clone result (also Table 2); a new appendix reports all controls above; the Discussion names reward design for cue-less search as open work. Code, configurations and checkpoints are released.
Figure 1. You are right, and we apologise: in the submitted version the plume in the bottom-right panel was a generated illustration of the deployment setting, not data. Our first reply said so, but the sentence was buried under our opening claim, "all reference-benchmark fields are physical PLIF measurements", which refers to the fields the benchmark computes on. We agree that this was ambiguous. We have replaced the panel with a recorded PLIF frame from the benchmark data, shown with log-scaled intensity so that faint filaments are visible, with source, agent and path drawn in (see the caption).
Concluding note to vuAn before the discussion period closes
We would like to again express our gratitude for the reviewer's careful and diligent handling of our manuscript, and for the important points being raised. We believe the resulting improvements have made the paper clearer and considerably deeper, as the reviewer also noted in their last reply, and we are grateful for that.
In direct response to Reviewer vuAn's comments and the discussion that followed, we have:
- characterised the turbulence the agent actually sees: intermittency, whiff and blank durations and concentration statistics of every test recording at the agent's 20 Hz sampling rate, by position in the field (Appendix F, Table 7);
- added a physical flow-bench measurement of the fast sensor's step response (Figure 4, Table 10), tying the control step to measured hardware rather than assumptions;
- validated the benchmark on a second, independent PLIF campaign (270 held-out episodes, Appendix H, Table 9), where the belief policy keeps its margin over every reactive rule;
- expanded the comparison with further navigation policies, both learned and heuristic (position-only RNN-PPO, a behaviour-cloned no-belief GRU and its fine-tuned variants, wind-relative and contact-bonus controls, tabular and clock baselines), with code, configurations and checkpoints released;
- traced the no-belief failure the reviewer asked about to its cause and reported the full set of sixteen controls and the return analysis in a new appendix, with the corresponding remarks in Sections 4 and 5.2 and in the Discussion;
- added the requested comparison with existing olfactory environments and benchmarks (OTTO, GADEN, GSL-Bench, NeuroMechFly), stating what each simulates or measures and where OSRL differs;
- replaced the illustration in Figure 1 with a recorded PLIF frame and corrected the caption;
- clarified the role of the prescribed wind (Section 6 and Appendix J): the policy receives only the mean wind direction, a slowly varying quantity that can be measured offline or by a fixed station and sent to the robot; the paper states this choice and its reason. We note that recent low-speed digital wind sensors (direction and speed from 0 to 3 m/s in a 20 mm package, see e.g. MinebeaMitsumi's MMS002 series) make onboard estimation increasingly feasible, which we intend to evaluate in future work.
We believe these additions address each of the reviewer's concerns with the detail they deserved, and that the revised paper is substantially stronger as a result. If the reviewer agrees, we would be very grateful if they would consider reflecting this in their score. If any point remains open, we would of course welcome the chance to clarify it in the time remaining.
Reply to vuAn
Thank you for the thoughtful review and the specific questions. Several aspects of the measurements and the robot experiment were under-explained; the revision expands them, adds several navigation policies plus a held-out evaluation on independent PLIF recordings, motivated by your questions on generalisation.
Measured fields and benchmark geometry. All reference-benchmark fields are physical PLIF measurements, not simulated dispersion: about 0.30 × 0.31 m, 20 Hz, 275 s, nominal wind 0.10, 0.15 and 0.20 m/s. The benchmark rescales them spatially to 3, 6 and 10 m while keeping the measured temporal evolution. This is an evaluation transformation, not a claim that the rescaled fields reproduce room-scale flows (now explicit in Sections 3.1 and 4.1). The 20 ppm is likewise a scaling factor: PLIF concentrations, normalised to the source intensity, are multiplied by a nominal value to obtain sensor-like units; intermittency and spatial structure are those of the measurements. The plume in Figure 2b is a recorded PLIF frame (the caption now names the recording and notes it is not synchronised with the episode); the plume in the Figure 1 schematic is an illustration.
In Equation 2, (k_t) is the recorded frame index, i.e. the replay clock; in the reference benchmark the control interval equals the frame interval, so (k_{t+1}=k_t+1). Actions determine where the robot samples the plume, not the recorded flow (now stated in Section 3.1).
Wind conditioning and the deployed controller. The belief likelihood weights candidate sources by their direction relative to upwind through (\exp(\beta_w\cos\psi)) with (\beta_w=0.5), now stated in Section 4.4. Removing it and retraining changes success from 91.1% to 90.2%. The robot runs a separately trained belief RNN-PPO controller with a dynamic MOx sensor model (see paragraph below). Its 4 × 2 m arena sits in a wind tunnel at nominal 2, 3 and 6 m/s, so transport is advection-dominated, not diffusive. "Prescribed wind" means the controller receives the known tunnel-flow direction rather than an onboard anemometer reading; indeed, onboard wind sensing at low speeds remains a practical challenge.
Sensor dynamics. Appendix J is substantially expanded. The training model uses first-order rise and recovery dynamics (0.5 s and 1.2 s) followed by nonlinear transduction, fitted to simultaneous PLIF and e-nose recordings of one plume. Further, we now characterise the robot's sensor on a flow bench with acetone steps at several concentration levels (ppm-range), deconvolving the delivery path (delay, tubing, chamber); Figure 4 and Table 10 give the step responses. At 100 ppm, onset is below 0.1 s and recovery 0.75 s. Slow tails of tens of seconds at low concentration are not captured by the first-order model, which we now state. The same sensor family resolves odour fluctuations at 40 Hz and above (Dennler et al., Science Advances 2024), so 20 Hz control is within its bandwidth. In the benchmark, the identity-trained belief RNN-PPO reaches (86.2 ± 2.2)% under the MOx model versus 91.1% with identity sensing; retraining under the MOx model gives (85.1 ± 5.0)% (Section 5.3).
Generalisation and related benchmarks. We agree that nine recordings are a limited set of plume conditions. Therefore, the revision therefore evaluates every policy, without retraining or retuning, on six recordings from a second data campaign (Stark et al., 2026, doi:10.21203/rs.3.rs-10165984/v1) with a different tunnel configuration, 0.30 m/s flow and varied source and turbulence conditions (Section 5.4, Appendix H). These files have a different layout and source-relative concentrations, so they also test the compatibility point you raise: a small converter writes them into the field interface, and no policy, sensor or evaluation code changed. Belief RNN-PPO reaches (90.0 ± 1.6)% success against (91.1 ± 0.8)% on the reference track; PMFS retains its rate (91.8% versus 92.7%), while several reactive and tabular methods deteriorate, partly because the changed normalisation affects fixed detection thresholds, as discussed. Certainly, this is a first independent test and not proof of generalisation to arbitrary environments.
GSL-Bench is now in the related work; its large obstructed simulated environments and OSRL's measured fields complement each other.
Finally, the extended comparisons exposed a flaw in the original track: with the source at the arena boundary, a gas-blind upwind sweep succeeded in 97% of episodes. The revised track uses fixed padded arenas (gas-blind 16.3%) and all methods are re-evaluated on it (window-aligned track: Appendix E). While retraining we also found that the belief implementation used to train the submitted checkpoints differed from the one used to evaluate them; we fixed it and retrained every learned condition, so the submitted numbers are superseded.
These additions clarify what is measured, what is transformed for evaluation, and how far the findings generalise.