acceptodds
Under review as a conference paper at ICLR 2027

OlfactorySearchRL: A Measured-Plume Benchmark for Learning to Navigate by Smell

14 views

Abstract

Olfactory navigation requires an agent to infer a hidden source from intermittent local measurements while choosing where to sample next. Progress in learning this behaviour depends on environments that combine realistic chemical structure with repeatable evaluation. We introduce OLFACTORYSEARCHRL (OSRL), a modular toolbox and reference benchmark that turns experimentally measured turbulent concentration fields into interactive learning environments. OSRL separates plume replay, sensing, observation processing, and control, so researchers can compare policies and representations on a common physical record. A fixed 135-episode track is used to compare navigation policies, where here we evaluate Infotaxis, PMFS, and RNN-PPO on three held-out recordings. We additionally retrain the RNN-PPO policy with a dynamic gas sensor model, and showcase successful robotic deployment. In particular, we demonstrate that a robotic agent equipped with chemoresistive gas sensors successfully reaches the source region in complex turbulent flow in 29 of 30 physical trials, without further policy fine-tuning. We release the toolbox and a reproduction package containing configurations, checkpoints, evaluation records, and analysis scripts. OSRL provides an experimental foundation for studying active sensing and state estimation in turbulent chemical environments.

Primary area:
applications to robotics, autonomy, planning
Keywords:
olfactory navigation, reinforcement learning, embodied AI, turbulent plumes, sim-to-real transfer, robotic olfaction, chemical sensing, partially observable decision processes, sensor modeling, benchmark, robotics

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_vuAnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors incorporate planar laser-induced fluorescence (PLIF) recordings of turbulent odour plumes, as gathered in another study, in a benchmark framework. The PLIF measurements provide concentration over time, together with the air flow and geometry in the environment. The authors made a software framework that allows to read in these measurements to become the inputs to a simulated robot. They then continue on to compare three odor source localisation methods, Infotaxis, PMFS, and RNN-PPO. They show that one of the learned methods works on a robot in small 4x2 m arena, the robot arrives within 0.3m from the source in 29/30 trials.

Critical strengths
  • Benchmarking is important, but challenging in robotics. This study tries to advance benchmarking for odor source localization by leveraging difficult-to-make measurements of an actual flow and gas field, in order to make a software package that others can use.
  • The paper immediately benchmarks three methods, infotaxis, PMFS, and RNN-PPO (two variants).
  • The validity of the simulation is verified by real robot experiments.
Critical weaknesses
  • The measurements used, seem quite limited: "The evaluated split uses six recordings for training and three for testing", so 9 sequences in total. The geometry is not explicitly mentioned, but based on the figures seems to be a small open space. The plumes seem to have some turbulence, judging on the images, but it is not exactly clear which ones are from the real measurements? All, also in Fig 1 bottom right? Hence, the question is how well methods trained on this dataset will generalize. Also, if one wants to incorporate different measurements, they may be made with a very different sensing setup, etc., so it may not be compatible... this can severely limit the usefulness of the proposed modular framework for others.
  • The real experiments are also quite limited: (a) the space is small (4x2m) and is completely open, making the odor probably spread mostly by diffusion, (b) it is not clear what kind of wind is present in the space, and in any case "wind is prescribed", meaning not measured on the robot - indeed, measuring the air flow would be challenging, especially for low air flows typically experienced indoors.
Questions for authors
  • Eq. 2: What is the recording index if not the time? Shouldn't kt also be present on the left side?
  • "The source concentration is set to 20ppm": What does this mean? I thought this was measured? Or did you normalize with respect to this value?
  • What is the size of the environment that was measured with PLIF?
  • What form does the wind-conditioning take for the belief update?
  • "we connect a learned controller to a mobile robot searching ..." -> Which learned controller? Belief-RNN-PPO?
  • How does the work relate to other benchmarks? For example, GSL-bench uses simulated flow and odor source spreading to simulate larger scale spaces with obstacles: Erwich, H. H., Duisterhof, B. P., & de Croon, G. C. (2024, May). Gsl-bench: High fidelity gas source localization benchmarking tool. In 2024 IEEE International Conference on Robotics and Automation (ICRA) (pp. 17672-17678). IEEE. There is a clear difference, as it is not based on real measurements. However, it was also validated with preliminary work with small drones.
  • "Gas measurements acquired near 1 kHz are filtered and decimated to 20 Hz, then converted to a baseline-relative proxy." What are the dynamics of the sensor? The sampling frequency may be high, but how fast does the measurement go up when concentration goes up, and how fast does it go down when the specific gas is gone? So what are the dynamics there?
4 comments
AuthorsOpenReview ↗

Reply to vuAn

Thank you for the thoughtful review and the specific questions. Several aspects of the measurements and the robot experiment were under-explained; the revision expands them, adds several navigation policies plus a held-out evaluation on independent PLIF recordings, motivated by your questions on generalisation.

Measured fields and benchmark geometry. All reference-benchmark fields are physical PLIF measurements, not simulated dispersion: about 0.30 × 0.31 m, 20 Hz, 275 s, nominal wind 0.10, 0.15 and 0.20 m/s. The benchmark rescales them spatially to 3, 6 and 10 m while keeping the measured temporal evolution. This is an evaluation transformation, not a claim that the rescaled fields reproduce room-scale flows (now explicit in Sections 3.1 and 4.1). The 20 ppm is likewise a scaling factor: PLIF concentrations, normalised to the source intensity, are multiplied by a nominal value to obtain sensor-like units; intermittency and spatial structure are those of the measurements. The plume in Figure 2b is a recorded PLIF frame (the caption now names the recording and notes it is not synchronised with the episode); the plume in the Figure 1 schematic is an illustration.

In Equation 2, (k_t) is the recorded frame index, i.e. the replay clock; in the reference benchmark the control interval equals the frame interval, so (k_{t+1}=k_t+1). Actions determine where the robot samples the plume, not the recorded flow (now stated in Section 3.1).

Wind conditioning and the deployed controller. The belief likelihood weights candidate sources by their direction relative to upwind through (\exp(\beta_w\cos\psi)) with (\beta_w=0.5), now stated in Section 4.4. Removing it and retraining changes success from 91.1% to 90.2%. The robot runs a separately trained belief RNN-PPO controller with a dynamic MOx sensor model (see paragraph below). Its 4 × 2 m arena sits in a wind tunnel at nominal 2, 3 and 6 m/s, so transport is advection-dominated, not diffusive. "Prescribed wind" means the controller receives the known tunnel-flow direction rather than an onboard anemometer reading; indeed, onboard wind sensing at low speeds remains a practical challenge.

Sensor dynamics. Appendix J is substantially expanded. The training model uses first-order rise and recovery dynamics (0.5 s and 1.2 s) followed by nonlinear transduction, fitted to simultaneous PLIF and e-nose recordings of one plume. Further, we now characterise the robot's sensor on a flow bench with acetone steps at several concentration levels (ppm-range), deconvolving the delivery path (delay, tubing, chamber); Figure 4 and Table 10 give the step responses. At 100 ppm, onset is below 0.1 s and recovery 0.75 s. Slow tails of tens of seconds at low concentration are not captured by the first-order model, which we now state. The same sensor family resolves odour fluctuations at 40 Hz and above (Dennler et al., Science Advances 2024), so 20 Hz control is within its bandwidth. In the benchmark, the identity-trained belief RNN-PPO reaches (86.2 ± 2.2)% under the MOx model versus 91.1% with identity sensing; retraining under the MOx model gives (85.1 ± 5.0)% (Section 5.3).

Generalisation and related benchmarks. We agree that nine recordings are a limited set of plume conditions. Therefore, the revision therefore evaluates every policy, without retraining or retuning, on six recordings from a second data campaign (Stark et al., 2026, doi:10.21203/rs.3.rs-10165984/v1) with a different tunnel configuration, 0.30 m/s flow and varied source and turbulence conditions (Section 5.4, Appendix H). These files have a different layout and source-relative concentrations, so they also test the compatibility point you raise: a small converter writes them into the field interface, and no policy, sensor or evaluation code changed. Belief RNN-PPO reaches (90.0 ± 1.6)% success against (91.1 ± 0.8)% on the reference track; PMFS retains its rate (91.8% versus 92.7%), while several reactive and tabular methods deteriorate, partly because the changed normalisation affects fixed detection thresholds, as discussed. Certainly, this is a first independent test and not proof of generalisation to arbitrary environments.

GSL-Bench is now in the related work; its large obstructed simulated environments and OSRL's measured fields complement each other.

Finally, the extended comparisons exposed a flaw in the original track: with the source at the arena boundary, a gas-blind upwind sweep succeeded in 97% of episodes. The revised track uses fixed padded arenas (gas-blind 16.3%) and all methods are re-evaluated on it (window-aligned track: Appendix E). While retraining we also found that the belief implementation used to train the submitted checkpoints differed from the one used to evaluate them; we fixed it and retrained every learned condition, so the submitted numbers are superseded.

These additions clarify what is measured, what is transformed for evaluation, and how far the findings generalise.

Reviewer_vuAnOpenReview ↗

Acknowledgement of rebuttal and revision

I thank the authors for their reply. They have performed substantial additional work, further improving the value of the article, e.g., adding more methods to the benchmarking, increasing the complexity of the task (comparing to a gas-blind up-wind movement), characterizing the dynamics of the e-nose, and testing the generalizability of the results with held-out environments.

Personally, I am a bit suprised by the bad performance of the no-belief versions of RNN-PPO. They perform worse than the reactive baselines, such as cast and surge and gas-blind up-wind motion. Small feedforward neural networks should be able to at least match those performances, and an RNN should in principle in terms of capacity be able to surpass them. So, in my eyes, there must be a problem with the training setup (reward function, curriculum, etc.). In the very least, I would recommend the authors to make a remark on a potential sub-optimal setup for training. Of course, if the authors do believe that RNN-PPO is fundamentally unable to at least match reactive behaviors' performance, I would love to hear it.

I asked whether all images are from real measurements. The authors say: "All reference-benchmark fields are physical PLIF measurements" --> so is the image in the bottom right of figure 1 really a real measurement field? It seems to have (1) way more detail, and (2) way more turbulence than all the other measurements. Please explain.

AuthorsOpenReview ↗

Reply to vuAn

We are very happy that the reviewer appreciates the improvements, and we thank them for pushing on the remaining points. The question whether something in our setup is wrong, or whether we believe recurrent PPO is fundamentally unable to match the reactive rules, has motivated us to run additional experiments, which we report below.

Is something wrong with the training? We went through reward terms, observation, wind geometry, optimiser, and whether the input supports a good policy. We found no bug; retraining reproduces the checkpoints. The evidence points to the reward, as you suspected.

All learned rows share one reward: a small per-step cost, a wall-proximity penalty, a collision penalty, and a progress term that pays for moving towards the source inside the plume. We designed it once, with the belief input in mind, and held it fixed for the ablations so that rows differ only in their inputs. For an agent without a belief, the reward is nearly silent: the source is upwind of only 41 % of starts, so before the first detection an upwind step gains on average a tenth of its length towards the source and a crosswind step nothing. Any move risks the wall penalty, standing still does not, so before the first detection nothing beats stopping. The trained no-belief policies are near-uniform, with "stop" as their most likely action.

Did we try to fix it? Yes, within this reward structure: we experimented with ungated progress, softer penalties, entropy bonuses, curricula, a thresholded detection input, a minibatch/BPTT optimiser, and combinations (sixteen). None moves the policy off uniform; the belief policy learns under all of them. We also gave the policy cast-and-surge's own ingredients, the time since the last detection and four wind-relative actions (8.1 +- 2.2 %), and a bonus per plume entry (5.4 +- 4.6 %): still uniform. The missing piece is not representation: we behaviour-cloned the unchanged GRU, on the identical 15-D input, from cast-and-surge in the training environment. Without RL it reaches 48.2 +- 9.0 % on the reference track and 31.6 +- 4.3 % held-out (cast-and-surge 84.6 / 25.8 %). PPO fine-tuning under our reward does not improve it (30.6 and 51.2 % with the two optimisers), again due to our reward: the clone earns about five times less per episode than standing still. Casting in these small arenas keeps the agent near the walls, and the wall terms cost more than the terminal reward returns. PPO converged to what the reward prefers. The reward was designed for the belief agent, which moves directly and rarely nears a wall; for cue-only search in this geometry it is ill-posed, as the new appendix now explains.

So is RNN-PPO fundamentally unable to match the reactive rules without a belief? We do not believe that, and will not claim it. The clone shows that the network can hold a cast-and-surge-class policy from this input; Position-only RNN-PPO (61.5 %) shows that the same learner and reward solve the task once the input has a spatial reference. Our no-belief row measures what this learner finds under a reward chosen for the belief agent. We expect that a reward designed for cue-only search (tolerating casting near walls, paying before the first contact), longer training and a proper sweep over seeds and hyper-parameters would raise it; the toolbox is built for such a study. Doing so would require re-tuning every learned row for a fair comparison, an interesting study of its own, that unfortunately does not fit the time left. However, we would like to clarify that the paper's claim does not rest on this row: the belief policy is compared with the reactive rules on the same local inputs, with position-only PPO under the same reward, and on the held-out campaign (see updated Section 5.2).

Changes to the paper. We add the remark you ask for: Section 4 and Appendix B state that the no-belief row measures what this learner finds under the shared reward and budget, not what the input supports. Section 5.2 gives the argument and the clone result (also Table 2); a new appendix reports all controls above; the Discussion names reward design for cue-less search as open work. Code, configurations and checkpoints are released.

Figure 1. You are right, and we apologise: in the submitted version the plume in the bottom-right panel was a generated illustration of the deployment setting, not data. Our first reply said so, but the sentence was buried under our opening claim, "all reference-benchmark fields are physical PLIF measurements", which refers to the fields the benchmark computes on. We agree that this was ambiguous. We have replaced the panel with a recorded PLIF frame from the benchmark data, shown with log-scaled intensity so that faint filaments are visible, with source, agent and path drawn in (see the caption).

AuthorsOpenReview ↗

Concluding note to vuAn before the discussion period closes

We would like to again express our gratitude for the reviewer's careful and diligent handling of our manuscript, and for the important points being raised. We believe the resulting improvements have made the paper clearer and considerably deeper, as the reviewer also noted in their last reply, and we are grateful for that.

In direct response to Reviewer vuAn's comments and the discussion that followed, we have:

  • characterised the turbulence the agent actually sees: intermittency, whiff and blank durations and concentration statistics of every test recording at the agent's 20 Hz sampling rate, by position in the field (Appendix F, Table 7);
  • added a physical flow-bench measurement of the fast sensor's step response (Figure 4, Table 10), tying the control step to measured hardware rather than assumptions;
  • validated the benchmark on a second, independent PLIF campaign (270 held-out episodes, Appendix H, Table 9), where the belief policy keeps its margin over every reactive rule;
  • expanded the comparison with further navigation policies, both learned and heuristic (position-only RNN-PPO, a behaviour-cloned no-belief GRU and its fine-tuned variants, wind-relative and contact-bonus controls, tabular and clock baselines), with code, configurations and checkpoints released;
  • traced the no-belief failure the reviewer asked about to its cause and reported the full set of sixteen controls and the return analysis in a new appendix, with the corresponding remarks in Sections 4 and 5.2 and in the Discussion;
  • added the requested comparison with existing olfactory environments and benchmarks (OTTO, GADEN, GSL-Bench, NeuroMechFly), stating what each simulates or measures and where OSRL differs;
  • replaced the illustration in Figure 1 with a recorded PLIF frame and corrected the caption;
  • clarified the role of the prescribed wind (Section 6 and Appendix J): the policy receives only the mean wind direction, a slowly varying quantity that can be measured offline or by a fixed station and sent to the robot; the paper states this choice and its reason. We note that recent low-speed digital wind sensors (direction and speed from 0 to 3 m/s in a 20 mm package, see e.g. MinebeaMitsumi's MMS002 series) make onboard estimation increasingly feasible, which we intend to evaluate in future work.

We believe these additions address each of the reviewer's concerns with the detail they deserved, and that the revised paper is substantially stronger as a result. If the reviewer agrees, we would be very grateful if they would consider reflecting this in their score. If any point remains open, we would of course welcome the chance to clarify it in the time remaining.

Reviewer_6DDdOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents OlfactorySearchRL (OSRL), a toolbox and reference benchmark for the olfactory navigation problem. The toolbox models olfactory navigation as a finite-horizon POMDP with an RL-style interface. The underlying state is st=(qt,kt,zt;θ)s_t = (q_t, k_t, z_t; \theta), where qtq_t is the agent pose, ktk_t is the index of the current frame of the recorded plume, ztz_t is the internal state of the sensor model and observation processor, and θ\theta fixes the recording, source, geometry, calibration, wind, sensor model and initial conditions. The agent’s position determines where the recorded concentration field is sampled, while the recording evolves independently of the agent. The agent never sees the state directly: the sampled concentration passes through the sensor model and the observation processor. The toolbox separates plume replay, sensor model, observation processor, task and reward, policy, and evaluation, so each component can be changed on its own. The current version implements four algorithms: Infotaxis, PMFS, and two recurrent PPO agents.

Critical strengths
  1. The manuscript is well-written, clear and easy to follow and the olfactory navigation problem is relevant

  2. Replaying measured turbulent plumes preserves the intermittency and spatial structure that simulators struggle to reproduce. The toolbox offers an executable environment on real data, and it gives researchers without a plume-imaging facility or a chemical-sensing robot a practical entry point

  3. The authors release the code, configurations, episode manifests, checkpoints and per-episode outputs, with three reproduction routes (regenerate the analyses, evaluate checkpoints, retrain)

Critical weaknesses

One of the major weaknesses is the lack of several important state-of-the-art methods for olfactory navigation proposed in the literature. For instance, classical nature-inspired heuristics such as cast-and-surge and circling are missing, and tabular Q-learning-based algorithms [1,2,3] are not included nor mentioned. These approaches rely on very little information and a minimal state formalization, and they are important because they are used to model navigation with limited resources, e.g., how certain animals navigate space using only the sense of smell. They would therefore provide meaningful baselines for assessing how much information and state estimation the learned policies actually need. Moreover, previous POMDP-based approaches such as [4,5] and RNN-based approaches such as [6] are discussed in the related work but not included as baselines. Since the goal of the work is to propose a reference benchmark for olfactory navigation algorithms, it is fundamental to include well-known algorithms. As result, no comparison with standard reactive heuristics (cast-and-surge, circling) or algorithms with limited state information is provided. Without them, the absolute performance of Infotaxis, PMFS and RNN-PPO is hard to interpret, and it is unclear whether the conclusions (the reliability and directness trade-off, and the gap caused by removing the belief) hold beyond the single learned method evaluated. Another limitation concerns the dataset used in the experiments. The arena sizes and turbulence conditions are limited, which considerably simplifies the navigation problem, while previous works, e.g., [2,3,4], consider either larger simulations or different sparsity levels (higher turbulence). Moreover, it is not clear whether so much information (pose, wind, domain geometry, peak concentration, etc.) is needed to achieve high navigation performance. The only ablation removes the whole belief package at once, and no component-wise ablation of the information given to the policies is provided.

[1] Rando, M., Heinonen, R. A., Qi, Y., & Seminara, A. (2026). Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery.

[2] Piro, L., Carbone, M., Biferale, L., Cencini, M., Heinonen, R. A., Rando, M., & Seminara, A. (2026). Smart strategies to navigate turbulent odor plumes reorienting to local wind.

[3] Rando, M., James, M., Verri, A., Rosasco, L., & Seminara, A. (2025). Q-learning with temporal memory to navigate turbulence. Elife, 13, RP102906.

[4] Rigolli, N., Reddy, G., Seminara, A., & Vergassola, M. (2022). Alternation emerges as a multi-modal strategy for turbulent odor navigation. Elife, 11, e76989.

[5] Aurore Loisy and Robin A Heinonen. Deep reinforcement learning for the olfactory search pomdp: a quantitative benchmark. The European Physical Journal E, 46(3):17, 2023.

[6] Satpreet H Singh, Floris Van Breugel, Rajesh PN Rao, and Bingni W Brunton. Emergent behaviour and neural dynamics in artificial agents tracking odour plumes. Nature machine intelligence, 5(1):58–70, 2023.

Questions for authors

Here I report some questions that arose during the review:

  1. The belief ablation removes accumulated evidence, pose and geometry registration, the wind-conditioned model and the source prior all at once. Can you provide a graded ablation (eg position only, belief without the domain prior, belief with wind removed, different belief resolutions, only stochastic odor concentration in current position) to show which part of the state information drives the performance gap?

  2. How does standard nature-inspired heuristics like cast-and-surge and circling and tabular q-learning strategies (see Weaknesses section) perform in the dataset considered?

2 comments
AuthorsOpenReview ↗

Reply to 6DDd

Thank you for the detailed review and the pointer to the minimal-state navigation literature. We agree that the original comparison was too narrow for a reference benchmark. The revision adds the reactive and tabular methods you suggested, a graded analysis of the source-belief representation, and a held-out evaluation on an independent PLIF campaign.

Additional baselines. We now compare cast-and-surge, circling, clock-state Q-learning (Rando et al., 2026; also with the parameterisation of Piro et al., 2026) and temporal-memory Q-learning (Rando et al., 2025), plus a gas-blind control, a belief-greedy controller and RNN-PPO variants with different information budgets. All are evaluated on the same fixed episodes, with parameters selected on the training recordings only (revised Table 2, Figure 2, Appendix I).

The gas-blind control exposed a flaw in the original track: the source sat at the upwind arena boundary, so wall-hugging and wind-following alone succeeded in 97% of episodes. The revised reference track places the source away from the wall with fixed, seeded arena placements; gas-blind success falls to 16.3%. All methods are re-evaluated on this track (123 nontrivial starts of 135); the window-aligned track remains in Appendix E. While retraining we also found that the belief implementation used to train the submitted checkpoints differed from the one used to evaluate them; we fixed it and retrained every learned condition, so the submitted numbers are superseded. We thank the reviewer to push for this broader comparison.

The expanded policy comparison is informative. Circling reaches 95.1% success, cast-and-surge 84.6%, clock-state Q-learning 72.9% and temporal-memory Q-learning 78.9%, but with indirect paths. Belief RNN-PPO reaches (91.1 ± 0.8)% with SPL 0.695 ± 0.029, against SPL 0.321 for circling and 0.385 for PMFS. The benchmark now separates reliable source finding from efficient navigation, see Figure 2.

On the POMDP and recurrent approaches: porting the model-based solvers would require adapting their observation models and sensing tasks, so we do not claim exact reproductions. Instead we added a full-belief RNN-PPO (83.2%), an odour-and-wind-only recurrent controller (0.5%) and the tabular methods above. One result bears directly on your question of how much information is needed: the tabular agents reach 73–79% on far less information than the no-belief recurrent policy, which collapses to barely moving. Section 5.2 now states that this collapse is a property of the learner under our shaped reward (a wall penalty and a stop action make waiting a local optimum before the first detection), not of the input: cast-and-surge on the same gas and wind input reaches 84.6%. The gap bounds what this learner finds under the fixed budget; the reactive rules bound what the input supports.

Graded belief ablation. Indeed, removing the whole belief alone did not show which components matter. We now separately remove wind conditioning, grid resolution, the arena prior, accumulated evidence and policy recurrence (revised Table 3), each retrained with three seeds except the explicitly marked evaluation-time recurrence intervention. From 91.1%, removing wind conditioning gives 90.2% and halving the grid resolution 90.0%; removing accumulated evidence gives 57.2% and removing the arena prior 12.5%; a policy trained without recurrence reaches 85.1%. What matters is accumulated evidence and the spatial constraint of the arena, not likelihood resolution or the wind term alone. A position-only control, which separates the belief from merely knowing one's location, reaches 61.5% and degrades much faster with arena width, so the belief carries information beyond positional regularities learned in training.

Arena conditions and generalisation. We now report plume intermittency and blank-duration statistics (Appendix F) and performance by recording and arena width (Appendix G). At 10 cm crosswind, detections occupy only 3–5% of a recording, with blanks of up to 8–19 s.

To test generalisation beyond the training conditions, which your review rightly asked for, we evaluate every policy, without retraining or retuning, on six recordings from an independent PLIF campaign (Stark et al., 2026, doi:10.21203/rs.3.rs-10165984/v1) with different release conditions, turbulence grids, a vertical tunnel and 0.30 m/s wind against 0.10–0.20 m/s in training (Section 5.4, Appendix H). Belief RNN-PPO holds at (90.0 ± 1.6)% success versus 91.1% on the reference track, PMFS reaches 91.8%, while several methods tuned on the original recordings transfer less well. We discuss the differing concentration normalisation between campaigns.

Larger domains, obstacles and multiple sources remain important next steps, and the revision states these limits explicitly. We believe the added baselines, ablations and held-out evaluation make the benchmark substantially more useful for comparing strategies and the information they need.

I thank the authors for the detailed response, which addressed all my concerns. I also appreciate the inclusion of the additional baselines and the comparison with them. I further appreciate that the authors recognized the exposed flaw and retrained the agents, demonstrating a significant effort.

I also checked the revised manuscript and verified that the authors have already included the new results and observations. I believe that the response to the review has strongly improved the content and quality of the manuscript and has allowed the authors to add useful observations (e.g. further comparisons and a more extensive analysis of which elements influence performance).

For these reasons, and considering the effort made by the authors in reperforming the experiments, I have decided to increase my score to 4 (Clear Acceptance).

Reviewer_JSQLOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces OlfactorySearchRL (OSRL), a toolkit that turns real video recordings of chemical smoke plumes into interactive testing grounds for robot navigation.

By separating how a robot senses odors from how it moves, the authors tested different search strategies and found that giving a robot a structured memory of where the gas likely comes from drastically improves its success.

They validated this system's effectiveness in the real world by having a mobile robot successfully find an acetone scent source in 29 out of 30 physical test runs without any extra tuning.

Critical strengths

Modular and reusable benchmark design:

  • The paper's introduction of OlfactorySearchRL (OSRL) cleanly separates plume dynamics, sensor transduction, observation processing, and agent control. This is important because it allows researchers to rigorously isolate and evaluate specific navigation policies or representation components using a standardized, shared physical record, and it doesn't require access to expensive PLIF imaging facilities.

Rigorous demonstration of state representation impact & real-world transfer:

  • The paper provides both a clear empirical baseline revealing a massive performance gap when structured source-belief features are removed (dropping success rates by over 59 percentage points) and a successful real-world robotic deployment (reaching the source in 29 out of 30 trials). This is also important, because it bridges the gap between simulated reinforcement learning and physical hardware, and also proves that the insights gained from the toolbox translate directly to real chemical-sensing robots.
Critical weaknesses

Limited setup for the physical real-world validation:

  • Although I am impressed by the real-world validation, the physical trials are restricted to a single source type and a very small set of conditions in a certain area. This leaves open questions about the true generalizability of the work

Reliance on a simple sensor:

  • The paper bypasses realistic gas-sensor dynamics which are only tested separately in the physical deployment subset. This is important because a benchmark meant to standardize RL evaluations for olfaction may mask the true robustness of policies against sensor degradation and temporal lag, and this may limit how cleanly performance in the simulated benchmark translates to real chemical-sensing hardware.
Questions for authors

In Section 5.2, removing the structured belief features causes success to drop from 70.7%70.7\% to 11.3%11.3\%. Given that the belief estimator uses an explicit, hand-engineered decay and dispersion likelihood function (Equation 6), to what extent is the learned RNN-PPO policy actually performing reinforcement learning over dynamic navigation strategies vs merely acting as a reactive lookup table that maps the hand-crafted belief vector to pre-computed geometric gradients?

Could you show the benchmark expanding beyond a single open-arena acetone source in order to test robustness against multi-source plumes, dynamic obstacles, and cluttered environments where line-of-sight belief estimators might face severe occlusion or false-positive source hypotheses?

1 comment
AuthorsOpenReview ↗

Reply to JSQL

Thank you for the positive assessment and the thoughtful questions. We took the question about the role of the learned controller particularly seriously, and added experiments to separate the contribution of the belief estimator from that of the policy. Due to the modular structure of the toolbox, this was possible with relatively little adjustments.

Does PPO learn more than following the belief? We now evaluate a belief-greedy controller that uses exactly the same engineered source belief but simply moves towards its most likely source location, without any learned policy. It reaches the source in 87.0% of episodes, compared with (91.1 ± 0.8)% for belief RNN-PPO. The difference in path efficiency is much larger: 0.298 versus 0.764, corresponding to SPL values of 0.259 and 0.695 (revised Tables 2–3). Thus, much of the information needed to find the source is indeed provided by the estimator, but directly following its mode produces considerably longer paths. The learned controller does more than implement this simple geometric rule. We also tested the role of the policy's own recurrent memory. Resetting its GRU state at every evaluation step gives 82.7% success, while a policy retrained without recurrent memory reaches 85.1%, compared with 91.1% for the full model. We therefore agree with your interpretation to some extent: the policy operates largely as a learned controller on a structured state, with most of the useful temporal information carried by the belief estimator. We now state this more explicitly. A further graded ablation identifies which parts of that structured state matter. Removing accumulated evidence reduces success to 57.2%, and removing the arena prior reduces it to 12.5%. In contrast, removing wind conditioning or halving the grid resolution changes success by only about one percentage point. These results are reported in Section 5.2 and Table 3.

Realistic sensor dynamics. We agree that identity sensing alone would give an incomplete picture. We therefore added a second benchmark protocol using the deployment MOx model, with asymmetric response and recovery dynamics. The identity-trained belief policy retains (86.2 ± 2.2)% success under this model, compared with 91.1% under identity sensing. A separately MOx-trained policy achieves (85.1 ± 5.0)%. In fact, this is the trained policy that the robot uses for navigation. Encouraged by the reviewers comments, we have also measured the robot's e-nose directly on a flow bench with acetone steps at various concentrations (ppm range), deconvolving the delivery path, and report the step responses in Appendix J (Figure 4, Table 10). The deviations from the simplified deconvolution model time constants give a more nuanced picture of the sensing constraints and their effects on navigation.

Scope and generalisation. We agree that multiple sources, dynamic obstacles, and cluttered environments are important future tests, but they are beyond the measured environments available here. To investigate generalisation beyond the original recordings, we instead evaluated all policies without retuning on six PLIF recordings from a second experimental campaign, with higher wind speed and different release conditions (Appendix H). Belief RNN-PPO retains (90.0 ± 1.6)% success, close to its 91.1% on the reference track, which is indeed very encouraging. We sincerely thank the reviewer for having motivated this additional study.

Two further corrections emerged during the revision. First, a gas-blind control exposed that the original arena placement made source finding possible by simply following (hugging) the upwind wall (97% success). We have replaced it with a fixed padded reference track, where the same control reaches only 16.3%, and re-evaluated all policies. Second, while retraining for the new track we found that the belief implementation used to train the submitted checkpoints differed from the one used to evaluate them; we fixed this and retrained every learned condition, so the submitted numbers are superseded. The window-aligned track itself is kept in Appendix E for reference.

The revised results strengthen the central observation: a structured source estimate makes learned olfactory navigation substantially more effective, while learning how to act on that estimate contributes to more direct routes. We appreciate your encouraging assessment and hope the additional evidence makes the scope of this result clearer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.