- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ForagerSim is a Unity benchmark for embodied search and foraging with configurable perception, scale, topology, dynamics and task, plus Gym and ROS 2 interfaces. The authors evaluate PPO, DreamerV3, a greedy frontier explorer and one human on eight worlds. They conclude that learned agents fall short in larger or maze-like topologies, so map-based reasoning is the bottleneck.
Critical strengths
-
It fills a real gap: fast, configurable, dynamic worlds with identical scoring for RL policies and ROS 2 stacks. The home-cue-withheld well-alternation task is a clever test of return-to-known-location behavior. This is the main reason I lean toward acceptance.
-
It is candid and reproducible: the Dreamer position shortcut and the failed curriculum are reported, and code, configs and seeds are shared. This makes the benchmark trustworthy as a community resource.
Critical weaknesses
-
The claim that map-based reasoning is the bottleneck is not isolated. PPO is memoryless and Dreamer runs near defaults, with no oracle-map test or learned-memory baseline. Scale, topology and budget vary together, and the multi-km, failure and 3D claims are untested.
-
The comparisons are weak. The frontier explorer fails in dynamic worlds by construction and is absent from the foraging tasks. The human is a single person, and with 3 seeds and standard deviations that sometimes exceed the means, claims like "near-human" are within noise.
Questions for authors
Clarifications:
- How are the ± values computed, and are the evaluation worlds held out from training?
- What happens if PPO and Dreamer receive an oracle occupancy grid? This would directly test the central bottleneck claim.
Suggestions: 3. How do recurrent or spatial-memory agents perform? 4. Single-axis sweeps (scale, topology and budget varied independently) and results with more human participants would strengthen the conclusions.
2 comments
Response to Reviewer H9gT (Part 2)
W1–W2, Q4: Single-axis sweeps and human and multi-km evaluation.
- Single-axis sweeps: Thank you for the suggestion. The current experiments already contain sweeps (in the sense of controlled changes between a set of scenarios) across different axes, which we have now separated more clearly in the reorganized section 4.1: a) Dynamics: we test 2 varying levels of dynamics in the 5houses-dynamic and 5houses-sar (same topology, different dynamics) and now also in the new dynamic variant of the memorymaze15x15 world (same search task and maze as in the memorymaze15x15 scenario, just with passage closing); b) Task complexity: we also test the independent effect of task complexity (4arm-submazes and 4arm-submazes-foraging have the same topology and scale, just a different task specified by swapping reward objects for wells in the world config and the episode length in the task config); and c) Topology/environment complexity: we test various topologies for both the search and foraging tasks. The episode length changes between some scenarios that have a different scale or a different task, but all baselines always have the same budget within a scenario and we draw our conclusion not by comparing absolute returns between scenarios, but from looking at how the differences between the baselines change across scenarios. As another example, the curriculum experiment does a smooth sweep of the environment scale while keeping the topology generation, budget, reward signal, actions, and observations the same (although the number of rooms and the distances between them do change), and users can easily define their own similar sweeps for any other environment through config-file edits with no code changes.
- Human reference and multi-km world evaluations. Regarding the human baseline, please note that the single participant is intended as a feasibility reference and an approximate upper bound, demonstrating that substantially higher performance is achievable with the same observations, rather than providing a statistical characterization of human performance. While we agree that more participants would provide a more reliable estimate of human performance, we do not expect this to change our main conclusions, and such an evaluation was not feasible within the rebuttal period. We also acknowledge that the multi-kilometer 3D scenarios (supported by the simulator) and some sensorimotor-failure settings remain outside the current quantitative evaluation. The tested agents already struggle in several low-dimensional 2D scenarios, which provide substantial room for improvement of the approaches (both learned and engineered) without adding visual-perception complexity.
Response to Reviewer H9gT (Part 1)
Thank you for the constructive feedback and for recognizing the value of ForagerSim, particularly the well-alternation task and the comparison of learned and engineered approaches. We hope the following responses address your concerns:
Q1: How are the ± values computed, and are the evaluation worlds held out from training?
W1, Q2: Testing the bottleneck claim by providing an explicit map (oracle-map) Good question. Following your suggestion, we have added new experiments with explicit map observations (Appendix A, Fig. 7-8) to investigate whether providing agents with a spatial representation improves performance. Rather than supplying a complete oracle map at the start, we give the agents a map that is incrementally constructed during the experiment from the agent’s original observation data (similar to the explicit map built by the frontier explorer), since the continuously built map conveys better information for the search task (i.e., it shows where the agent has not been yet, which allows for discovering strategies akin to frontier-based exploration, which the complete map would not allow) and for the foraging task, it can also aid navigation since it shows the already visited passages. We evaluated the effect of providing an explicit map on the most challenging 4arm-submazes topology, for both search and well-alternation tasks. Visualizations and results of the runs are in the new section, and in summary, they show that having the explicit map does help Dreamer greatly in the search task, but it did not help it with the foraging task (even though the map shows the way to the home well, the agent’s position, and the well positions). This suggests that the challenge of these scenarios lies not only in the construction of maps, but also in the ability to use them to perform goal-directed navigation.
Q3: Performance of recurrent and spatial-memory baselines. We have tested recurrent (the RecurrentPPO implementation in StableBaselines) in preliminary experiments, but even with some parameter tuning, its performance was always near zero and it was not able to train well. For this reason, we have instead focused on Dreamer as a strong RL baseline with memory (and evaluated it with multiple hyperparameter options in the curriculum experiment). If by spatial-memory you mean the SRU spatial recurrent unit presented in Yang et al. 2026 (IJRR), we have not tested this, but future investigations could definitely test whether the spatially-enhanced memory systems can perform well in ForagerSim.