acceptodds
Under review as a conference paper at ICLR 2027

IR-SIM: A Lightweight Declarative Simulator for Navigation Learning and Benchmarking

Abstract

Developing navigation policies requires simulation scenarios that support repeatable training and evaluation. Despite the availability of numerous simulators, constructing diverse navigation scenarios often involves writing simulator-specific code, using complex graphical interfaces, performing substantial manual configuration, or relying on resource-intensive computing platforms, making scenarios difficult to reproduce and reuse across experiments. To this end, we develop the Intelligent Robot Simulator (IR-SIM), a lightweight declarative simulator implemented as a Python library to support navigation learning and benchmarking through rapid construction of reusable and diverse scenarios and efficient execution on accessible platforms. IR-SIM represents scenarios as human-readable YAML configurations, in which necessary objects, behaviors, sensors, maps, and environment parameters can be flexibly specified and composed. This declarative representation makes IR-SIM friendly to large language model (LLM)-powered agents, enabling them to compose and modify executable navigation scenario files through the provided agent skills, instead of writing a large amount of code that is difficult to reuse. Despite its lightweight implementation, IR-SIM provides the essential components for navigation simulation, while retaining interfaces to high-fidelity simulators for downstream validation. Experiments demonstrate that IR-SIM runs up to several hundred times faster on the evaluated CPU platform, while agent skills reduce mean LLM-based scenario construction time by more than 50% across both evaluated models. The experiments further show ability to support reproducible benchmarking and RL-based navigation policy learning.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Lightweight robot simulator, reinforcement learning, LLM in simulation, Python Library

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ssUQOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

IR-SIM is a lightweight 2D simulator for navigation learning and benchmarking, including multi-robot collision avoidance and robot navigation among pedestrians. YAML configurations specify robots, obstacles, kinematics, behaviors, and sensors, while custom controllers can be integrated through Python. The system provides CPU and batched GPU implementations, together with skills that help coding agents construct and run scenarios. Experiments cover cross-simulator execution speed, scenario construction with two language models, benchmarking of eight navigation controllers, and SARL policy training.

Critical strengths
  1. The system has practical value for navigation research requiring repeated scenario construction and execution. Scenario configurations, common motion behaviors, sensors, and learning interfaces are integrated into one tool, potentially reducing repeated development work in multi-robot navigation experiments.
  2. Tests cover different robot counts, LiDAR and sensor-free settings, single-core CPU execution, and batched GPU execution, with shared layouts and recorded input actions. IR-SIM is faster in the evaluated large-scale multi-robot settings
  3. Scenario construction is evaluated through execution and requirement satisfaction. The validator checks execution, scenario requirements, task completion, and collisions. On the evaluated tasks, skills reduce Opus's mean authoring time from 204 to 98 seconds and achieve approximately 95% scenario validity with both models.
Critical weaknesses
  1. Section 4.1 emphasizes cross-simulator speed advantages. However, Appendix B.1 states that MVSim uses octagonal robots and Box2D contact forces, whereas IR-SIM uses circular robots and a collision-stop rule in this test. Matching layouts and input actions does not ensure the same simulation workload. The timing impact of these differences is not assessed.
  2. Appendix C.3 states that all conditions receive documentation and source code, but only the IR-SIM skill condition additionally receives the authors' skill package. No corresponding skill-package condition is reported for VMAS or PyRoboSim. Table 2 therefore supports the practical advantage of the supplied IR-SIM skills package, but does not establish how much of the authoring-time and validity gap would remain if other tools received similar support.
  3. The second contribution highlights approximately 95% scenario validity. However, Appendix C.6 explicitly states that skill version 0.9.1 was refined using failures from these brief families, and describes the result as an upper bound rather than an independent measurement. No results are reported on new tasks after freezing the skills.
Questions for authors
  1. For the comparison with MVSim, can collision-handling requirements be aligned, or timings be reported separately for motion updates, collision handling, and LiDAR? How much do octagonal versus circular footprints and contact forces versus stopping affect the results?
  2. Can at least one baseline, VMAS or PyRoboSim, receive comparable API guides, scenario templates, and behavior-parameter guidance, followed by an authoring-time and validity comparison under the same model and tool budgets?
  3. After freezing the skill version, can scenario validity and authoring time be evaluated on new tasks not used for development or debugging?
7 comments
AuthorsOpenReview ↗

Author Response Part 1

We appreciate the reviewer’s feedback regarding the fair comparison and skill generalization. W1 & Q1 Fair Comparison with MVSim: We acknowledge the reviewer’s concern regarding the fair comparison with MVSim: the stop rule and the circular robot footprint in IR-SIM lead to a lighter simulation workload. Thus, we have added an experiment that compares IR-SIM with MVSim under the same robot footprint and collision handling requirements, and we report the time per component as requested. The results are summarized in the following table:

Time per component (ms per step at 10 / 100 / 1,000 robots)

PlatformSimulatorConfigurationMotion update (ms)Collision handling (ms)LiDAR (ms)Total (ms)Speedup over MVSim
CPUMVSimreference0.0378 / 0.36 / 60.0285 / 0.31 / 3.740.277 / 6.74 / 96.20.344 / 7.42 / 1061.00x / 1.00x / 1.00x
CPUIR-SIMdefault (circular robots)0.137 / 1.03 / 11.50.109 / 0.398 / 5.080.308 / 3.09 / 450.611 / 4.58 / 65.20.56x / 1.62x / 1.63x
CPUIR-SIMaligned with MVSim (robot footprint and collision handling)0.18 / 1.37 / 14.40.187 / 0.862 / 11.10.502 / 9.76 / 1190.918 / 12.0 / 1510.37x / 0.62x / 0.70x
GPUIR-SIMaligned with MVSim (robot footprint and collision handling)1.07 / 1.09 / 1.152.24 / 2.78 / 3.060.923 / 0.965 / 5.244.15 / 4.77 / 9.070.08x / 1.56x / 11.7x

From the results, under the same robot footprint and collision handling requirements, on the CPU, MVSim is 1.4 to 2.7 times faster than IR-SIM, which confirms the reviewer’s concern. Between the default and the aligned IR-SIM rows, the octagonal footprint accounts for most of the difference: it raises the IR-SIM step time by 1.3 / 2.4 / 2.2 times at 10 / 100 / 1,000 robots, mainly in the LiDAR, and the contact handling adds a further 6-17%.

On the GPU, under the same requirements, IR-SIM is 1.56x and 11.7x faster than MVSim at 100 and 1,000 robots, while MVSim is faster at 10 robots, where the fixed GPU overhead dominates. MVSim is compared in its CPU mode, which is its fastest configuration. This demonstrates the advantage of the IR-SIM batching mechanism accelerated by the GPU at large numbers of robots. We will modify our claim on MVSim comparison and add these results to the revised paper.

W2 & Q2 Comparable API Guides and Support for VMAS: We provide IR-SIM-skill-style support for VMAS: a support package with API guides, scenario and runner templates, behavior guidance, and review lists. As we are not the developers of VMAS, we built this package with a coding agent (Fable 5.2) and checked every statement in it against VMAS. It was written without access to the unseen scenarios, frozen before they were run, and evaluated with the same models and harness. The results are summarized in the following table:

Unseen scenarios U1–U6, with a support package for VMAS

SimulatorExecutable (%)Valid (%)Spec. acc. (%)TurnsTime (s)Cost (USD)Lines PythonLines YAML
Opus 5.5
IR-SIM + skills100.0100.0100.016.21920.8911659
VMAS + support package100.0100.0100.013.91730.732210
Sonnet 5.5
IR-SIM + skills94.466.791.810.1640.228155
VMAS + support package100.0100.0100.013.3820.281550
Haiku 5.5
IR-SIM + skills100.066.796.919.61960.0413060
VMAS + support package100.088.997.814.72030.052030

With the support package, VMAS achieves the same validity as IR-SIM + skills with Opus 5.5, and a higher validity with Sonnet 5.5 and Haiku 5.5. We therefore acknowledge that skill-style support also works for other simulators. The advantage of IR-SIM + skills is a shorter deliverable with all three models and. In addition, VMAS does not support declarative configuration: its scenario is task specific Python code (155–221 lines here), which is hard to reuse for another task or controller, while by design the IR-SIM scenario is a short YAML file (55–60 lines) that is separate from the control code and can be reused.

We will add these results to the revised paper and narrow our agent skill contribution accordingly: we will no longer claim a validity advantage from the skills over other simulators, we will state that the skills mainly reduce the turns and the code of IR-SIM scenarios, and we will emphasize the compact, reusable declarative scenarios and the lower authoring time and cost with the weaker models.

AuthorsOpenReview ↗

Author Response Part 2

W3 & Q3 Agent skill generalization: We clarify that the IR-SIM skills were refined based on failures by adding general rules to improve the skill performance, rather than tailoring the skills to the specific evaluated task families. To validate the effectiveness of the IR-SIM skills in unseen scenario families, we have added an experiment on six unseen scenarios (U1–U6), which were not used during the skill design and the skill is frozen, with three models (Opus 5.5, Sonnet 5.5, and Haiku 5.5) and three baselines (IR-SIM, VMAS, and PyRoboSim). The results are summarized in the following table:

Unseen scenarios U1–U6

SimulatorExecutable (%)Valid (%)Spec. acc. (%)TurnsTime (s)Cost (USD)Lines PythonLines YAML
Opus 5.5
IR-SIM + skills100.0100.0100.016.21920.8911659
IR-SIM83.383.383.320.81910.8815665
VMAS100.0100.0100.023.12140.842750
PyRoboSim94.472.292.821.92090.8723519
Sonnet 5.5
IR-SIM + skills94.466.791.810.1640.228155
IR-SIM83.377.882.912.1670.1910753
VMAS88.955.685.913.9850.231870
PyRoboSim88.966.783.218.9900.2816612
Haiku 5.5
IR-SIM + skills100.066.796.919.61960.0413060
IR-SIM100.072.297.820.82040.0414456
VMAS77.850.072.820.02580.062540
PyRoboSim88.950.085.723.82390.0623417

From the unseen scenarios U1–U6 results, we can see that IR-SIM + skills consistently achieves the highest executable percentages across all three models, which shows that the skills carry over to unseen scenarios in producing executable scenarios. We acknowledge that, for the weaker models, the validity on the unseen scenarios is lower than the approximately 95% reported on the original briefs, and we narrow our claim accordingly.

In addition, we acknowledge that the validity improvement with Opus 5.5 over the other simulators is not statistically significant. The reason is that Opus 5.5 is a stronger model, which can construct valid scenarios directly without strongly relying on the skills or the simulator itself.

However, for the weaker models, Sonnet 5.5 and Haiku 5.5, IR-SIM, with or without the skills, shows higher validity than VMAS, and its cost and time are also the lowest among all the simulators, which indicates that our simulator design can effectively enhance the performance of weaker models in unseen scenarios. For these models, the skills do not raise the validity further; they reduce the turns and the code, and raise the executable percentage with Sonnet 5.5.

Moreover, when VMAS receives a comparable support package, it reaches the same or higher validity than IR-SIM + skills (100.0 / 100.0 / 88.9% against 100.0 / 66.7 / 66.7% with Opus 5.5 / Sonnet 5.5 / Haiku 5.5; see W2 & Q2 above), so skill-style support also works for other simulators.

In the revision, we will narrow our agent skill contribution accordingly: we will not claim a validity advantage from the skills, and we will emphasize the compact declarative scenarios and the lower authoring time and cost of IR-SIM with the weaker models. We will also report the results on the unseen scenarios.

I would like to thank the authors for their detailed supplementary experiments; the more objective comparison has given me a deeper understanding of this method’s strengths and weaknesses.

Reviewer_BAHsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

IR-SIM is a Python simulator for planar robot navigation. YAML configurations compose robot models, geometry, behaviors and sensors, with vectorized execution and optional GPU batching. The paper also provides skills for LLM-based scenario construction and interfaces for policy learning and external simulators. Evaluation covers CPU/GPU execution, scenario authoring with two LLMs, a multi-robot navigation benchmark and SARL training through TorchRL.

Critical strengths

The simulator combines reusable scene descriptions, built-in behaviors and learning interfaces in a practical Python workflow. I see value in making repeatable navigation experiments easier to set up, and the authoring validator checks actual behavior rather than just whether generated code runs.

The execution benchmarks use shared inputs and explicit timing boundaries that include sensing and collision checks. At 1,000 robots with LiDAR, IR-SIM takes 178 ms per step versus 280 ms for MVSim (Table 5), and the batched GPU results also show useful throughput gains over JaxNav and VMAS.

Critical weaknesses

The strongest authoring result is not an independent test: App. C.6 says the skills were refined on failures from the evaluated brief families. With only IR-SIM receiving curated skills, the roughly 95% validity and time savings leave open how much transfers to new scenarios or comes from the simulator interface itself.

The large RL speedups provide limited evidence of an advantage from the simulator itself. Table 12's matched TorchRL control takes 0.74 h with CrowdSim versus 0.76 h with IR-SIM for five humans, and 1.44 versus 1.27 h for ten, so most of the gain over the original pipeline comes from batching.

Questions for authors

How do frozen skills perform on previously unused scenario families, with comparably curated API recipes for the baselines?

Does IR-SIM retain a training-time advantage over batched CrowdSim when both use GPU network inference and updates? Table 12 includes the matched CPU comparison, but only the IR-SIM pipeline uses GPU network computation.

3 comments
AuthorsOpenReview ↗

Author Response Part 1

We appreciate the reviewer’s feedback regarding the validation of experimental gains. We have added experiments to demonstrate agent-skill generalization to unseen scenario families (W1 & Q1) and clarify the RL training speedup benefits with IR-SIM by adding more navigation policies training (W2 & Q2).

W1 & Q1 Agent skill generalization. We clarify that the IR-SIM skills were refined based on failures by adding general rules to improve the skill performance, rather than tailoring the skills to the specific evaluated task families. To validate the effectiveness of the IR-SIM skills in unseen scenario families, we have added an experiment on six unseen scenarios (U1–U6), which were not used during the skill design and the skill is frozen, with three models (Opus 5.5, Sonnet 5.5, and Haiku 5.5) and three baselines (IR-SIM, VMAS, and PyRoboSim). The results are summarized in the following table:

Unseen scenarios U1–U6

SimulatorExecutable (%)Valid (%)Spec. acc. (%)TurnsTime (s)Cost (USD)Lines PythonLines YAML
Opus 5.5
IR-SIM + skills100.0100.0100.016.21920.8911659
IR-SIM83.383.383.320.81910.8815665
VMAS100.0100.0100.023.12140.842750
PyRoboSim94.472.292.821.92090.8723519
Sonnet 5.5
IR-SIM + skills94.466.791.810.1640.228155
IR-SIM83.377.882.912.1670.1910753
VMAS88.955.685.913.9850.231870
PyRoboSim88.966.783.218.9900.2816612
Haiku 5.5
IR-SIM + skills100.066.796.919.61960.0413060
IR-SIM100.072.297.820.82040.0414456
VMAS77.850.072.820.02580.062540
PyRoboSim88.950.085.723.82390.0623417

From the unseen scenarios U1–U6 results, we can see that IR-SIM + skills consistently achieves the highest executable percentages across all three models, which shows that the skills carry over to unseen scenarios in producing executable scenarios. We acknowledge that, for the weaker models, the validity on the unseen scenarios is lower than the approximately 95% reported on the original briefs, and we narrow our claim accordingly.

In addition, we acknowledge that the validity improvement with Opus 5.5 over the other simulators is not statistically significant. The reason is that Opus 5.5 is a stronger model, which can construct valid scenarios directly without strongly relying on the skills or the simulator itself.

However, for the weaker models, Sonnet 5.5 and Haiku 5.5, IR-SIM, with or without the skills, shows higher validity than VMAS, and its cost and time are also the lowest among all the simulators, which indicates that our simulator design can effectively enhance the performance of weaker models in unseen scenarios. For these models, the skills do not raise the validity further; they reduce the turns and the code, and raise the executable percentage with Sonnet 5.5.

Moreover, when VMAS receives a comparable support package, it reaches the same or higher validity than IR-SIM + skills (100.0 / 100.0 / 88.9% against 100.0 / 66.7 / 66.7% with Opus 5.5 / Sonnet 5.5 / Haiku 5.5), so skill-style support also works for other simulators.

In the revision, we will narrow our agent skill contribution accordingly: we will not claim a validity advantage from the skills, and we will emphasize the compact declarative scenarios and the lower authoring time and cost of IR-SIM with the weaker models. We will also report the results on the unseen scenarios.

AuthorsOpenReview ↗

Author Response Part 2

W2 & Q2: RL training speedup: We agree with the reviewer that, under matched batching, the additional benefit over the batched CrowdSim in Table 12 is small. However, we clarify that the batched CrowdSim is not the original CrowdSim implementation, which does not support batching: it runs the CrowdSim simulator inside the batched environment that we built for IR-SIM (the IR-SIM batch bridge), so the batching gain in Table 12 also comes from our IR-SIM training interface. Simulation batching, including both sensor and world batching, is a core mechanism of IR-SIM that contributes to efficient RL training.

As requested, we add the comparison with the batched CrowdSim (through the IR-SIM batch bridge), with GPU network inference and updates in both pipelines:

HumansConfigMethodSim. time (ms)Net. time (ms)Time (h)Success (%)Collision (%)Nav. time (s)
5C (GPU)IR-SIM batch bridge + CrowdSim + TorchRL0.290.060.429100.00.010.45
5D (GPU)IR-SIM + TorchRL0.350.080.443100.00.010.45
10C (GPU)IR-SIM batch bridge + CrowdSim + TorchRL1.290.090.73199.80.012.41
10D (GPU)IR-SIM + TorchRL0.670.120.57199.80.012.25
20C (GPU)IR-SIM batch bridge + CrowdSim + TorchRL5.580.132.18099.00.417.80
20D (GPU)IR-SIM + TorchRL1.180.130.77798.81.017.54

With GPU network inference and updates in both pipelines, the two are on par at 5 humans (IR-SIM is about 3% slower), IR-SIM needs 22% less training time at 10 humans, and 64% less at 20 humans. The difference comes from the simulator time, which grows with the crowd size faster for CrowdSim (0.29, 1.29 and 5.58 ms per decision at 5, 10 and 20 humans) than for IR-SIM (0.35, 0.67 and 1.18 ms), while the network time is comparable for both.

The advantage of IR-SIM therefore lies in scenes with a large number of agents. The 20-human pair uses a proportionally larger crossing circle to keep the crowd density. We will add these rows to Table 12 in the revised paper.

Additionally, to further highlight the benefits of IR-SIM with batch mechanism, we add the experiment of RL training using the NavRL algorithm [1] in IR-SIM and compared the training time with that of NavRL's implementation in Isaac Sim. The results are as follows:

Training simulatorTraining time (h)Success at 60 moving obstacles (%)Success at 80 moving obstacles (%)Success at 100 moving obstacles (%)Success at 120 moving obstacles (%)
Isaac Sim (GPU)9.9778.5 ± 2.565.7 ± 2.567.5 ± 2.754.0 ± 4.8
IR-SIM (GPU)1.0192.9 ± 2.486.1 ± 1.986.2 ± 4.878.9 ± 4.7

Success rates are the mean ± standard error over the evaluations in Isaac Sim with 1,024 robots each (10 for the Isaac Sim row, 5 for the IR-SIM row).

Because IR-SIM is a 2D simulator, both training runs use a planar adaptation of NavRL with the same PPO hyperparameters, and both policies are evaluated in Isaac Sim with NavRL's evaluation protocol. Each run uses the same NVIDIA RTX 4090 (24 GB) separately.

From the results, we observe that about 1 h of training in IR-SIM achieves higher success rates than about 10 h of training in Isaac Sim. The key reason is that IR-SIM’s batched simulation is lightweight: on the same GPU it trains 16,384 robots in parallel (6.1 GB), where Isaac Sim fits 1,536 (19.1 GB), and it processes 3.0 times more frames per second. This shows that IR-SIM can significantly accelerate RL training on another navigation task, and that its advantage lies in simulating a large number of robots.

[1] Xu, Zhefan, et al. "NavRL: Learning safe flight in dynamic environments." IEEE Robotics and Automation Letters 10.4 (2025): 3668-3675.

Dear Reviewer BAHs,

As the discussion period ends in just over one day (November 18, AoE), we would like to follow up on our response. We would sincerely appreciate it if you could let us know whether it has addressed your concerns.

To summarize our response briefly:

  • Agent skill generalization (W1 & Q1). We evaluated the frozen IR-SIM skill package on six unseen scenarios. With all three models, the skills reduced the number of authoring turns and the amount of generated Python code, but scenario validity did not improve consistently, so we will narrow our claim accordingly in the revision.
  • RL training speedup (W2 & Q2). The original CrowdSim implementation does not support batching, the batched CrowdSim baseline in Table 12 runs inside the batched environment we built for IR-SIM, so batching is itself a mechanism of IR-SIM. As requested, we added the comparison with GPU network inference and updates in both pipelines. With 5 humans, training times are comparable. While, IR-SIM needs 22% less training time with 10 humans and 64% less with 20 humans, so the advantage grows with the crowd size. We also added an RL training experiment on another navigation task using NavRL, where about 1 h of training in IR-SIM achieves higher success rates than about 10 h in NavRL's Isaac Sim implementation, with both policies evaluated in Isaac Sim.

In addition, we will open-source the full IR-SIM codebase and the skill package for the community to use and improve.

If our response has addressed your concerns, we would be grateful if you would consider updating your assessment. We are glad to clarify any remaining points.

Thank you for your time and constructive feedback.

Best regards,
The Authors

Reviewer_MgY5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents IR-SIM, a lightweight declarative simulator for navigation learning and benchmarking. It supports YAML-based scenario construction, different robot models, sensor simulation, and LLM-assisted authoring through agent skills. Experiments evaluate simulation efficiency, scenario generation, and RL training performance against existing simulators.

Critical strengths

1. Practical and well-developed system. IR-SIM provides a convenient framework for constructing reusable navigation scenarios. Its support for different robot models, sensors, and external simulator interfaces makes it a potentially useful tool for robotics research.

2. Encouraging efficiency results. The experiments show strong simulation efficiency, particularly in large multi-robot scenarios. The agent skills also substantially reduce scenario construction time and cost, with approximately 95% validity across two evaluated models.

Critical weaknesses

1. Limited methodological novelty. The main contributions focus on integrating existing simulation techniques, declarative configurations, and agent skills. While the engineering effort is valuable, the methodological advances over existing lightweight simulators could be more clearly established.

2. Further validation of the experimental gains would be helpful. Table 12 shows that much of the RL training speedup comes from batching, with a smaller additional benefit from IR-SIM in the tested settings. For agent skills, the validity improvement with Opus is not statistically significant (p=0.43), and the skills were refined using failures from the evaluated task families. This leaves some uncertainty about their effectiveness on unseen scenarios.

Questions for authors
  1. Could the authors further clarify the methodological contributions of IR-SIM compared with existing lightweight declarative simulators?

  2. Could the authors provide evidence of agent-skill generalization to unseen scenario families and clarify whether the additional RL training speedup observed under matched batching extends to other navigation tasks?

5 comments
AuthorsOpenReview ↗

Author Response Part 1

We appreciate the reviewer’s feedback regarding the methodological novelty and the experimental validation. We clarify the methodological contributions of IR-SIM (W1 & Q1), add agent skill generalization experiments with unseen scenarios (W2 & Q2), and add RL training speedup experiments to clarify the RL training benefits with IR-SIM (W2 & Q2).

W1 & Q1 methodological contributions. We clarify that IR-SIM’s main methodological contribution is parameterized and batched simulation of complex scenarios. The dominant scenario computations, including LiDAR ray casting, collision queries involving heterogeneous objects, group behavior simulation, and multi-world simulation, are parameterized and batched, and the kinematic state updates are batched. This enables batched state updates and inference, supporting highly efficient simulation and RL training in CPU and GPU platforms.

The technical challenge is to run geometric queries efficiently when a scene contains moving robots and obstacles of different shapes. IR-SIM handles this with a batched query pipeline: the scene geometry is converted into shared arrays, static geometry is cached, only nearby sensor and object pairs are tested, and the intersections are computed in blocks. Each sensor then keeps its nearest hit, and collision detection also uses batched spatial queries over all object shapes. This removes repeated geometric work and per-query overhead, while scenarios can still be composed freely.

Other lightweight declarative simulators, such as PyRoboSim and MVSim, do not support batching, while the batched simulators such as VMAS and GPUDrive, use a limited set of built-in agent shapes because they focus on the subfield of navigation, which limits their applicability to more complex scenarios.

Finally, we also would like to clarify that our submission falls within the primary area of infrastructure, software libraries, hardware, and systems, with an emphasis on system level contributions in design and implementation. We would be grateful if the reviewer could reconsider the concern regarding methodological novelty, taking into account the system level contributions of IR-SIM for navigation simulation and reinforcement learning research.

W2 & Q2: RL training speedup. We agree with the reviewer that, under matched batching, the additional benefit over the batched CrowdSim in Table 12 is small. However, we clarify that the batched CrowdSim is not the original CrowdSim implementation, which does not support batching: it runs the CrowdSim simulator inside the batched environment that we built for IR-SIM, so the batching gain in Table 12 also comes from our IR-SIM training interface. Simulation batching, including both sensor and world batching, is a core mechanism of IR-SIM that contributes to efficient RL training.

Thus, IR-SIM’s benefits for RL-based navigation policies training could extend to other navigation tasks. As supporting evidence, we have added RL training navigation experiments using the NavRL algorithm [1] on IR-SIM platform and compared the training time with that of NavRL's implementation in Isaac Sim. The results are as follows:

Training simulatorTraining time (h)Success at 60 moving obstacles (%)Success at 80 moving obstacles (%)Success at 100 moving obstacles (%)Success at 120 moving obstacles (%)
Isaac Sim (GPU)9.9778.5 ± 2.565.7 ± 2.567.5 ± 2.754.0 ± 4.8
IR-SIM (GPU)1.0192.9 ± 2.486.1 ± 1.986.2 ± 4.878.9 ± 4.7

Since IR-SIM is a 2D simulator, both training runs use a planar adaptation of NavRL with the same PPO hyperparameters, and both policies are evaluated in Isaac Sim with NavRL's evaluation protocol. Each run uses the same NVIDIA RTX 4090 (24 GB) separately.

From the results, the policy trained in IR-SIM for about 1.01h achieves higher success rates than the policy trained in Isaac Sim for about 9.97h. The key reason is that IR-SIM’s batched simulation is lightweight: on the same GPU it trains 16,384 robots in parallel (6.1 GB), where Isaac Sim fits 1,536 (19.1 GB), and it processes 3.0 times more frames per second. This shows that IR-SIM can significantly accelerate RL training on another navigation task while maintaining or improving policy performance.

[1] Xu, Zhefan, et al. "NavRL: Learning safe flight in dynamic environments." IEEE Robotics and Automation Letters 10.4 (2025): 3668-3675.

AuthorsOpenReview ↗

Author Response Part 2

W2 & Q2 Agent skill generalization. We clarify that the IR-SIM skills were refined based on failures by adding general rules to improve the skill performance, rather than tailoring the skills to the specific evaluated task families. To validate the effectiveness of the IR-SIM skills in unseen scenario families, we have added an experiment on six unseen scenarios (U1–U6), which were not used during the skill design and the skill is frozen, with three models (Opus 5.5, Sonnet 5.5, and Haiku 5.5) and three baselines (IR-SIM, VMAS, and PyRoboSim). The results are summarized in the following table:

Unseen scenarios U1–U6

SimulatorExecutable (%)Valid (%)Spec. acc. (%)TurnsTime (s)Cost (USD)Lines PythonLines YAML
Opus 5.5
IR-SIM + skills100.0100.0100.016.21920.8911659
IR-SIM83.383.383.320.81910.8815665
VMAS100.0100.0100.023.12140.842750
PyRoboSim94.472.292.821.92090.8723519
Sonnet 5.5
IR-SIM + skills94.466.791.810.1640.228155
IR-SIM83.377.882.912.1670.1910753
VMAS88.955.685.913.9850.231870
PyRoboSim88.966.783.218.9900.2816612
Haiku 5.5
IR-SIM + skills100.066.796.919.61960.0413060
IR-SIM100.072.297.820.82040.0414456
VMAS77.850.072.820.02580.062540
PyRoboSim88.950.085.723.82390.0623417

From the unseen scenarios U1–U6 results, we can see that IR-SIM + skills consistently achieves the highest executable percentages across all three models, which shows that the skills carry over to unseen scenarios in producing executable scenarios. We acknowledge that, for the weaker models, the validity on the unseen scenarios is lower than the approximately 95% reported on the original briefs, and we narrow our claim accordingly.

In addition, we acknowledge that the validity improvement with Opus 5.5 over the other simulators is not statistically significant. The reason is that Opus 5.5 is a stronger model, which can construct valid scenarios directly without strongly relying on the skills or the simulator itself.

However, for the weaker models, Sonnet 5.5 and Haiku 5.5, IR-SIM, with or without the skills, shows higher validity than VMAS, and its cost and time are also the lowest among all the simulators, which indicates that our simulator design can effectively enhance the performance of weaker models in unseen scenarios. For these models, the skills do not raise the validity further; they reduce the turns and the code, and raise the executable percentage with Sonnet 5.5.

Moreover, when VMAS receives a comparable support package, it reaches the same or higher validity than IR-SIM + skills (100.0 / 100.0 / 88.9% against 100.0 / 66.7 / 66.7% with Opus 5.5 / Sonnet 5.5 / Haiku 5.5), so skill-style support also works for other simulators.

In the revision, we will narrow our agent skill contribution accordingly: we will not claim a validity advantage from the skills, and we will emphasize the compact declarative scenarios and the lower authoring time and cost of IR-SIM with the weaker models. We will also report the results on the unseen scenarios.

Dear Reviewer MgY5,

As the discussion period ends in just over one day (November 18, AoE), we would like to follow up on our response. We would sincerely appreciate it if you could let us know whether it has addressed your concerns.

To summarize our response briefly:

  • Methodological contribution (W1 & Q1). The main methodological contribution of IR-SIM is the parameterized and batched simulation of complex scenarios, including batched LiDAR and collision queries over objects of different shapes. Existing lightweight declarative simulators (PyRoboSim, MVSim) do not support batching, and existing batched simulators (VMAS, GPUDrive) use a limited set of built-in shapes.
  • RL training speedup (W2 & Q2). The batched CrowdSim baseline in Table 12 runs inside the batched environment we built for IR-SIM, so batching is itself a mechanism of IR-SIM. As supporting evidence on another navigation task, NavRL, about 1 h of training in IR-SIM reaches higher success rates than about 10 h in NavRL's Isaac Sim implementation, with both policies evaluated in Isaac Sim.
  • Agent skill generalization (W2 & Q2). We evaluated the frozen skill package on six unseen scenarios. With all three models, the skills reduced the number of authoring turns and the amount of generated Python code. Scenario validity did not improve consistently, so we will narrow our claim accordingly in the revision.

In addition, we will open-source the full IR-SIM codebase and the skill package for the community to use and improve.

If our response has addressed your concerns, we would be grateful if you would consider updating your assessment. We are glad to clarify any remaining points.

Thank you for your time and constructive feedback.

Best regards,
The Authors

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

We thank the area chair and reviewers for the time spent on our submission. Below is a brief summary of the main concerns and of how we addressed them in the discussion and in the revised PDF (changes in blue).

  1. Methodological contribution (Reviewer MgY5). We clarified that the main contribution of IR-SIM is the parameterized and batched simulation of complex scenarios, including batched LiDAR and collision queries over objects of different shapes. Existing lightweight declarative simulators (PyRoboSim, MVSim) do not support batching, and existing batched simulators (VMAS, GPUDrive) use a limited set of built-in shapes.

  2. Fair comparison with MVSim (Reviewer ssUQ). We added a comparison with the robot footprint and collision handling aligned, with the time reported per component. On the CPU, MVSim is 1.4 to 2.7 times faster under these aligned settings, and we revised our claim accordingly. On the GPU, under the same settings, IR-SIM is 1.56x and 11.7x faster with 100 and 1,000 robots.

  3. Agent skill generalization (all reviewers). We evaluated the skills on six unseen scenarios with three models and built a comparable support package for VMAS. The skills improve the execution success rate and reduce the authoring turns and the generated code, but scenario validity does not improve consistently, and VMAS with comparable support reaches the same or higher validity. We narrowed our claim accordingly: we no longer claim a validity advantage from the skills, and we emphasize the compact, reusable declarative scenarios and the lower authoring time and cost with weaker models.

  4. RL training speedup (Reviewers MgY5 and BAHs). With GPU networks in both pipelines, IR-SIM and the batched CrowdSim are on par with 5 humans, and IR-SIM needs 22% and 64% less training time with 10 and 20 humans. On another navigation task, NavRL, about 1 h of training in IR-SIM reaches higher success rates than about 10 h in NavRL's Isaac Sim implementation.

  5. We will open-source the full IR-SIM codebase and the skill package for the community to use and improve.

Finally, we sincerely thank the Area Chair and all the reviewers for their time and constructive feedback.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.