- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces FireVLN-Bench, comprising 60 indoor scenes with paired clean/degraded observations and two simulated evaluation tracks: continuous navigation and execution with robot dynamics. FireNav freezes the navigator and reconstruction encoder, uses differences in action prediction loss to supervise selective fusion of reconstruction features, and limits interference through bounded residuals and clean-observation retention. Compared with Restored RGB + Adapter, it improves SPL by 4.9/6.0 percentage points on PE/CE, supported by component ablations, evidence interventions, and efficiency analyses.
Critical strengths
- The method directly addresses the stated problem and receives targeted empirical support.
Preserving the native visual pathway while admitting reconstruction evidence according to navigation utility reasonably addresses the risk of reconstruction corrupting task-relevant cues. Removing utility calibration reduces PE/CE SPL by 4.0/4.9 points.
- The closed-loop gains are substantial, with favorable computational overhead.
Under a shared navigation backbone and execution setup, FireNav outperforms reconstructed-RGB and navigation-adapter baselines, while routing improvements extend across multiple reconstruction encoders.
Critical weaknesses
- Decision utility calibration is not fully specified for the slow planning branch.
Equation (4) and Appendix A.3 define evidence utility through waypoint, heading, and arrival prediction losses, whereas the slow branch produces subgoals; the paper does not explain how an evidence intervention in this branch translates into action predictions, including whether subgoal selection and the fast policy are rerun. Since routing across both branches is part of the proposed method, this omission prevents reproduction of a central training procedure rather than merely leaving out an implementation detail.
- The evaluation does not establish whether the gains reflect safer navigation decisions in fire environments.
The benchmark explicitly provides risk-aware paths accounting for fire proximity, dense smoke, and blocked routes, yet the paper does not specify which paths supply expert targets or how SPL accounts for safety-driven detours, while Table 9 reports only collisions, falls, stuck episodes, and false stops. Consequently, the results cannot distinguish improved hazard avoidance from more efficient goal reaching under visual degradation, leaving the benchmark’s fire-specific value insufficiently validated.
Questions for authors
-
How is utility calibration implemented for the slow planning branch? After intervening on its evidence, do you regenerate the subgoal and run the fast policy to compute the action loss, and how are gradients handled through discrete subgoal selection?
-
Does isolated single-view utility predict the benefit of joint evidence fusion? Please report the fraction of admitted evidence with positive utility on held-out states, the rejection rate of harmful evidence, and whether utility signs remain consistent when other evidence is already admitted.
-
How are language instructions, expert trajectories, and safety evaluation aligned? Please clarify instruction quality control, whether expert targets follow shortest or risk-aware paths, and how SPL is normalized; can existing trajectories provide fire-proximity or smoke-exposure measurements?
-
Are the reported gains reproducible and statistically robust? Please provide the training and evaluation configurations, baseline parameter counts, and uncertainty estimates across training seeds and test scenes.