- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Fail2Scene, a failure-driven curriculum learning framework for embodied navigation. The framework optimizes navigation policies via reinforcement learning by diagnosing the causes of navigation failures, identifying recurring failure patterns, and generating targeted training environments.
Critical strengths
-
The paper addresses a critical challenge in embodied navigation: how to leverage an agent’s own failure experiences to construct more informative training environments, rather than simply collecting more trajectories or generating scenes randomly.
-
On the unseen GOAT-Bench environment, Fail2Scene achieves a Success Rate (SR) of 70.50% and a Success weighted by Path Length (SPL) of 47.62%—outperforming the "Random Curriculum" baseline by 9.35 and 4.46 percentage points, respectively. Performance improvements on A-EQA tasks further demonstrate the method's versatility across various embodied tasks.
-
The authors analyze several key components of the framework, including the identification and validation of failure mechanisms, as well as the adaptive curriculum update process. These experiments provide robust evidence for the effectiveness of the overall framework.
Critical weaknesses
-
The proposed framework primarily integrates existing techniques—such as Large Language Model (LLM)-based failure attribution, scene generation, curriculum learning, and the GRPO algorithm. While the overall pipeline is effective, the technical innovation beyond the combination of these existing components is relatively limited.
-
Although the paper compares Fail2Scene against baselines like "Random Curriculum," "Failure Replay," and SAGE, it lacks comparisons with other highly relevant adaptive curriculum learning and environment generation methods, such as "Prioritized Level Replay" or "regret-based environment design." Including these comparisons would better highlight the advantages of the strategy of "generating environments based on failure mechanisms."
-
While the paper confirms that the generated scenarios do indeed contain the diagnosed failure conditions, it remains unclear whether the resulting training samples consistently and effectively represent these conditions. Furthermore, differences in scenario complexity and trajectory distribution could also be contributing factors to the observed performance improvements.
-
The experimental section reports results from only a single independent training run, with no estimates of variance provided. Although the appendix includes experiments with real-world robots, these are limited to qualitative demonstrations and lack systematic, quantitative comparisons.
Questions for authors
see weaknesses