- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PARSE, an LLM-guided reward design framework for learning when an agent should request expert assistance. It trains a preference-conditioned interaction policy to balance task success and interaction cost while keeping the navigation policy frozen. PARSE uses GPT-5.5 to generate and iteratively refine reward functions through mutation, crossover, and exploration. Candidate rewards are evaluated across 11 preference settings and selected based on their Pareto hypervolume, interaction-rate coverage, and monotonicity. Experiments on unseen HM3D PointNav scenes show that PARSE achieves the highest hypervolume (0.7581 vs. 0.6715 for random help) and 82.8% success at a 20% interaction rate, outperforming the strongest baseline by 4.9 percentage points. The learned reward also transfers to ObjectNav without modification.
Critical strengths
1. Evaluating a reward by the set of behaviors its preference-conditioned policy induces, with screening for span and monotonicity, fits deployment settings with varying assistance budgets. The collapse of scalar-objective search to extremes (99.98% vs. 0.01% interaction) illustrates the motivation well.
2. Training, ranking and feedback use disjoint MP3D splits, and final evaluation uses unseen HM3D scenes. The appendix provides full prompts, the final reward code and per-method compute budgets.
3. A single policy covers a wide range of interaction rates, with monotone control and smooth interpolation to unseen λ (App. E). The final reward (Eq. 13) is readable and transfers to ObjectNav.
Critical weaknesses
-
The baselines do not isolate the contribution of the search.
- The Eureka variants use objectives that degenerate by construction. With a shortest-path expert, maximizing Success means always asking, and the ratio ΔSuccess/IR drives interaction to zero. A fair scalar baseline would rank candidates by the HV of their own preference-response set, which is PARSE's own final selection criterion.
- The key control is missing: the human reward plus a λ-dependent help cost, trained as a preference-conditioned policy.
- The human-reward results are taken from Igbinedion & Karaman (2024) rather than rerun with the same frozen policy, expert and episodes.
-
Statistical evidence is not enough.
- The search samples the LLM at temperature 1.0 but appears to be run only once for PARSE and for each ablation, so the ablation gaps (0.7581 vs. 0.6782 and 0.7058) may fall within search-to-search variance.
- HV largely rewards coverage: the human reward scores below random help (0.6089 vs. 0.6715), and baseline coverage depends on the authors' sweep grids.
- The matched-rate comparison is a single spline-interpolated point (20% interaction). The HULA, MC and random-help baselines use one run each, without confidence intervals.
-
Limited scope and novelty. Evaluation covers one navigation task family with a perfect planner as the expert. The components (LLM evolutionary reward search, hypervolume-contribution selection, preference-conditioned policies) are established. The search costs 45 training runs of 250k steps plus 96 GPT-5.5 calls, for a 4.9-point gain at one operating point.
Questions for authors
See weaknesses