- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies long-horizon heterogeneous robot exploration where some actions may provide limited immediate reward but improve future task feasibility. The authors introduce the concept of future team feasibility and propose LIFT, a world-model-based framework that predicts how candidate joint plans affect future executable behaviors. LIFT represents robot plans using semantic action embeddings, learns latent dynamics models to predict future states, rewards, and plan feasibility, and performs imagined rollouts to select actions with higher long-term value. Experiments in simulated multi-robot exploration environments with heterogeneous robots demonstrate that LIFT can improve exploration performance and reduce failures compared with several reinforcement learning baselines.
Critical strengths
-
Interesting problem formulation. The paper identifies an important limitation of conventional exploration objectives: actions that are valuable for future decision making may not provide immediate information gain or reward. The notion of future team feasibility provides an intuitive perspective for long-horizon robot decision making.
-
Well-designed system architecture. The combination of semantic action representations, latent world modeling, and imagination-based planning is technically coherent. The action semantic embedding is a reasonable design choice to handle dynamically changing plan spaces.
-
Meaningful benchmark scenarios. The proposed environments include heterogeneous robots and enabling interactions such as opening doors and operating elevators, which better reflect long-horizon coordination challenges than standard exploration benchmarks.
4.Comprehensive empirical analysis. The paper provides several ablations investigating the role of feasibility prediction, teacher demonstrations, and different components of the proposed framework.
Critical weaknesses
-
The method relies heavily on a highly structured planning abstraction. The proposed approach assumes that a deterministic planner can generate a set of executable joint plans, and LIFT only needs to select among these candidates. This shifts a substantial part of the decision-making difficulty into the handcrafted planner and limits the applicability to settings where such structured action spaces are available.
-
The generality of future team feasibility remains unclear. Although the proposed concept is interesting, the current formulation is tightly coupled to predefined skills and symbolic plan representations. It is unclear whether the same idea can extend to continuous manipulation tasks or scenarios where feasible actions are not explicitly enumerated.
-
The experimental evaluation is limited to simulation. The experiments are conducted in Habitat-based simulation environments. While the benchmark is carefully designed, the paper does not demonstrate whether the proposed approach transfers to real robotic systems or more realistic uncertainties.
-
Comparison with modern model-based RL methods is insufficient. The baselines mainly include classical reinforcement learning approaches. Since LIFT is based on latent world models and imagined rollouts, stronger comparisons with recent model-based RL methods would be necessary to better establish the advantage of the proposed approach.
-
The performance improvements are relatively modest. Although LIFT improves some metrics, the gains over strong baselines are not consistently large, and in some settings baseline methods achieve comparable or better exploration performance. More evidence is needed to demonstrate that future feasibility modeling provides substantial practical benefits.
Questions for authors
-
How does LIFT extend beyond the predefined joint plan space generated by the deterministic planner? The current approach appears to depend on enumerating feasible plans beforehand. Could the authors clarify whether future team feasibility can be applied when the action space is continuous or when new skills emerge?
-
How sensitive are the results to the quality of the planner and skill library? Since the planner determines the available candidate actions, could the authors provide analysis of how performance changes with weaker or incomplete plan generation?
-
How computationally expensive is the imagined rollout procedure? Could the authors report the inference-time overhead introduced by feasibility prediction and multi-step planning compared with standard exploration policies?
Thank you. We added direct interventions, prediction diagnostics and complete decision timing. Revisions are blue; full protocols/results are in Appendix F and Supplement Sec. 11.
1. Planner dependence. The constructor handles current admissibility, precedence, assignment and ordering. LIFT learns forecasts and a selection policy within this interface; it cannot choose omitted actions. We added all-valid, current-mask persistence and training-only slot-frequency baselines. Original Full exceeds these at validation h=5/10 under the conditioning limits below, supporting predictive information, not planner independence.
2. Discrete versus continuous formulation. Sec. 3.1 now separates history-conditioned admissibility from its bounded discrete implementation. A continuous extension would need an action-conditioned feasibility model, a sampler/optimizer and grounded execution checks. Our task-level PICK/PLACE does not demonstrate continuous manipulation; this remains an extension.
3. Simulation/real robots. Quantitative evidence is from simulation. We corrected the comparison table that could imply a demonstrated real-robot advantage over PPO. Controlled misses, delays and noise do not establish physical robustness; hardware/interface descriptions are compatibility records, not comparative trials.
4. Modern model-based baselines. We strengthened within-architecture evidence with three-seed component removals and a separate prediction study: 12 models, 4,000 updates each, identical fixed replay (10,627 loaded items), matching initialization and sampled batches within each seed, final checkpoints only. Arms are original Full, class-balanced Full, matched balanced mask-off, and no validity loss/mask. Balanced variants are separately labeled. An unsupervised head is only a diagnostic control, so we include strong occupancy baselines. These fixed-replay studies do not replace the still-missing matched online DreamerV3/TD-MPC2 comparison.
5. Modest gains. Full/mask-off/loss-and-mask-off coverage is 95.90/94.87/94.69%, and minimum-floor coverage is 95.21/92.87/94.65%. Completion remains similar (128/127/129 of 150); return favors removing both components, and task/seed effects are mixed. These are three seeds on 50 reused starts. Separately, original Full's validation invalid AP at h=5/10 is 0.499/0.723 versus slot-prior 0.268/0.390; Brier errors are 0.103/0.173 versus 0.117/0.212. Primary h=1 AP is below the prior (0.238 vs 0.255), and h=5/10 use only 27/15 shared windows. Later descriptors are predicted, but recorded future action slots remain inputs; targets are future menu-slot validity. Thus the study supports bounded multi-step prediction, not semantic-action counterfactual validity or proven closed-loop necessity. All arms/modes and horizons are reported.
6. Q1: New skills and learning. New skills need execution/admission contracts, representations and transition data; we do not demonstrate autonomous skill discovery. Existing acquisition/retention evidence is narrower: the same 55 elevator SELECT states improve from 0/55 to 55/55, and two fixed rollouts change from timeout to success, retaining behavior over 32 further updates with teacher policy supervision effectively removed. This concerns enabling behavior within existing skills, not online test-time learning or open-ended self-evolution.
7. Q2: Incomplete planners. On 50 matched starts, Full completes 44/50 with the original menu, 43/50 with 20% random deletion, 44/50 with 40% deletion, and 41/50 after slot shuffling. The loss/mask-off branch gives 44/47/45/43 of 50. Deletion protects status quo and rounds down; shuffling retains the candidate set. Controls, contexts and masks move together, with selected-slot mapping checked. K32/16/8 prefix restrictions each yield 45/50 for Full. These quantify dependence, not superior robustness, invariance, or recovery of essential missing actions; they use earlier short-continuation checkpoints on five reused assets.
8. Q3: Computation. H=15 imagination occurs during actor training, not each evaluation decision. Three serial 50-episode repeats on TITAN RTX give complete decision medians 1163/1172/1174 ms and P95 about 2255 ms; policy medians are 8.79/8.80/8.81 ms, planner-core about 724-727 ms. Timing runs from observation acquisition after the prior skill to next-plan receipt, including processing, proposals, encoding, IPC and synchronization, excluding skill execution/reset/startup/training. Each episode's first decision and ten global warm-ups are excluded. Component quantiles are not additive. Learner-step medians are 0.314/0.300/0.306 s for Full/mask-off/loss-and-mask-off, with overlapping ranges, not isolated head cost. No paired speedup over Priority is claimed.