- Rating
- 2
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces PAVE, a world-action policy-learning framework intended to separate two roles of mixed-quality trajectories: transitions from both successful and unsuccessful rollouts are retained as predictive supervision, whereas only high-quality behavior is used as a preferred imitation target. The actor uses local and trajectory-relative multi-horizon predictive objectives. An offline critic estimates trajectory quality, and two-fold cross-fitting is used to generate quality labels without scoring an episode with a critic trained on that episode. The resulting binary quality label is provided to the policy as a conditioning signal during training, while only the positive condition is used at deployment. The paper evaluates the method on LIBERO, LIBERO-Plus, RoboTwin, and two physical S1 tasks. Its strongest experimental evidence is a controlled routing experiment showing that, at a fixed predictive-pair budget, using all valid trajectories for prediction while restricting behavioral preference to high-quality trajectories outperforms success-only predictive training. The paper also includes extensive ablations, implementation details, and a discussion of future-information boundaries.
Critical strengths
- The paper isolates a meaningful empirical principle about mixed-quality trajectories. The strongest result is the fixed-predictive-pair-budget routing control: retaining all valid trajectories for predictive supervision while restricting preference/imitation to high-quality behavior substantially outperforms success-only predictive training. Because the predictive-pair budget is controlled, this experiment rules out the trivial explanation that the method merely benefits from more data. This is a useful and potentially reusable finding for robot learning from heterogeneous rollout pools.
- The treatment of future-information boundaries is unusually careful. The manuscript provides explicit forward-graph arguments and a parameter/loss accounting for why future observations do not influence deployed actions. The cross-fitting implementation details—for example, preserving episode IDs and using a consistent excluded-fold critic for both endpoints of a score—also demonstrate strong engineering care. These aspects improve the paper’s reproducibility and distinguish it from many empirical robotics submissions.
Critical weaknesses
- The reported experimental record contains unresolved inconsistencies, which undermines confidence in the quantitative claims. Multiple tables appear to report different standard deviations for the same nominal configurations despite identical means. For example, the reported uncertainty for “Local + QC” differs across tables for both LIBERO-Plus and RoboTwin Random. The base actor described as the same configuration in the captions of different ablations likewise has inconsistent standard deviations. More seriously, “unlabeled BC” appears with different means in two tables that both describe it as sharing the same base actor and rollout pool. These may be explainable by different seed subsets, reruns, or aggregation procedures, but the manuscript does not identify the runs or explain the reporting protocol. Since uncertainty estimates are central to nearly every empirical comparison, the authors need to reconcile these discrepancies with seed-level results, run identifiers, aggregation rules, and corrected tables. Until then, it is difficult to assess the reliability of the reported improvements.
- The current ablations do not identify the claimed mechanisms or provide a credible hyperparameter-selection protocol. The paper reports many one-sided design and hyperparameter comparisons in which the selected default is consistently the best-performing option, but it provides no training-side validation curves or independently held-out model-selection results. A statement that the grid was fixed before evaluating test benchmarks is not sufficient evidence of this protocol. The issue is particularly acute for cross-fitting: the fold count, total critic-update budget, and downstream policy result change together. In the reported study, two-fold cross-fitting outperforms five-fold cross-fitting in downstream success, while five-fold appears better according to held-out critic metrics. Thus, the study does not isolate a fold-exclusion effect from optimization-budget or other effects, and the criterion used to choose the default is unclear. Relatedly, the “action-conditioned value learning” framing is not established by the current evidence. The reported state-value alternative outperforms the action-conditioned (Q) alternative, while the action-conditioned predictive head is inactive during annotation. The residual benefit attributed to the predictive head is small and reported without uncertainty. Likewise, the manuscript itself acknowledges that the future-query tokens are a static bank whose sample-specific future semantics are unverified, yet it provides no probe or prediction-accuracy analysis. These components should either receive direct validation or be described more modestly.
Questions for authors
- Result provenance and reporting consistency. Please provide, for every repeated configuration whose uncertainty differs across tables, the exact seed set, run IDs, code commit, evaluation episode set, and aggregation procedure. In particular, please reconcile the repeated “Local + QC” entries, the base-actor entries in the relevant ablations, and the inconsistent “unlabeled BC” means. Were these independent reruns, subsets of runs, or reporting errors? Please provide a corrected, seed-level result table.
- Model selection and cross-fitting attribution. For the default settings used in the main comparisons, please provide the training-side validation protocol and curves used to choose the fold count, quality threshold, condition-dropout probability, number of horizons, and temporal schedule. For cross-fitting specifically, can the authors run or provide a fixed-total-update-budget comparison over , jointly reporting critic quality and downstream policy success? How do label agreement measures compare between in-sample, two-fold, and five-fold labels rather than using two-fold as its own reference?
- Evidence for the proposed predictive mechanisms. Please report held-out prediction accuracy for the actor’s predictive objective, compared with at least a constant target, random target, and current-frame-embedding baseline. Can the authors include a shuffled/random predictive-target control with matched loss scale and optimization budget? This would distinguish informative future prediction from generic auxiliary-loss regularization. Please also clarify how and the local/multi-horizon loss weights were selected, and provide sensitivity results or state explicitly that they were fixed without tuning.
- Physical-rollout provenance. Who collected the cumulative physical rollouts shared by PAVE and the control: PAVE, the control, or a separate behavior policy? If the pool was collected once and shared, please revise “physical evolution” language to distinguish offline post-training on a fixed rollout pool from an online policy-evolution loop. Please also report hardware/control details, operator interventions, safety procedures, and seed-level physical outcomes. With three seeds, the organizing-task difference appears insufficiently resolved to support a strong aggregate claim without an interval or paired analysis.
- Positioning. Please discuss the relationship between binary advantage/quality conditioning and established offline-RL approaches such as advantage-weighted behavior cloning, IQL/AWR-style policy extraction, filtered BC, and return/percentile conditioning. Please also discuss why fixed trajectory-relative anchor positions outperform random relative offsets here, despite prior temporal predictive-learning results often favoring random offsets.