- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents TimeValue, a language-conditioned robotic value model trained on large-scale heterogeneous data using timestamp-derived temporal-distance supervision. The approach combines absolute and relative temporal prediction with random temporal sampling, temporal-order shuffling, value-isolation attention, and auxiliary language supervision. The resulting model is evaluated through out-of-distribution trajectory ranking, real-world policy post-training, demonstration filtering, and inference-time action selection. Overall, I consider this a substantial empirical contribution with practical value, although its conceptual novelty is relatively limited. My main concern is whether the experiments sufficiently isolate the benefit of absolute time-to-go supervision over normalized progress. My current recommendation is weak rejection, but I am open to raising my score if these concerns are adequately addressed during the rebuttal phase.
Critical strengths
Broad evaluation of practical utility. The evaluation covers both dataset-level trajectory ranking and downstream robotic applications, including online and offline reinforcement learning, data filtering, and inference-time value guidance. This provides meaningful evidence that the learned model can be used in a zero-shot manner for various purposes.
A scalable and useful training recipe. Learning from timestamps and completion boundaries reduces dependence on explicit preference annotation. Applying this approach to a heterogeneous mixture of over 7,000 hours and approximately three million segments is a valuable contribution toward reusable robotic value models.
Critical weaknesses
The central comparison between absolute time-to-go and normalized progress needs stronger isolation. Given that replacing normalized progress estimation with temporal-distance supervision is the first item in the contribution list, I would expect a clean ablation supporting this argument in the main body. I acknowledge that the paper includes Robometer and other progress-based reward models as baselines. However, TimeValue could outperform these methods because of a larger, higher-quality training dataset. I am not sufficiently convinced of the advantage of cost-to-go supervision itself.
The conceptual novelty is limited. Time-to-go value learning has precedents in policy improvement. For example, learns values corresponding to negative remaining steps for successful episodes, with failure penalties and task-level normalization. TimeValue’s use of absolute seconds and its emphasis on a transferable value model are meaningful distinctions, but the underlying temporal-value interpretation is not new. The proposed shortcut-suppression mechanisms also appear necessary and highly intuitive when predicting time-to-go autoregressively. I do not think they provide much deeper insight.
Questions for authors
-
The time-to-go label decreases as a demonstration proceeds, even if the robot makes a mistake along the way. If the robot drops an object halfway through a trajectory, what teaches the model that this is a regression rather than progress?
-
What does TimeValue filter out in the Drawer task? The drop from 40% to 0% is quite substantial. The paper attributes this to limitations of the camera view, but could the filtering also be removing necessary actions that do not immediately make progress, such as repositioning before grasping?
-
Why is inference-time value guidance evaluated only on Bread and Steak? I would be interested in whether it also helps on Drawer and Bimanual, where judging the outcome of an action may require more precise spatial or coordination information.
-
I found an ablation in Table 9 where, under causal attention, normalized progress achieves 0.667 and absolute time with fixed bins achieves 0.675—a fairly small difference. However, these results are still confounded. Could you provide a more direct comparison in the main body to demonstrate the advantage of predicting time-to-go? Can you provide deeper insight on why TimeValue advance Robotmeter?
4 comments
Q1&Q2
Q1. How the model detects regression (e.g., dropping an object) when per-rollout labels always decrease
Intuition. Holding an object near its goal state typically precedes immediate placement, whereas dropping the object requires re-grasping and recovery. Because the pre-drop state is predominantly followed by direct completion while the post-drop state entails a prolonged recovery phase, the dropped-object state exhibits a strictly larger expected remaining time across the demonstration distribution. Because TimeValue estimates the conditional expectation of remaining time given the observed state across the dataset—rather than memorizing individual trajectory indices—it naturally assigns a higher time-to-go (lower value) upon task regression, even though recorded timestamps decrease monotonically within each individual rollout.
Formal analysis (Appendix G.3). Formally, TimeValue estimates , where denotes the empirical time-to-go. Appendix G.3 analyzes transitions from a pre-mistake state , where demonstrations follow either the direct expert branch or, with probability , the mistake branch , and follow the expert policy elsewhere (both branches ultimately succeed and share the continuation from the recovered state ). Let , , and denote the expected durations of the direct transition , the mistake transition , and the recovery segment , respectively, where true regression implies . The difference in expected remaining time between the mistake state and the pre-mistake state is:
which is strictly positive whenever the mistake probability satisfies:
Under this natural distributional condition, the population regression target increases sharply upon entering the mistake state and decreases as recovery progresses, proving that conditional expectation over monotonic rollout labels faithfully captures non-monotonic task progress. Figure 11 (Appendix G.3) illustrates this mechanism schematically, and Figure 5 (Appendix C.4) demonstrates it empirically on real trajectories.
Q2. What is filtered out in the Drawer task and whether necessary preparatory/corrective actions are removed
Our quantitative audit in Section 6 and Appendix D.5 identifies two primary factors driving the drop in Drawer success from to under offline data filtering:
1. Reduced state coverage. Advantage-based filtering prunes a substantial fraction of candidate training transitions, narrowing state-space coverage on precision-critical tasks. Under our filtering configuration (window , discount , retention criterion ), the filter retains of candidate action-chunk starts on Drawer and removes (). Across all four tasks, the frame-weighted removal rates over all source episodes (spanning both successful and unsuccessful trajectories) are:
| Task | Candidate training starts removed (%) |
|---|---|
| Bread Basket Placement | |
| Steak Serving with a Spatula | |
| Box-in-Drawer Placement | |
| Bimanual Box Transfer |
2. Removal of retry and recovery segments. Confirming your intuition, filtering disproportionately prunes retry attempts and corrective adjustments that teach the policy how to recover from initial misalignment. In our case study of a successful Drawer demonstration containing two missed grasp attempts prior to a successful grasp and lift (Appendix D.5, Figure 10), the filter removes of candidate starts () across the two unsuccessful attempts and re-approach phases, while retaining of starts () during the final successful grasp-and-lift execution. Discarding these corrective segments deprives the imitation policy of essential recovery behaviors, aligning with classic findings on corrective demonstrations.
Q3&Q4
Q3. Why inference-time value guidance is evaluated only on Bread and Steak
As noted in Section 6, the frozen Cosmos3-Edge-2B world-model proposal policy achieves unguided success on Bread and on Steak, but on Drawer and Bimanual despite joint multi-task fine-tuning across all four tasks. We therefore evaluated inference-time value guidance on the two tasks where the proposal distribution generates viable trajectories. With the proposal model held completely fixed, TimeValue guidance boosts mean success from to —outperforming Robometer guidance () and demonstrating that TimeValue accurately selects superior candidate rollouts at test time without policy updates.
Because inference-time value guidance operates by re-ranking four sampled action/video proposals, its performance is bounded by the support of the proposal distribution: a value model cannot synthesize a successful trajectory if all candidate proposals fail. Since the unguided base model yields success on Drawer and Bimanual, valid candidate rollouts rarely appear among the sampled proposals. Extending value guidance to these high-precision tasks alongside stronger world-model proposal backbones is a promising direction for future work.
Q4. Why Table 9 shows a small gap () for fixed bins, and deeper insight into why TimeValue advances over Robometer
Why fixed bins yield only a gain over normalized progress. Uniform (fixed) binning is poorly matched to the heavy-tailed distribution of physical remaining times, resulting in severe support under-utilization: the majority of the 256 uniform support centers fall in sparsely populated long-duration tail regions where few training clips reside, whereas the sub- horizon—where the vast majority of manipulation interactions occur—is compressed into roughly 20 coarse bins. This coarse discretization blunts gradient resolution and limits fine-grained temporal discrimination. By contrast, quantile binning allocates 217 of the 256 support centers () to the critical regime (Tables 7–8), unlocking the full benefit of physical temporal-distance supervision ( vs. ).
| Horizon | Quantile support centers (cumulative) | Share of all support centers |
|---|---|---|
| 19 | ||
| 75 | ||
| 119 | ||
| 165 | ||
| 217 | ||
| 241 | ||
| 254 |
Why TimeValue advances over Robometer. We present our controlled, unconfounded comparisons directly in Table 2 and Section 5 (with full breakdowns in Appendix C.2), demonstrating that TimeValue's superiority arises from three synergistic pillars:
1. Cross-dynamics formulation and heterogeneity handling. Episode-normalized progress requires well-defined trajectory start points and rescales every trajectory to by its total duration, discarding physical time-to-go and rendering identical remaining task segments inconsistent across rollouts of differing initial distances or idle pauses. In contrast, our cross-dynamics temporal-distance formulation (Section 3; Appendix G) predicts remaining cost-to-go in physical seconds conditioned on embodiment metadata, preserving suffix invariance and naturally accommodating diverse robot morphologies, control frequencies, and actuation speeds.
2. Scalable training recipe and shortcut-free architecture. Scaling a generalist robotic value foundation model requires a low-cost supervision signal that extracts dense value structure from heterogeneous corpora in the wild. Because physical temporal-distance targets require only frame timestamps and completion cutoffs—avoiding costly human preference annotations or paired failure trajectories—our objective scales seamlessly across real-robot demonstrations, simulation rollouts, and egocentric human videos. Architecturally, TimeValue combines random temporal sampling and order shuffling with training-time value isolation and inference-time causal attention, eliminating sequence-order shortcuts and enforcing visually grounded value estimation.
3. Consistent empirical superiority. Under strictly matched data, architecture, and training recipes, quantile-binned temporal distance elevates Mean Kendall's from to , translating directly into substantial downstream gains across offline IQL policy learning, online DSRL sample efficiency, and inference-time value guidance.
Dear Reviewer pFgc,
Thank you again for your careful reading. We have revised the manuscript in response to your comments. The following pointers may help you find the changes.
- W1 — Temporal distance versus normalized progress. The dark green row and discussion in Section 5, Table 2, bring the supervision-target comparison into the main text. Appendix C.2, Tables 9–12, gives the detailed ablations and the composition of the matched training subset.
- W2 — Contribution relative to prior time-to-go methods. The purple closing sentence of Temporal distance learning in Section 2 clarifies our contribution: learning in physical seconds across heterogeneous datasets and transferring a frozen value model across tasks and embodiments. The purple Monte Carlo supervision paragraph in Section 3 defines the target.
- Q1 — How decreasing timestamp labels can reveal a setback. The purple analysis in Appendix G.2–G.3, particularly Figure 11, illustrates how a mistake can increase expected remaining time even though the labels along each recorded trajectory decrease. Appendix C.4, Figure 5, shows a corresponding example on a real trajectory.
- Q2 — What filtering removes on Drawer. The purple discussion in Section 6 and Appendix D.5 examines missed grasps, re-approaches, and placement. Figure 10 shows which parts of two demonstrations were removed, and Table 15 gives the removal rate for each task.
- Q3 — Why value guidance was evaluated on Bread and Steak. The purple sentence in Section 6, under Inference-time value guidance, explains the task selection and reports the unguided policy’s 0% success on Drawer and Bimanual.
- Q4 — The small fixed-bin gain and the role of quantile binning. The dark green Value target and tokenization paragraph in Appendix C.2 discusses the three results in Table 9: 0.667 for normalized progress, 0.675 for fixed-bin temporal distance, and 0.704 for quantile-bin temporal distance. Appendix B.2, Tables 7–8, shows how the quantile bins are allocated.
When you have had a chance to read the revision, we would be grateful to hear whether these additions address your concerns.
Response to Reviewer PFgc Weaknesses
We thank Reviewer PFgc for the constructive review and for recognizing the breadth of our empirical evaluation. Your feedback helped us sharpen the formal problem definition and Monte Carlo supervision analysis in Section 3 and Appendix G, as well as expand the controlled comparisons in Table 2 and Appendix C.2 (Tables 9–12).
W1. Isolating the contribution of time-to-go supervision from data scale and architecture
We completely agree that cleanly isolating temporal-distance (cost-to-go) supervision from normalized progress is central to our primary contribution. In the revised manuscript, we have placed a direct, unconfounded ablation of the supervision objective into the main body (Section 5, Table 2), holding the training dataset, 8B architecture, and training recipe strictly fixed. Furthermore, because the processed RBM-1M release lacks the original frame-to-physical-time mappings required to construct targets in physical seconds (and original Robometer also relies on trajectory-preference pairs), we pair this main-body ablation with two additional controlled studies in Appendix C.2 (Tables 9–12) to rule out dataset size and quality as confounding factors:
1. Clean main-body ablation of cost-to-go supervision vs. normalized progress. Holding the training data, 8B backbone architecture, and training recipe strictly fixed, replacing normalized progress with quantile-binned temporal-distance (cost-to-go) supervision improves Mean Kendall's τa from 0.667 to 0.704:
2. Ruling out dataset size. When subsampling our corpus without replacement to match RBM-1M using shared-source embodiment quotas while preserving instruction coverage, TimeValue-4B (trained with its standard recipe; Appendix C.2, Tables 10–12) still outperforms original Robometer:
3. Ruling out dataset quality alone. Retraining the progress-only variant of Robometer within our framework using normalized-progress and success losses (excluding preference pairs), while retaining its original progress tokens, prediction heads, and image preprocessing, yields substantially lower ranking alignment than TimeValue on the same dataset:
Together, the clean main-body ablation (0.704 vs. 0.667 under identical data and architecture) and the two dataset-controlled studies (0.6633 vs. 0.655 at matched scale; 0.679/0.704 vs. 0.5593 on the identical data pipeline) provide direct, unconfounded evidence that cost-to-go supervision itself—rather than dataset size or quality alone—is a primary driver of TimeValue's advantage.
W2. Conceptual novelty relative to prior time-to-go value learning
Distinction in formulation. While π0.6∗ (RECAP) also adopts Monte Carlo time-to-go targets, its successful-episode targets are defined as negative remaining control steps normalized by each task's maximum episode length, paired with a terminal failure penalty (Sections IV-A and V-C). By contrast, TimeValue predicts expected remaining duration in physical seconds over a unified, dataset-level quantile support; our supervision is extracted directly from timestamped demonstrations with identified completion cutoffs, completely eliminating task-specific horizon normalization.
Distinction in role and zero-shot transferability. More fundamentally, our value model serves a distinct paradigm. RECAP fine-tunes a task-specific critic on in-domain deployment rollouts to compute advantages for policy iteration. In contrast, TimeValue is built as a generalist, zero-shot value foundation model: we establish a scalable training recipe over large-scale, heterogeneous multi-embodiment corpora so that a single frozen checkpoint transfers zero-shot to unseen tasks and embodiments without target-domain fine-tuning.
Necessity of shortcut suppression. Finally, while random temporal sampling, order shuffling, and value isolation are clean and modular by design, they are indispensable for scaling multi-frame value learning: removing temporal shuffling alone causes mean Kendall's τa to collapse from 0.704 to 0.159 (Table 2), demonstrating that naive autoregressive training fails to learn transferable value representations without explicit shortcut suppression.