acceptodds
Under review as a conference paper at ICLR 2027

STORC: Spatio-Temporal Output-Space Residual Correction for Robot Video Prediction

Abstract

Robot video prediction models often lose accuracy in new scenes and tasks. As target frames are gradually revealed, they expose prediction errors and provide feedback for correcting future predictions, yet reliably and efficiently estimating future errors from limited history remains challenging. To address this challenge, we propose STORC, a spatio-temporal output-space residual correction framework requiring only one forward pass of the frozen backbone per prediction clip and no online gradient updates. STORC analytically estimates local relationships between prediction changes and residual changes at a regional scale, then extrapolates future residuals along the cached prediction trajectory, anchored at the latest observed residual. To control extrapolation risk, Causal Predictive Shrinkage (CPS) evaluates propagation against residual persistence on the latest completed transition held out from fitting and adaptively shrinks the propagation increment. To preserve fine spatial detail, STORC fuses regional and native-resolution corrections using the same shrunk propagation coefficients. Under a unified causal feedback protocol, STORC achieves an average MSE reduction of 34.8% relative to the strongest evaluated adaptation baseline in each of 45 backbone-split combinations, spanning five backbones and nine test splits on BridgeData, RoboNet, and BAIR. Core latency measurements show a geometric mean speedup of 3.91 over the evaluated adaptation baselines. Small-scale offline evaluation on self-collected laboratory videos demonstrates transfer without additional training or tuning.

Primary area:
learning on time series and dynamical systems
Keywords:
Robot Video Prediction, Output-Space Adaptation, Distribution Shift, Causal Feedback, Spatio-Temporal Prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Ew2aOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes STORC, a residual correction framework for robot video prediction that uses newly observed ground-truth frames to improve cached future predictions without additional backbone inference or online gradient updates. The method consists of three main components: (1) Local Analytic Residual Modeling (LARM), which estimates local relationships between prediction changes and residual changes through spatially aggregated ridge regression; (2) Causal Predictive Shrinkage (CPS), which evaluates the reliability of residual propagation against residual persistence using a held-out historical transition and adaptively adjusts the propagation strength; and (3) a dual-resolution correction mechanism that combines regional and native-resolution residual estimates to balance estimation stability and spatial detail preservation. The authors evaluate STORC across five prediction backbones and nine test splits from BridgeData, RoboNet, and BAIR. The results show improvements in prediction accuracy and computational efficiency over the evaluated test-time adaptation baselines, with an average MSE reduction of 34.8% relative to the strongest evaluated adaptation baseline in each setting.

Critical strengths

Strength 1: Well-motivated and efficient feedback-based correction The paper addresses a practical problem in robot video prediction: efficiently improving cached forecasts using newly observed feedback. The proposed framework is conceptually simple and well-motivated, combining local residual modeling, historical reliability assessment, and dual-resolution correction in a coherent manner. Importantly, STORC requires neither additional backbone inference nor online gradient updates, making it a computationally efficient approach to prediction correction. This simplicity and efficiency are valuable for applications where repeated model inference is computationally expensive.

Strength 2: Extensive empirical evaluation and consistent performance improvements The paper provides an extensive evaluation across five prediction backbones and nine test splits from three robot video datasets, covering both standard and distribution-shifted conditions. STORC consistently outperforms the evaluated parameter-adaptation baselines in prediction accuracy while maintaining low computational overhead. Furthermore, the component ablations and mechanism analyses provide evidence that LARM and CPS offer additional improvements beyond basic residual correction. These results support the effectiveness and practical applicability of the overall framework across different prediction architectures and evaluation conditions.

Critical weaknesses

Weaknesses 1: Insufficient Comparisons with Existing Residual Correction Methods The main concern is the lack of comparisons with representative existing residual correction methods. The paper identifies limitations of residual persistence and temporal extrapolation, and proposes LARM and CPS to address them. However, the main experiments in Table 1 primarily compare STORC with test-time adaptation methods rather than closely related residual correction approaches. Consequently, the reported 34.8% MSE reduction demonstrates the effectiveness of the overall framework against the evaluated adaptation baselines but does not establish its relative advantage within the residual correction literature. Furthermore, Table 3 shows that the residual-persistence-based variant with regional fusion already achieves most of the overall MSE improvement, while LARM and CPS provide relatively modest additional gains (1.3%–5.8%) across the eight evaluated settings. Although these results support the effectiveness of the proposed components, comparisons with representative existing residual correction methods under matched evaluation conditions are needed to better assess the significance of their incremental contributions.

Weaknesses 2: Contribution of Prediction-Driven Residual Propagation A key motivation for LARM is that cached future predictions provide useful information for estimating residual evolution. However, the ablation study in Table 3 primarily evaluates LARM against regional residual persistence, which does not model temporal residual changes. As a result, it remains unclear whether the observed improvements specifically arise from exploiting future prediction trajectories or from temporal residual extrapolation more generally. A comparison with a simple residual-only temporal extrapolation method, under the same spatial correction and feedback protocol, would provide more direct evidence for the distinctive contribution of prediction-driven residual modeling.

Weaknesses 3: Limited Justification for the CPS Shrinkage Strategy The specific shrinkage formulation used in CPS lacks sufficient justification. While the proposed strategy improves performance under distribution shifts, Table 3 shows slight MSE degradation on the standard splits, indicating that its benefits are not consistent across all evaluation conditions. Although the paper provides evidence that historical confidence is associated with future propagation benefits, it remains unclear whether the proposed shrinkage formulation is preferable to simpler propagation control strategies. Comparisons with alternatives such as fixed shrinkage or binary gating based on the same historical validation would help establish the necessity and effectiveness of the specific CPS design.

Questions for authors

Question 1: Comparison with Existing Residual Correction Methods (Primary Concern) The paper aims to address limitations of existing residual correction approaches, yet the main experiments primarily compare STORC with test-time adaptation methods. Could the authors provide comparisons with representative existing residual correction methods under the same causal feedback protocol, or explain why such comparisons are not feasible? In particular, how does STORC compare with these methods in terms of prediction accuracy and computational efficiency? This is my primary concern and the main factor influencing my current rating of 2 (Weak Rejection). If the authors can convincingly address this concern and provide sufficient evidence to establish the incremental contribution of STORC over closely related methods, I would consider increasing my rating to 3 (Weak Acceptance).

Question 2: Contribution of Prediction-Driven Residual Propagation Could the authors compare LARM with a simple temporal residual extrapolation baseline that relies only on historical residual changes, without using future prediction trajectories? A comparison under identical spatial correction and feedback settings would help clarify whether the observed improvements specifically result from exploiting future prediction information rather than temporal extrapolation itself.

Question 3: Justification of the CPS Shrinkage Strategy What motivates the specific shrinkage formulation adopted in CPS? Could the authors compare it with simpler alternatives, such as fixed shrinkage or binary gating based on historical validation? Given the mixed effects of CPS across standard and distribution-shifted settings in Table 3, these comparisons would help clarify whether the proposed adaptive shrinkage strategy provides meaningful advantages over simpler propagation control mechanisms.

2 comments

Thank you for recognizing the motivation and efficiency of our framework and for suggesting targeted comparisons. We have added ORCA, residual-only extrapolation, and binary-gating experiments to address your questions.

Q1. Comparison with Existing Residual Correction Methods

We now include ORCA (NeurIPS 2026) [2] in main-text Table 1, alongside COSA (ICLR 2026) [1]. Both perform output-space residual correction: COSA uses a gated linear adapter, and ORCA learns errors conditioned on historical inputs and base forecasts. STORC uses the latest observed residual as an explicit anchor, analytically propagates residual changes along cached predictions, and shrinks only the propagation increment, without online gradient updates. Native and Regional Persistence are internal ablations of this framework; their strong performance establishes the value of basic correction, while LARM/CPS comparisons isolate additional gains.

With causal feedback at p=2,4,6,8, STORC has lower MSE and LPIPS than ORCA in all 45 backbone–split combinations. Average relative reductions are approximately 34.00% MSE and 35.92% LPIPS, computed per setting from Table 1’s displayed values and then averaged equally. Appendix A.2, Table 6 documents the official-core implementation, online updates, replay, state resets, and global configuration selection. ORCA requires online adapter updates, replay, and routing, whereas STORC uses analytic correction without online gradient updates.

Q2. Prediction-Driven versus Residual-Only Extrapolation

We added Res-Extrap and Res+CPS to main-text Table 5, with definitions in Appendix A.11. Res-Extrap anchors at the latest residual and linearly extrapolates the mean historical residual increment, without future prediction trajectories. It retains the same regional/native fusion, clipping, frozen TAU, and 256 clips per RoboNet split. All methods use only the prefix through p=4 and are evaluated over h=5:10 without later feedback. Res+CPS computes its own held-out propagation error using the same validation split, rather than borrowing LARM’s confidence.

LARM reduces suffix-average MSE by 4.71–13.04% versus Res-Extrap across the four splits. With CPS applied to both, STORC improves over Res+CPS by 6.01–14.50%. These comparisons support prediction-driven propagation over the specified residual-only extrapolator; all individual results appear in Table 5.

RoboNet splitRes-ExtrapLARMRes+CPSSTORC
Standard65.3758.3862.4558.69
Dynamics132.45115.17126.26108.49
Scene113.50108.16108.4496.99
Joint342.52306.92327.58280.07

Mean MSE over h=5–10; compare Res-Extrap with LARM, and Res+CPS with STORC.

Q3. Motivation and Comparison of CPS Shrinkage

CPS compares propagation with residual persistence on the same held-out transition (Eq. (4)) and converts their relative error reduction into a continuous shrinkage weight (Eq. (5), Section 3.3). A small historical advantage therefore produces a small correction rather than full propagation, while the observed residual anchor is preserved. We added Binary Gating, which uses identical LARM errors and the same 16×16 grid, but enables propagation only where E_prop < E_hold. Fixed coefficients 0.3/0.5/0.7 and global adaptive shrinkage are also included in Appendix A.11, Table 16; the key comparison is in main-text Table 5.

On shifted RoboNet splits, CPS lowers mean MSE by 6.67–7.23% versus the best tested fixed coefficient and 3.08–4.63% versus global shrinkage. It improves over binary gating by 0.39–4.58% across all four splits. Unshrunk LARM remains best on Standard, and binary gating is slightly better at steps 5–6 there. The results support continuous validation-based shrinkage particularly under shifts, with the Standard trade-off retained in the reported results.

RoboNet splitLARM (no shrinkage)Binary GatingCPS
Standard58.3858.9258.69
Dynamics115.17111.22108.49
Scene108.16101.6496.99
Joint306.92291.40280.07

Same fixed-prefix protocol; fixed-coefficient and global-shrinkage results are in Table 16.

References

[1] Im and Kwon. COSA: Context-aware Output-Space Adapter for Test-Time Adaptation in Time Series Forecasting. ICLR, 2026.

[2] Dai et al. Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Models. NeurIPS, 2026.

Reviewer_eKNdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes STORC, a training-free output-space residual correction framework for robot video prediction. Given a cached forecast from a frozen backbone and ground-truth frames revealed during the clip, STORC fits a locally averaged scalar ridge regression between prediction increments and residual increments on a pooled grid (LARM), shrinks the propagated increment using a held-out comparison with residual persistence on the latest transition, and applies the shrunk coefficients at both pooled and native resolutions. No additional backbone inference or online gradient updates are required.

Critical strengths
  • The proposed method is very efficient, as it runs the frozen backbone only once per clip and corrects the cached forecasts in closed form. The experiments show only 3.3–3.5 ms/clip of additional latency over Source.

  • They show the generality of their method, which uses only the cached RGB outputs and does not require any access to the model internals. With this, their approach can be applied to various models without any modification.

  • CPS is designed to fall back to residual persistence by continuously adjusting the propagation strength through an out-of-sample check. On the shifted RoboNet splits, CPS is better than fixed coefficients, global shrinkage, and binary gating.

Critical weaknesses

Major Concerns:

  • The core components are standard statistical tools. The ablations show that the proposed modeling adds only a modest increment over simple persistence:
  • The five adaptation baselines are time-series TTA methods ported through video-compatible calibration interfaces. Their improvements over Source are mostly marginal. -The main protocol reveals ground-truth frames every two steps within the same clip. This paper does not show that the gains translate into better action selection, planning, or closed-loop control.
  • CPS slightly worsens MSE and SSIM on both Standard splits. The paper attributes this to over-shrinking but does not compare against simpler alternatives. The pooling-grid study shows the default 16×16 is not optimal: 32×32 gives better MSE on task and scene shift and the best LPIPS in all eight combinations. Hyperparameter sensitivity therefore matters, and robustness on RoboNet/BAIR is not examined.

Minor Concerns:

  • Different camera views of the same physical trajectory may appear in both training and testing. This is disclosed, but it complicates interpretation of in-domain results.
  • Accuracy tables report point estimates without paired clip-level confidence intervals or variation across backbone training seeds. The timing repetitions measure timing variability only.
  • Reforecast has lower LPIPS on RoboNet Standard for all five backbones and on several other RoboNet splits, so the "higher MSE but lower LPIPS" trade-off stated in Section 4.4 holds mainly on BridgeData.
Questions for authors
  • Can the authors add Native/Regional Persistence as rows in Table 1 for all 45 combinations, and report how much LARM + CPS improves over the best persistence variant on RoboNet and BAIR?
  • Do the adaptation baselines receive the same within-clip ground-truth feedback as STORC, and how are their learning rates and update budgets tuned? Why is Reforecast excluded from the headline comparison given that it is more accurate in MSE and cheaper than two of the adaptation baselines?
  • Can the authors report performance under delayed or irregular feedback and longer horizons on the shifted splits, compare CPS against simpler shrinkage rules, and provide paired clip-level confidence intervals for the main improvements?
5 comments

Thank you for your detailed feedback on the contribution of residual correction, the fairness of baseline comparisons, and robustness. We clarify the framework and present the additional evidence below.

Q1. Contribution of the Framework and Its Components

Our first contribution is causal, gradient-free correction of cached video forecasts. Native and Regional Persistence are internal ablations of this framework; LARM and CPS add prediction-driven propagation and reliability control. COSA [1] learns gated corrections from forecasts and recent observation statistics, while ORCA [2] learns contextual residuals with error-based routing. STORC explicitly preserves the latest revealed residual as an anchor and analytically propagates its changes along cached predictions. Persistence already corrects forecasts at p=2; LARM and CPS become active from p=4. Thus, gains over persistence measure the added mechanisms, while comparisons with external methods evaluate the complete framework.

Appendix A.10, Table 15 places both persistence variants alongside STORC across all 45 combinations: STORC improves MSE and LPIPS throughout, with MSE reductions of 1.67–5.47% over Regional on RoboNet/BAIR, calculated from displayed values. Main-text Table 4 reports the ten TAU/PFGNet component comparisons with 10,000 paired clip-level bootstrap resamples per 256-clip test set; all STORC − Regional intervals lie below zero. Figure 6 isolates performance after fixed p=4 feedback, complementing full-horizon averages that include four steps before propagation activates. We also added ORCA to Table 1, where STORC improves both metrics in all 45 settings.

Q2. Feedback Fairness, Configuration, and Reforecast

We use the baselines’ official implementations and configurations, with video-compatible interfaces and frozen backbones. Appendix A.2, Table 6 documents architectures, learning rates, update budgets, and state retention. TAFAS, PETSA, COSA, δ-Adapter, and ORCA update at p=2,4,6,8; ST-TTC retains its completed-observation FIFO mechanism. All follow causal observation availability, with method-specific updates. For ORCA, one global configuration was selected from four learning-rate/L1 candidates on three test-trajectory-disjoint development panels before evaluation; test states are reset. The δ-Adapter validation comparison is in Appendix A.15, Table 20.

Table 1 compares STORC with the adaptation baselines, including the added ORCA results. Appendix A.14, Table 19 reports Reforecast separately: it wins MSE in 39/45 settings, while STORC wins LPIPS in 35/45. Reforecast reruns the backbone and generally requires an observation window matching its input length; forming a fresh window requires enough revealed frames. Under the evaluated 2→10 protocol, rollouts at p=2,4,6,8, plus the initial forecast, give five backbone calls versus one for STORC. Reforecast is faster than TAFAS and PETSA, while STORC is faster than all three. We clarify that the MSE/LPIPS trade-off is split-dependent.

Q3. Shrinkage, Feedback, and Robustness

Main-text Table 5 and Appendix A.11, Table 16 compare CPS with fixed c=0.3, 0.5, 0.7, global adaptive shrinkage, binary gating using the same historical validation, and unshrunk LARM. Using TAU on the same 256 clips per RoboNet split, with fixed p=4 feedback and evaluation over h=5:10, CPS lowers mean MSE by 6.67–7.23% versus the best tested fixed coefficient and 3.08–4.63% versus global shrinkage on shifted splits. Its reduction versus binary gating is 0.39–4.58% across all four splits; unshrunk LARM remains best on Standard. Appendix A.12, Table 17 extends grid sensitivity to RoboNet/BAIR: 16×16 minimizes MSE in 9/10 settings, while the preferred LPIPS grid varies.

RoboNet splitBest tested fixed cGlobalBinaryCPS
Standard59.0559.2858.9258.69
Dynamics116.94111.94111.22108.49
Scene103.92100.78101.6496.99
Joint301.67293.68291.40280.07

Mean MSE over h=5–10. The best tested fixed coefficient is selected retrospectively within each split from {0.3, 0.5, 0.7}. Unshrunk LARM remains better on Standard (58.38); complete results are in Table 16.

The shifted-split fixed-prefix results (Figure 6; Appendix A.6) show sustained correction through h=10 without further feedback; Appendix A.6 also varies feedback intervals d=1,2,4,8. These tests examine stale and regularly spaced feedback within ten frames; delayed/irregular arrivals, partial observations, longer horizons, and downstream control remain untested. Appendix A.13, Table 18 quantifies 94.41% cross-camera trajectory overlap and shows retained gains on 124 clips from 92 trajectories absent from training and validation. Table 4’s intervals quantify clip-level uncertainty with fixed checkpoints, not variation across backbone training seeds.

References

[1] Im and Kwon. COSA: Context-aware Output-Space Adapter for Test-Time Adaptation in Time Series Forecasting. ICLR, 2026.

[2] Dai et al. Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Models. NeurIPS, 2026.

Thank you to the authors for the detailed rebuttal and the additional persistence, shrinkage, and unseen-trajectory analyses. While they clarify some of my concerns, the novelty of the proposed method remains limited. I will therefore maintain my original score.

Reviewer_ZwbHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes STORC, a spatio-temporal output-space residual correction framework that refines cached robot video predictions using causally revealed observations. Local Analytic Residual Modeling estimates regional relationships between prediction changes and residual changes, while Causal Predictive Shrinkage uses a held-out historical transition to regulate residual propagation. Regional and native-resolution corrections share the resulting coefficients, requiring one frozen backbone forward pass per prediction clip and no online gradient updates. Evaluation across five backbones and nine test splits from BridgeData, RoboNet, and BAIR, together with component ablations, efficiency measurements, and offline evaluation on ten laboratory clips, examines prediction quality, computational cost, and transfer across distribution conditions.

Critical strengths
  • STORC addresses a useful computational constraint: incorporating newly revealed observations without additional backbone inference or online gradient updates. The local analytic estimation and shared propagation coefficients provide a lightweight correction procedure. Table 4 reports a modest increase in core latency over Source, supporting the practical value of the approach when repeated backbone inference is expensive.

  • The experiments cover three established robot video datasets, five backbones, and multiple scene, task, and dynamics shifts. Table 1 reports lower MSE and LPIPS than the five evaluated adaptation baselines across all 45 combinations. This breadth provides encouraging evidence that the complete correction framework is useful across different prediction architectures and evaluation conditions.

  • The causal execution protocol is clearly specified, including the use of only revealed observations and the restriction against revising previously evaluated outputs. Native Persistence, Reforecast, component ablations, and fixed-prefix analyses provide useful comparisons for understanding correction behavior. The paper also acknowledges the accuracy and efficiency trade-off with Reforecast and the limitations of the laboratory evaluation.

Critical weaknesses
  • The main benchmark comparisons with TAFAS, PETSA, ST-TTC, COSA, and δ-Adapter are insufficiently documented to assess implementation fidelity and competitiveness. Appendix A.2 states that these methods use video-compatible calibration interfaces but provides limited detail about adapter architectures, optimization settings, update counts, and state retention across clips. Although Appendix A.8 specifies settings for δ-Adapter in the laboratory evaluation, equivalent details are not provided for the main benchmark experiments. The common evaluation pipeline described in Appendix A.4 is helpful, but it does not resolve these implementation questions.

  • The additional benefits of LARM and CPS are less comprehensively established than the improvement of the complete correction framework. Calculated from the values in Table 3, adding LARM and CPS reduces MSE by approximately 1.3% to 5.8% relative to the “+ Regional” variant, while CPS slightly worsens MSE in both Standard settings. Figure 4 provides additional mechanism analysis, but corresponding component comparisons on RoboNet and BAIR and uncertainty estimates for prediction accuracy are not reported. These omissions limit assessment of the consistency and significance of the incremental contributions.

  • Appendix A.1 states that the BridgeData Standard split is partitioned at the camera-stream level and that different camera views of the same physical trajectory may occur in training and testing. The manuscript does not establish whether this overlap actually occurs or quantify its extent. This leaves the independence of the Standard evaluation unclear and limits its interpretation as generalization to previously unseen trajectories. The concern applies specifically to this split and does not by itself invalidate the relative comparisons or the shifted evaluations.

Questions for authors
  • Clarification: Please specify the video-compatible implementation of each adaptation baseline, including adapter architecture, learning rate, updates per reveal, parameter selection, and state retention across clips. How were these settings selected and checked for the main benchmark experiments, particularly for the additive output-correction variant of δ-Adapter?

  • Additional evidence: Could the authors report Native Persistence and the “+ Regional” variant on RoboNet and BAIR under the same causal feedback protocol? Quantifying uncertainty in the accuracy differences would also help assess whether the incremental benefits of LARM and CPS are consistent across datasets.

  • Clarification and additional evidence: Does the BridgeData Standard partition actually place different camera views of the same physical trajectory in training and testing? If so, what proportion of the test set is affected, and do the main improvements persist when evaluation is restricted to trajectories absent from training?

1 comment

Thank you for your careful review and constructive suggestions on baseline implementation, component evaluation, and trajectory overlap. We address each point below.

Q1. Baseline Implementation and Configuration

We use the official implementations and configurations of the five adaptation baselines with video-compatible interfaces and frozen backbones. Appendix A.2, Table 6 now documents architectures, initialization, trainable parameters, learning rates, update schedules, and cross-clip state retention, checked against our code and archived runs. For δ-Adapter, the main benchmark uses identity initialization, an additive correction bound of 0.1, and learning rate 10⁻⁴, with one update per revealed prefix and one completed-target update affecting subsequent clips; states reset between splits.

The original δ-Adapter paper [1] defines both additive and multiplicative correction. We select the additive form for direct residual correction, supported by lower MSE on 320 PredBench BridgeData validation clips [2] per backbone: 510.48 vs. 514.65 for TAU and 558.11 vs. 561.50 for PFGNet (Appendix A.15, Table 20). Both variants used five epochs of calibration followed by causal online updates; the main benchmark instead uses identity initialization and fixed online settings across all 45 combinations.

Q2. Persistence Baselines and Incremental Component Gains

Main-text Table 4 extends the ablation to TAU/PFGNet on RoboNet and BAIR, using the same 256 clips per split and 10,000 clip-level paired bootstrap resamples. STORC improves over Regional in all ten settings, with every 95% interval for STORC − Regional below zero. LARM lowers mean MSE throughout, although its intervals cross zero on both R-Scene settings; CPS improves shifted RoboNet and BAIR but slightly worsens both R-Std. settings.

BackboneRoboNet/BAIR settingsMSE decrease: STORC vs. Regional95% CIs entirely below zero
TAU50.94–10.965/5
PFGNet50.71–8.565/5

Absolute MSE decreases; complete per-split means and paired intervals are in Table 4.

Figure 6 complements these full-horizon averages by isolating TAU's predictions at h=5–10 after a fixed p=4 reveal, when LARM and CPS become active, without subsequent feedback. On the three shifted RoboNet splits, CPS improves over LARM at every subsequent step; on Standard, unshrunk LARM has the lower suffix-average MSE. This makes the post-activation contribution visible, whereas full-horizon averages also include the first four steps before propagation activates. Complete Native/Regional/STORC comparisons across all 45 combinations are in Appendix A.10, Table 15.

Q3. Trajectory Overlap in BridgeData

We follow the official PredBench splits [2]. Our audit confirms that 3,748/3,970 Standard test clips (94.41%) share physical trajectories with training through different camera views. On 124 clips from 92 trajectories absent from both training and validation, STORC retains MSE reductions of 23.23–25.20% versus Source and 0.65–2.95% versus Regional across all five backbones (Appendix A.13, Table 18). Figure 5, Lab-collected additionally shows clearer robot-arm and object contours on self-collected videos with frozen BridgeData-trained TAU, without additional backbone training or STORC tuning.

References

[1] Liang et al. The Forecast After the Forecast: A Post-Processing Shift in Time Series. ICLR, 2026.

[2] Wang et al. PredBench: Benchmarking Spatio-Temporal Prediction across Diverse Disciplines. ECCV, 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.