- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes FlowLap, an execution-aware inference framework for flow-matching video-action world models. It overlaps speculative joint video-action generation with execution of a committed action prefix, then accepts the candidate suffix through timing, identity, and continuity checks or falls back to generation from refreshed real observations. Persistent context is updated only from real observations and executed controls. An optional residual adapter, trained with the backbone frozen, conditions the action branch on the estimated execution-time state. Across 50 RoboTwin tasks, FlowLap reduces mean transition wait from 896.6 to 39.1 ms on FlowWAM and from 839.9 to 232.4 ms on LingBot-VA. Calibration further reduces the latter to 71.8 ms while increasing success from 86.0% to 92.0%. These results demonstrate reduced boundary waiting rather than reduced model computation
Critical strengths
- Useful controls for the scheduling benefit.
The efficient synchronous baselines use the same reduced denoising budgets as FlowLap. The substantial additional wait reductions therefore support overlapping inference with execution rather than merely reducing denoising work. Results cover two independently implemented backbones.
- Explicit runtime safeguards and detailed reporting.
The framework specifies prefix ownership, real-context updates, and fresh-context fallback. The appendices provide complete task-level outcomes, candidate statistics, and increased model-side cost, making the benefits and limitations more transparent.
Critical weaknesses
- The gate’s contribution needs cleaner isolation.
RTC is described as differing from FlowLap only in the validity gate, yet the separate “pure overlap (no gate)” control performs substantially worse than RTC, 80.7% versus 88.7% on FlowWAM. RTC retains native continuation decisions, and FlowLap also projects actions before gating. These distinctions require explicit controls separating candidate rejection from action projection and other safeguards.
- Limited statistical support for success improvements.
Evaluation uses only three episodes per task under one fixed initialization seed. Uncalibrated FlowLap improves over RTC by only four episodes on FlowWAM and one on LingBot-VA, without uncertainty estimates or repeated evaluations. These results are promising but insufficient to establish reliable improvements in task success.
- Deployment benefits remain narrowly evaluated.
The gate is evaluated before the new boundary observation, so reduced waiting does not establish faster reactions to environmental changes. Experiments cover only clean, easy simulation without external disturbances, while reported model-side time increases by 118–335% over efficient synchronous controls. End-to-end completion time and disturbance-response measurements would better establish the practical value of this trade-off.
Questions for authors
What precisely distinguishes RTC from “pure overlap”? Can you isolate candidate rejection from action projection while keeping scheduling and other safeguards fixed?
Can you repeat calibration on/off with identical model-RNG and attention-backend settings, and include RTC with the same calibration?
Can you provide paired episode outcomes and repeated evaluation seeds with uncertainty estimates for the success-rate differences?
How do total episode time, observation-to-action latency, and success change under shorter execution windows or controlled external disturbances?
9 comments
Response to reviewer N3rt (2/3)
Q3. Can you repeat calibration on/off with identical model-RNG and attention-backend settings, and include RTC with the same calibration?
reply: Execution-state calibration improves RTC directly, and FlowLap with the same calibration delivers an additional gain in both success and latency. We evaluated four configurations on the 50-task LingBot-VA suite, with three episodes per task: RTC and FlowLap, each with and without calibration. The configurations use the same execution window and budget of two video-generation steps and five action-generation steps. RTC and FlowLap retain prefix handling and real-context refresh; FlowLap additionally validates candidate commitment. Calibration conditions the action branch on the anticipated execution state through a residual adapter while keeping the backbone frozen.
| Configuration | SR | Mean wait (ms) |
|---|---|---|
| RTC | 85.3% | 264.1 |
| RTC + calibration | 88.0% | 105.4 |
| FlowLap | 86.0% | 232.4 |
| FlowLap + calibration | 92.0% | 71.8 |
Calibration improves execution-state alignment under both scheduling protocols. Adding calibration to RTC raises SR from 85.3% to 88.0% and reduces mean wait from 264.1 to 105.4 ms, a 60.1% reduction. Adding calibration to FlowLap raises SR from 86.0% to 92.0% and reduces mean wait from 232.4 to 71.8 ms, a 69.1% reduction. These comparisons show that future-state conditioning improves the actions proposed for later execution under both RTC and FlowLap.
Validated commitment provides an additional benefit when calibration is present in both methods. With the same calibration, FlowLap reaches 92.0% SR and 71.8 ms mean wait, gaining 4.0 percentage points and reducing wait by 31.9% relative to calibrated RTC. The adapter improves actions for their anticipated execution state, and the gate checks whether the resulting continuation remains compatible when consumed. Together, the four configurations distinguish the benefit of future-state conditioning from the additional benefit of validated admission. The calibration ablation now includes the RTC-calibrated control.
Q4. Can you provide paired episode outcomes and repeated evaluation seeds with uncertainty estimates for the success-rate differences?
reply: Repeated evaluations show higher mean success for FlowLap than RTC on both backbones, with calibration bringing LingBot-VA to 94.00% mean SR. We repeated evaluation with seeds {1,2,4}, using 50 tasks and three episodes per task for each seed: 150 episodes per seed and 450 episodes per configuration. These are evaluation seeds for the reported policies. The table reports mean SR and sample standard deviation across the three evaluation-seed success rates.
| Model | Configuration | Mean SR | SD |
|---|---|---|---|
| FlowWAM | Efficient sync | 95.78% | 2.69% |
| FlowWAM | RTC | 83.11% | 3.67% |
| FlowWAM | FlowLap | 91.11% | 3.36% |
| LingBot-VA | Efficient sync | 94.22% | 2.78% |
| LingBot-VA | RTC | 87.56% | 0.77% |
| LingBot-VA | PAINT | 85.11% | 2.69% |
| LingBot-VA | SEAM | 90.00% | 0.67% |
| LingBot-VA | FlowLap | 90.44% | 0.77% |
| LingBot-VA | FlowLap + calibration | 94.00% | 2.40% |
The seed-level outcomes show that the FlowLap–RTC success ordering persists in every repeated LingBot-VA evaluation. Each entry below is the number of successful episodes out of 150.
| LingBot-VA configuration | Seed 1 | Seed 2 | Seed 4 |
|---|---|---|---|
| RTC | 132/150 | 130/150 | 132/150 |
| FlowLap | 135/150 | 137/150 | 135/150 |
| FlowLap + calibration | 144/150 | 137/150 | 142/150 |
The multi-seed results support a success advantage over RTC across two independently implemented backbones. On FlowWAM, FlowLap achieves 91.11% mean SR versus RTC's 83.11%, an 8.00-percentage-point gain. On LingBot-VA, it achieves 90.44% versus 87.56%, a 2.88-point gain, with both methods having a seed-level SD of 0.77 percentage points. FlowLap completes 3, 7, and 3 additional episodes relative to RTC across seeds {1,2,4}. Calibration further raises the mean to 94.00%, compared with efficient synchronous execution's 94.22%, while its seed-level outcomes exceed RTC in all three evaluations. The aggregate statistics and individual seed counts together show that the success improvement persists across repeated evaluations. The appendix and supplementary material provide the corresponding task-level result records.
Response to reviewer N3rt (3/3)
Q5. How do total episode time, observation-to-action latency, and success change under shorter execution windows or controlled external disturbances?
reply: Physical-robot results demonstrate lower execution-boundary stalls with improved task success, and the launch-timing ablation shows how available overlap time affects these gains. The SO-ARM101 evaluation uses a 5-DoF arm, a 1-DoF gripper, a top-view RGB camera, and an RTX 4090, with 50 episodes each for StackCube and PickNPlace. The measurements include real camera observations, actuator execution, communication, and host scheduling.
| Task | Configuration | SR | Mean wait (ms) | p50 wait (ms) | p90 wait (ms) | Acceptance |
|---|---|---|---|---|---|---|
| StackCube | RTC | 86% | 202.163 | 30.562 | 986.841 | -- |
| StackCube | FlowLap | 88% | 222.334 | 31.924 | 1,204.948 | 0.861 |
| StackCube | FlowLap + calibration | 92% | 68.221 | 34.170 | 34.170 | 0.973 |
| PickNPlace | RTC | 96% | 292.440 | 33.910 | 1,652.645 | -- |
| PickNPlace | FlowLap | 100% | 69.669 | 32.679 | 32.679 | 0.833 |
| PickNPlace | FlowLap + calibration | 100% | 36.843 | 36.843 | 36.843 | 1.000 |
Transition wait measures the stall visible to the robot at an action-chunk handoff. It is computed as , where is the end of the current execution interval and is when the next action becomes available to the executor. Mean wait summarizes average blocking, and p90 captures long transition stalls. Candidate acceptance is the fraction of speculative candidates accepted for execution. SR is the fraction of successfully completed episodes. The p50 wait is the median transition wait.
Calibrated FlowLap sharply reduces the transition stalls that accumulate during physical task execution. Relative to RTC, it raises StackCube SR from 86% to 92%, reduces mean wait by 66.3% to 68.221 ms, and reduces p90 wait by 96.5% to 34.170 ms. On PickNPlace, it raises SR from 96% to 100%, reduces mean wait by 87.4% to 36.843 ms, and reduces p90 wait by 97.8%. These host control-loop measurements quantify the boundary-wait component of completion time; the task success rates give the corresponding end-to-end outcomes.
High acceptance under real sensing and actuation shows that validated overlap remains effective on the robot. Uncalibrated FlowLap accepts 86.1% of candidates on StackCube and 83.3% on PickNPlace; calibration raises these rates to 97.3% and 100%. On StackCube, calibration simultaneously increases acceptance and reduces p90 wait from 1,204.948 to 34.170 ms. Conditioning on the anticipated execution state improves the continuation supplied to the gate, and real-context refresh anchors subsequent generation to physical observations. The controller preserves its rejection and fresh-context fallback rules throughout deployment.
Response to reviewer N3rt (1/3)
We thank the reviewer for the careful review and constructive suggestions. Below, we address each concern with experimental results and mechanism-level explanations.
Q1. What precisely distinguishes RTC from “pure overlap”?
reply: RTC retains native continuation decisions, executor-owned prefix handling, suffix completion, and real-observation boundary refresh. Pure overlap directly consumes an early-ready candidate. Both omit the FlowLap handoff-validity gate.
Q2. Can you isolate candidate rejection from action projection while keeping scheduling and other safeguards fixed?
reply: Candidate rejection and action projection address complementary handoff requirements, and separate controls expose their contributions on both backbones. Rejection checks whether a speculative candidate remains eligible for the current execution state and handoff; projection adjusts the replaceable suffix to satisfy boundary constraints while preserving the executor-owned prefix. We report projection-only and rejection-only controls alongside the complete protocol on the same 50-task suite, with three episodes per task. The table also includes calibrated FlowLap on LingBot-VA so that the effect of improving candidate quality can be assessed separately from the component comparisons. SR denotes the fraction of successfully completed episodes, and mean wait measures blocking at action-chunk transitions.
The complete handoff protocol gives the highest success rate on FlowWAM. FlowLap reaches 91.3% SR; projection-only and rejection-only reach 84.7% and 84.0%, respectively, giving gains of 6.6 and 7.3 percentage points. Rejection removes stale or temporally invalid candidates, and projection aligns the unexecuted suffix with the committed boundary. Applying both operations gives the controller a continuation that satisfies the scheduling and boundary-consistency requirements together. The results support the contribution of the complete handoff protocol on this backbone.
Execution-state calibration improves the candidate supplied to the complete handoff protocol on LingBot-VA. With the full protocol retained, calibration raises SR from 86.0% to 92.0% and reduces mean wait from 232.4 to 71.8 ms, a 69.1% reduction. The resulting configuration also exceeds projection-only and rejection-only, which achieve 87.3% and 88.7% SR with 94.1 and 98.3 ms mean wait. These comparisons establish two distinct findings: the component controls measure the outcomes of suffix adjustment and candidate rejection, while the calibration comparison measures the benefit of conditioning actions on their anticipated execution state before the full gate evaluates commitment. The ablation section reports these controls and their distinct roles.