- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents Solis, a unified video diffusion transformer that jointly models future observations, actions, and progress-derived values. It combines temporal block-causal attention, diffusion forcing, and rolling sampling to retain partially denoised trajectories across control cycles, enabling frequent closed-loop feedback at reduced inference cost. It reports 98.4% average success across the four LIBERO suites and 88.8%/89.1% on a 12-task RoboTwin 2.0 subset (clean/randomized), and it is demonstrated on two real-robot manipulation tasks.
Critical strengths
The proposed model supports action generation, future prediction, and value estimation through different conditioning modes, providing a versatile framework for robotic tasks. The method achieves competitive LIBERO performance, reasonable RoboTwin results, with real-robot validation.
Critical weaknesses
-
The paper lacks ablation studies to isolate the contributions of video and value prediction to policy performance. All main closed-loop results use joint generation. World-model mode is not used for planning, and evaluator mode is not validated for action selection. Therefore, the practical advantages of jointly modeling all three modalities remain unclear.
-
The rolling sampler improves average LIBERO success by only 0.3 percentage points over the non-rolling baseline. More importantly, the comparison is mixed with different feedback frequencies (rolling replans every four actions, whereas non-rolling executes 16 actions open loop). Therefore, the gains cannot be distinctively attributed to retaining partially denoised trajectories.
Questions for authors
-
How does an action-only variant with the same backbone and rolling schedule perform? Can the authors demonstrate that joint video and value prediction improves closed-loop control, or that evaluator mode can meaningfully distinguish successful from failed trajectories?
-
Can the authors compare rolling and non-rolling sampling under matched feedback frequencies, and report the performance–computation trade-off? This would clarify whether the benefits arise from retaining partially denoised states rather than simply from observing more frequently.
-
Can the authors clarify how the 9.1 Hz feedback rate (in Table 2) is measured, including hardware and timing methodology?
5 comments
Response to Reviewer ivkr (Part 2/2)
W2 / Q2: Matched feedback and sampling cost
Matched-feedback comparison. Following your suggestion, we compare rolling and non-rolling sampling using the same checkpoint and joint-generation mode within each benchmark, with matched feedback intervals, open-loop execution lengths, and forward passes per query. Both obtain a new observation after every four executed actions on LIBERO or 16 on RoboTwin and use one network evaluation per query after initialization.
Rolling retains partially denoised future chunks across queries and refines them under new observations. Non-rolling generates the trajectory from fresh noise at each query and executes only its first action chunk.
LIBERO. Results average four suites with 500 episodes each.
| Sampler | Trajectory reuse | Feedback interval | Open-loop length | Forward passes per query | Success (%) |
|---|---|---|---|---|---|
| Rolling | Yes | 4 | 4 | 1 | 98.3 |
| Non-rolling | No | 4 | 4 | 1 | 97.20 |
RoboTwin. Results cover all 50 tasks, with 100 episodes per task in each setting.
| Sampler | Trajectory reuse | Feedback interval | Open-loop length | Forward passes per query | Clean (%) | Randomized (%) |
|---|---|---|---|---|---|---|
| Rolling | Yes | 16 | 16 | 1 | 88.94 | 87.60 |
| Non-rolling | No | 16 | 16 | 1 | 86.98 | 85.40 |
With feedback intervals, open-loop execution lengths, and forward passes per query matched, rolling exceeds non-rolling by 1.10 percentage points on LIBERO, 1.96 points on RoboTwin clean, and 2.20 points on RoboTwin randomized.
Q3: Measurement of the 9.1 Hz feedback rate
The 9.1 Hz feedback rate reported in Table 2 was measured on an NVIDIA RTX PRO 6000 GPU. The measured inference time includes VAE encoding of visual observations and denoising for action generation, corresponding to approximately 110 ms per policy update. We measure the average inference latency over repeated policy updates and report its reciprocal as the feedback rate.
Invitation to Discuss Any Remaining Concerns
Dear Reviewer ivkr,
Thank you for the time and care you have devoted to reviewing our work. As the discussion period draws to a close, we would be grateful if you could take another look at our responses and revised manuscript when your schedule permits. Your feedback has helped us improve the paper, and we would welcome any further questions or concerns.
We appreciate your time and consideration.
Best regards, The Authors
Response to Reviewer ivkr (Part 1/2)
Thank you for your review and questions. We address the contributions of video and value prediction, the sampling comparison, and the feedback-rate measurement below.
W1 / Q1: Contributions of video and value prediction
Training ablations. We compare three fine-tuning variants: action prediction only, action and future-frame prediction, and the full model with value prediction. All variants use the same backbone, start from the same jointly pretrained model, and are evaluated on LIBERO with the K=4 rolling sampler. The prediction objectives are changed during fine-tuning. The table reports average success rates (%).
The action-and-frame model reaches 97.8%, compared with 91.65% for action only, a difference of 6.15 percentage points. The full model reaches 98.3%, a further 0.50 points above the variant without value prediction and 6.65 points above action only. The full model achieves the highest control success rate among the three variants.
Role of value. Value is an auxiliary progress signal that can indicate the progress of the current task. Enabling the model to be aware of this provides a global understanding of the task which can benefit the task execution. We have included a clearer statement in the revised paper.
The table below shows predictions from five failed RoboTwin rollouts. The values in the table are rescaled to 0–1, where 1 corresponds to the start of a demonstration and 0 to its end.
These examples show that predicted value can reflect local stagnation, repeated attempts, and renewed progress during execution. For a clearer visualization of this correspondence, please refer to the aligned video keyframes and value curves in Appendix D, Figs. 5–9 of the revised manuscript.
Action selection with world modeling and value guidance. We also use jointly predicted values to select action candidates on three RoboTwin tasks, using rolling sampling with the same model weights as in the 50-task RoboTwin evaluation reported in this rebuttal. Each candidate path contains jointly predicted future observations, actions, and values, so world modeling participates in candidate generation, while the predicted value decrease provides the selection criterion. For each block, we generate two candidates from different noise samples, maintaining 24=16 candidate paths across four blocks. At each query, we select the candidate for the leading block with the larger predicted value decrease and retain the eight paths sharing that block. As the horizon shifts, each retained path is extended with two candidates for the new trailing block, restoring the total to 16 paths. Candidate noise seeds are determined by the base seed (0), environment seed, block index, and candidate index.
We evaluate both methods using the same 100 environment seeds for each task in each of the clean and randomized settings. Each cell reports ordinary rolling / value-guided success rates.
handover_blockpick_diverse_bottlespick_dual_bottles