- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes STaC-WAM to address the inference overhead caused by depth coupling between the video backbone and action expert in implicit world-action models. Its core component is a spatio-temporal action-conditioning bridge that groups multi-layer video KV features, preserves structural information through object-slot compression, Object Assignment Alignment (OAA), and Temporal Similarity Alignment (TSA), and reconstructs action-conditioning features through object and detail pathways. This design reduces the action expert from 30 to 6 layers while keeping the video backbone unchanged. The paper reports an improvement in average LIBERO success rate from 97.60% to 98.85% and a reduction in end-to-end inference latency from 192.83 ms to 91.68 ms, with additional evaluations on RoboTwin, five real-robot tasks, and a subset of VLABench tasks.
Critical strengths
- STaC-WAM reduces inference latency from 192.83 ms to 91.68 ms while improving LIBERO success from 97.60% to 98.85%, demonstrating a promising performance–latency trade-off.
- Combining object-centric compression with a complementary detail pathway is a reasonable approach to preserving control-relevant information, with component ablations supporting their complementarity.
- Experiments cover LIBERO, the full RoboTwin benchmark, and five real-robot tasks, providing evidence across simulated and physical manipulation settings.
Critical weaknesses
- Table 3 compares compression strategies and alignment objectives, but lacks an MLP compression baseline trained with the same TSA supervision and does not report parameter counts or latency for each variant. It therefore remains unclear how much of STaC’s improvement comes from structured compression rather than auxiliary supervision or differences in model capacity.
- Faster-WAM already decouples video-backbone and action-head depths through cross-layer KV fusion, while Light-WAM also explores compact action interfaces built from multi-layer video representations. The paper should clarify the additional value of its object structure, temporal alignment, and dual-path reconstruction relative to these designs and provide corresponding comparisons.
- The manuscript does not sufficiently specify the TSA prediction head, how the spatial prior is incorporated into the detail decoder, or key training hyperparameters and real-robot evaluation protocols.
Questions for authors
- How does MLP compression with the same TSA supervision compare with full STaC in success rate, parameter count, and latency?
- What advantages does STaC provide over the interfaces in Faster-WAM and Light-WAM under comparable settings?
- Could you specify the TSA head, spatial-prior integration, training hyperparameters, and real-robot evaluation protocol, and provide the teacher checkpoint or pseudo-labels needed to reproduce OAA?
5 comments
Response to Weaknesses (2)
W3. Implementation and evaluation details
TSA prediction head. The head takes the reconstructed KV features from the current latent frame, which are also supplied to the Action DiT. For the K branch in LIBERO, the input consists of 98 tokens, each represented by a 3,072-dimensional vector. The head applies LayerNorm followed by a linear projection from 3,072 to 98 dimensions to each token. Each output score corresponds to one position in the next latent frame. This yields a matrix, where entry scores the predicted similarity between current position and future position .
For group and branch , the logits are . We apply softmax along the future-position axis so that each row is a distribution over positions in the next latent frame. During training, we use cross-entropy to match this prediction to a target distribution computed from the cross-frame cosine similarities of the normalized, group-averaged source features. The next latent frame is used only to construct the target; it is not an input to the prediction head. K and V have separate heads, and neither head is run at inference.
Spatial prior in the detail decoder. Row of the slot-mediated prior indicates which source positions are favored when reconstructing output position . The detail decoder also computes content scores for these source positions. It first updates learned positional queries by cross-attending to the pooled source features . It then projects the updated queries to and the pooled features to , yielding content logits .
We smooth the spatial prior by mixing it with a uniform distribution, , and add it as a log bias before the final retrieval softmax:
.
Here, and are learned positive scalars, and indexes source positions in the current frame. Before normalization, the retrieval weight is proportional to . This gives preference to source positions consistent with the slot assignments while allowing the content scores to adjust the retrieval. The decoder reads , then applies LayerNorm and a linear projection before the token-wise router fuses the resulting detail features with the object-path features.
Training. We use eight H200 GPUs for each benchmark. LIBERO uses a per-GPU batch size of 16 for 10 epochs, and RoboTwin uses 24 for five epochs. Both use AdamW with , , weight decay 0.01, gradient accumulation of one, and an initial learning rate of with cosine decay. The main STaC configuration uses three slots and one Slot Attention iteration. We set , , and the TSA temperature .
Real-robot evaluation. We use a GALAXEA R1 Lite mobile manipulator with two 6-DoF arms and two grippers. The policy receives wrist and third-person camera observations at resolution. We evaluate five tasks spanning basic manipulation and real-time decision-making. We collect 80 expert demonstrations per task and evaluate 50 rollouts per task, with small random perturbations to initial object positions and poses.
We will include these details in the revision.
Response to Questions
Thank you for your thoughtful questions and for engaging so closely with our work. We appreciate the opportunity to explain our design choices more clearly and share additional results.
Q1. Success rate, parameter count, and latency of MLP + TSA and full STaC
| ID | Interface | LIBERO Avg. SR (%) | RoboTwin Clean/Randomized SR (%) | Bridge/Total Params. (B) | End-to-End Latency (ms) |
|---|---|---|---|---|---|
| A4 | MLP + TSA | 97.75 | 84.32 / 85.40 | 0.11 / 5.33 | 74.42 |
| A5 | Large MLP + TSA | 97.80 | 86.28 / 87.44 | 1.38 / 6.59 | 113.84 |
| A9 | Full STaC (TSA + OAA) | 98.85 | 92.25 / 92.11 | 1.38 / 6.59 | 91.68 |
Compared with MLP + TSA (A4), full STaC improves success by 1.10 percentage points on LIBERO and 7.93/6.71 points on RoboTwin Clean/Randomized at an additional 17.26 ms. Compared with the parameter-matched Large MLP + TSA (A5), full STaC improves success by 1.05 points on LIBERO and 5.97/4.67 points on RoboTwin Clean/Randomized while reducing latency by 22.16 ms.
Q2. Advantages under comparable settings
Faster-WAM (Zhao et al.) fuses neighboring video layers by taking a weighted sum of K/V at each token position. STaC uses object slots to guide the retrieval of local details from the source K/V when reconstructing each output patch. This reconstruction runs once per observation and produces six K/V groups for a six-layer Action DiT. Each denoising step therefore executes six action blocks, compared with 30 in Faster-WAM, eight of which read video K/V.
Light-WAM pools features from three selected video layers into one vector shared by all action steps. STaC retains spatially indexed K/V, so different action queries can attend to different image regions as the action chunk is denoised.
Table A tests the bridge with the backbone, layer grouping, six-layer Action DiT, and TSA objective fixed, with OAA disabled. Replacing our equal-weight grouped K/V fusion control (A6) with the STaC bridge (A8) improves success by 2.70 percentage points on LIBERO and 5.68/5.86 points on RoboTwin Clean/Randomized. Against the parameter-matched Large MLP + TSA (A5), STaC improves success by 0.40 points on LIBERO and 2.34/1.90 points on RoboTwin.
In the matched RTX 5090D benchmark reported in our response to Weakness 2, STaC-WAM runs 2.07× faster than Faster-WAM (91.68 vs. 189.63 ms), with similar success rates reported in the respective papers. Light-WAM runs faster in the same timing benchmark (67.57 ms), while STaC-WAM’s reported RoboTwin success rates are 15.85/15.81 percentage points higher in the Clean/Randomized settings.
Q3. TSA head, spatial prior, training, real-robot protocol, and OAA resources
See our response to Weakness 3 for the implementation and evaluation details. The VideoSAUR teacher checkpoints used to generate the OAA masks for LIBERO, RoboTwin 2.0, and the real-robot experiments are all available at Hugging Face.
If the new results and clarifications address your main concerns, we would appreciate it if you could reconsider your overall assessment.
Response to Weaknesses (1)
Thank you for taking the time to review our work so carefully. We greatly appreciate your thoughtful and constructive feedback, which helped us clarify what STaC contributes and strengthen the experimental evaluation.
W1. Improvement from structured compression
Table A. Interface and supervision ablations
Table A compares the original MLP + TSA (A4), the parameter-matched large MLP + TSA (A5), and Slot/STaC + TSA (A8). A5 and A8 use the same six-layer Action DiT and TSA objective, with OAA disabled in both. Each has 1.38B bridge parameters and 6.59B total inference parameters.
Under these matched conditions, A8 improves success over A5 by 0.40 percentage points on LIBERO and 2.34/1.90 points on RoboTwin Clean/Randomized, while reducing measured latency from 113.84 to 91.68 ms. Adding OAA gives the full STaC model (A9), which reaches 98.85% on LIBERO and 92.25%/92.11% on RoboTwin Clean/Randomized at 91.68 ms.
W2. Relation to Faster-WAM and Light-WAM
Faster-WAM reduces the cost of explicit future conditioning by restricting visual K/V reads to 8 of its 30 action layers. Each denoising step still runs all 30 layers. STaC-WAM compresses the K/V from the current observation into six groups, enabling a six-layer action expert. In our matched RTX 5090D benchmark, STaC-WAM is 2.07× faster, and the two papers report similar success rates on LIBERO and RoboTwin 2.0. Zhao et al.
Light-WAM reduces computation with a compact video backbone, learned-query pooling over three selected video layers, and single-pass action regression. Its training efficiency study reports 1.99B loaded parameters, 0.44B trainable parameters, and 4.25× the training throughput of Fast-WAM. STaC-WAM draws on all 30 video layers and combines object slots with a spatial detail pathway to condition a six-layer action expert. Across the two papers, STaC-WAM reports RoboTwin 2.0 success rates about 16 percentage points higher in both settings. The two systems differ in backbone, model size, and action decoder. Li et al., 2026
On the RTX 5090D, STaC-WAM and Zhao et al.’s Faster-WAM were timed with five action-denoising steps, while Light-WAM used its native single-pass action head. Reported success rates follow each paper’s evaluation settings.