- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The reviewer uses "human" to denote my hand-written opinions in the review and the reviewer uses "Opus" to denote Opus 5.5. The reviewer uses the incognito mode so no data will be shared to Claude.
Human: This paper proposes a progressive sparse forcing mechanism to reconfigure the action-to-world conditioning. The proposed approach DAAR scores the usefulness of the visual token w.r.t. the action token. It outputs an action-world relevance matrix (eq.3). It then gathers only the most relevant attention values (eq. 5), which is called the sparse-forcing attention. Then it uses a progressive schedule to ensure the later stages contain increasingly compact visual visibility.
Critical strengths
Human:
-
The proposed mechanism is somewhat novel.
-
The DARR is simple and probably at low cost. It could generally benefit the domain.
-
It demonstrates strong performance on two distinct domains: robots and AD.
Critical weaknesses
Human:
-
No video demos are provided. This is required if the authors claim advancement in closed-loop simulation.
-
The author claims they have done experiments in autonomous driving scenarios. However, no video demos or figure visualizations are provided. Only presenting numbers is not sufficient to demonstrate the superiority of the proposed approach. In AD scenarios, the reviewer wishes to see whether the proposed approach could change agent behavior or not.
-
The closed loop simulation performance are saturated (>98%). It's hard to say whether the proposed mechanism is really better than the baseline.
-
It's hard to see whether the proposed mechanism would deteroiate the base model performance because of the proposed mechanism.
-
Figure 1 isn't clear and doesn't make much sense. It's hard to know how the "attention values" are computed and what kind of normalization is applied. And it's hard to say whether larger values mean "stronger attention". It's hard to say whether the visualization really discovers an issue. Probably manipulating the features would harm the base model performance. It's hard to say whether a mistake may exist in such a visualization, which may reverse the conclusion.
-
Opus finds a lot of unresolved issues.
Opus (verified by the reviewer):
The LIBERO gains are within noise, and no variance is reported anywhere. LIBERO is near saturation: the overall gains are +0.5 (IDM), +0.6 (JD), and +1.0 (FastWAM), from 50 trials per task and, as far as I can tell, a single training seed. JD-WAM even drops on Spatial (99.0 → 98.6). Without seeds or confidence intervals, these results show parity at best. The same concern applies to the per-axis LIBERO-Plus numbers, where some axes contain only a few hundred episodes, and to the Table 5 schedule ablation, where all variants fall within 96.6–97.6. The driving results also have no variance across training runs.
The control needed to support "learned visual focusing" is missing. The central claim is that PSF works because the router learns to select action-relevant visual tokens. But restricting visual visibility in deep layers is also a form of structured token dropout, which could plausibly explain the OOD gains on its own, since it reduces reliance on spurious visual context. The ablation (Table 5, left) compares only schedules of the same learned router against the dense baseline.
The driving evaluation is limited, and the efficiency analysis is incomplete. Driving is open-loop ADE/FDE only, with no collision, off-road, or closed-loop metrics. "Long-horizon planning" is a stretch for 3–4 s horizons with the cache refreshed from ground-truth observations at every chunk.
Questions for authors
Opus:
How does PSF compare with random Top-K at the same schedule, with score-bias-only (no sparsity), and with Top-K from backbone attention logits, on LIBERO-Plus?
Can you quantify grounding, for example as attention mass on object, gripper, and target masks in LIBERO, for dense vs. PSF?
What is the latency of dense DriveWAM at v = 1, a = 2, and which single configuration does "+PSF-b" correspond to?
How many training seeds were used, and what is the variance on LIBERO, LIBERO-Plus, and driving?
How do the reproduced baselines compare with the published FastWAM, Lingbot-VA, DreamZero, and DriveWAM numbers, and how were PSF's hyperparameters selected?
How does performance change as the deepest-stage retention drops below 0.4?
8 comments
3. Matched-step driving latency and the definition of “+PSF-b”
Addresses Q3 and the reviewer-verified Opus concern about inference efficiency.
Driving performance and latency under explicit denoising configurations.
We provide the requested dense result alongside PSF at the same budget and the selected DriveWAM-best configuration.
| Method | Video steps | Action steps | Latency (ms) ↓ | ADE@3s ↓ | FDE@3s ↓ | ADE@4s ↓ | FDE@4s ↓ |
|---|---|---|---|---|---|---|---|
| DriveWAM | 1 | 2 | 409.1 | 0.597 | 1.448 | 0.945 | 2.465 |
| DriveWAM-best | 1 | 5 | 936.9 | 0.480 | 1.310 | 0.830 | 2.380 |
| DriveWAM + PSF | 1 | 2 | 436.7 | 0.466 | 1.258 | 0.793 | 2.239 |
At the same two-step action-denoising budget, PSF improves all four trajectory metrics with an additional 27.6 ms (6.7%) of latency. As detailed in “Latency breakdown and speedup attribution,” of our response to Q6bN, routing and selection overhead can outweigh sparse-attention savings for small, single-camera inputs, whereas computational benefits emerge with larger visual contexts, including higher-resolution and multi-view inputs.
Our primary goal is to improve action prediction. Compared with DriveWAM-best, two-step PSF achieves lower errors on all four metrics while reducing latency from 936.9 to 436.7 ms (2.15× speedup), demonstrating improved accuracy with fewer denoising steps.
What does “+PSF-b” denote?
We apologize for the ambiguous notation. “b” denotes the selected best-performing configuration, with one video-denoising step and three action-denoising steps. Its reported latency (573.1 ms), ADE@4s (0.76), and FDE@4s (2.21) all correspond to this single configuration, rather than separately selected settings for different metrics. We have clarified these video/action step counts in Section 4 and Tables 2–3 of the revised manuscript, with reviewer-specific changes marked in purple.
4. Baseline reproduction and comparison with reported results
Addresses the baseline-comparison part of Q5.
Implementation and comparability.
We reproduce FastWAM using its official code. For the other manipulation layouts, we follow the same codebase to train a non-autoregressive IDM-WAM following the Lingbot-VA architectural design and a JD-WAM following the DreamZero-style joint-denoising design. These are controlled layout implementations. In particular, Lingbot-VA uses an autoregressive rollout framework, and DreamZero does not provide corresponding LIBERO results for a direct comparison.
Below, we compare reference scores with our reproductions. The IDM-WAM and JD-WAM reference rows concern the corresponding layout variants, not the complete original models.
The reference results do not report multi-seed statistics, whereas our reproductions report mean ± SD across three independent training runs. Training-run variability may therefore contribute to the differences between the reported and reproduced scores, although the available reference results do not allow us to quantify this contribution.
LIBERO success rates (%).
| Method | Result | Spatial | Object | Goal | Long | Overall |
|---|---|---|---|---|---|---|
| FastWAM | Reported | 98.2 | 100.0 | 97.0 | 95.2 | 97.6 |
| FastWAM | Our reproduction | 95.7 ± 0.9 | 99.4 ± 0.5 | 97.1 ± 0.8 | 92.9 ± 0.8 | 96.3 ± 0.7 |
| IDM-WAM | Reference variant | 98.8 | 97.8 | 97.8 | 97.6 | 98.0 |
| IDM-WAM | Our reproduction | 98.8 ± 0.5 | 98.1 ± 0.5 | 97.7 ± 0.3 | 97.7 ± 0.5 | 98.1 ± 0.4 |
| JD-WAM | Reference variant | 99.6 | 99.4 | 98.2 | 96.8 | 98.5 |
| JD-WAM | Our reproduction | 99.2 ± 0.3 | 99.7 ± 0.3 | 98.2 ± 0.2 | 96.1 ± 0.6 | 98.3 ± 0.2 |
We closely follow the released code and experimental settings for each reproduced baseline. Our overall results differ from the reference scores by −1.3 percentage points for FastWAM, +0.1 for IDM-WAM, and −0.2 for JD-WAM. PSF is evaluated against these reproduced baselines using the same training hardware, training seeds, data, optimization settings, and evaluation protocol.
DriveWAM: offline trajectory errors.
| Result | ADE@3s | FDE@3s | ADE@4s | FDE@4s |
|---|---|---|---|---|
| Reported | 0.47 | 1.35 | 0.83 | 2.47 |
| Our reproduction | 0.47 ± 0.01 | 1.34 ± 0.01 | 0.84 ± 0.01 | 2.49 ± 0.01 |
The reproduced DriveWAM results closely match the reported values, with differences of at most 0.02 across these metrics.
LIBERO-Plus: comparison with a third-party FastWAM reproduction.
For additional context, we compare with the FastWAM results reported by FasterWAM. These are a third-party reproduction, not original FastWAM results.
| FastWAM result | Background | Camera | Language | Lighting | Object layout | Initial states | Sensor noise | Overall |
|---|---|---|---|---|---|---|---|---|
| Reported by FasterWAM | 45.70 | 18.75 | 70.08 | 83.23 | 62.73 | 45.69 | 29.84 | 49.14 |
| Our reproduction | 51.1 ± 5.5 | 18.3 ± 2.0 | 70.0 ± 0.6 | 84.8 ± 1.5 | 63.6 ± 0.9 | 45.8 ± 1.7 | 36.9 ± 6.4 | 51.4 ± 1.9 |
Our reproduction has a higher overall mean (51.4% vs. 49.14%), although this cross-publication comparison is contextual rather than controlled. The main evidence for PSF remains the matched baseline-versus-PSF comparisons in G1 and G2.
We have clarified the baseline implementations in Section 4 and Appendix D.1 of the revised manuscript (marked in purple).
How were PSF's hyperparameters selected?
We selected the three-router configuration based on the stage-count ablation in Table 5 of the original manuscript and fixed the default retention schedule across layouts and benchmarks. The additional comparison with the 0.3-decrement schedule, [0.9, 0.6, 0.3], further supports this choice: it reaches 97.2% overall LIBERO success, compared with 97.6% for the default schedule.
We sincerely appreciate Reviewer ezYk’s recognition of the mechanism’s simplicity, potential usefulness, and results across manipulation and driving. Your detailed comments have helped us identify where additional evidence and clearer explanations are needed.
Several concerns are addressed in our shared official comments:
Revisions addressing the shared concerns are marked in red in the revised manuscript; their locations are listed in our general official comments.
Below, we address the remaining concerns individually, including the visualization procedure and the requested experimental details.
1. Video demonstrations
Addresses the reviewer’s Weaknesses 1–2.
We apologize for not providing sufficient video demonstrations in the original submission. We have added closed-loop manipulation videos on LIBERO, rollouts under LIBERO-Plus perturbations, and open-loop driving trajectory visualizations to the supplementary material.
1.1 Additional closed-loop driving evaluation
Addresses the reviewer’s Weakness 2 and the concern about open-loop driving evaluation.
Following your suggestion to examine whether PSF changes driving behavior, we adapt DriveWAM and DriveWAM+PSF to closed-loop simulation on the official local AlpaSim validation set, following the DriveWAM baseline settings and adapting its standard outputs to the simulator. Both variants use the same evaluation protocol.
Compared with the adapted baseline, PSF reduces close encounters and off-road rate while improving progress and both AlpaSim scores. The closed-loop videos further illustrate the resulting driving behavior. We have added the evaluation protocol and results to the appendix, with reviewer-specific additions marked in purple,
Because the baseline model DriveWAM was not originally evaluated on this closed-loop benchmark, we treat this as an exploratory comparison of matched adaptations, rather than a comparison with specialized closed-loop driving systems.
2. Attention computation and visualization
Addresses the reviewer’s Weakness 5 and the visualization aspect of Q2.
How are the attention values computed?
The displayed values originate from post-softmax attention weights. Softmax is computed over all allowed video and action keys. For each map, we average over attention heads and the first 10 action queries, corresponding to the first 0.5 seconds of actions. We then display the primary-camera visual tokens from the first conditioning frame as a 7×7 grid.
What do the colors and gray grids represent?
Each heatmap is independently normalized for display. Colors therefore indicate relative attention within a panel, rather than comparable absolute attention magnitudes across layers or models. In the PSF visualizations, gray grids denote visual tokens excluded by the router; they are not inferred from low attention values.
What does this visualization establish?
The maps illustrate spatial attention patterns in the displayed examples; they do not independently establish causal importance or object-level grounding. G3 provides complementary behavioral evidence through matched selected-versus-unselected feature perturbations. We have clarified the visualization procedure in Appendix D.4 of the revised manuscript.