- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes SOAR, a one-step policy for robotic manipulation that predicts an action trajectory's discrete-cosine-transform (DCT) structure, uses low-, mid-, and high-frequency features to condition a direct action generator, and calibrates its output using the predicted spectrum. MeanFlow consistency and an endpoint objective train the calibrated action while inference uses one generator evaluation. On 3D Adroit and Meta-World benchmarks, the paper reports 83.9% overall success across 53 tasks at 8.7 ms latency, compared with 75.9% for the reproduced one-step MP1 baseline. It also reports one-step results on LIBERO and RoboTwin 2.0 and real-world π0.5 comparisons: 84% versus 88% on a 25-configuration tidy-up task, and 9/10 versus 8/10 successful dynamic pick-and-place trials. Ablations on the Adroit tasks suggest contributions from multi-band conditioning, calibration, and endpoint alignment.
Critical strengths
- SOAR gives a coherent formulation for adding explicit temporal structure to a one-step action generator. The DCT prediction, multi-band conditioning, action-space calibration, and inference-endpoint objective are defined mathematically, and the endpoint objective is shown to reduce to direct action matching at the one-step boundary. This makes the method's connection between training and inference relatively clear.
- The empirical evaluation is broad for a visuomotor-policy paper: it includes 53 simulated tasks, two multi-task VLA benchmarks, latency measurements, ablations, and two real-robot settings. The strongest controlled result is the 53-task comparison against reproduced one-step baselines, where SOAR reports a substantial success-rate gain at similar inference latency. The real-robot results also make the efficiency claim practically relevant.
Critical weaknesses
- Real-world robot evaluations contain only 25 tidy-up configurations and 10 dynamic pick-and-place trials. The tidy-up success differs by merely one case (21/25 vs 22/25), and dynamic pick-and-place differs by one successful trial (9/10 vs 8/10). Such small differences cannot support a reliable task success advantage. While the reduced empty-grasp rate is promising, the paper does not report per-trial variability or uncertainty estimates. Furthermore, cross-paper benchmark comparisons may adopt inconsistent evaluation protocols. These limitations weaken the claims on robustness and generalization, though the demonstrated one-step latency reduction remains valid.
- Multiple key components are inherited from existing one-step MeanFlow methods, including the dispersive regularizer from MP1. The proposed spectral calibration is essentially a simple residual correction operating in DCT space. Current ablations only test component effects on three Adroit tasks. It remains unclear how much of the full benchmark performance gain originates from the novel spectral mechanism, rather than the established one-step training recipe or hyperparameter tuning interactions.
- The overall novelty is limited; the proposed method appears to stack multiple existing techniques. The design uses DCT variation as a loss regularizer and smoothed actions as another loss regularizer, where the smoothing operation is implemented via the neural model. The design directly adds DCT-transformed condition features and raw condition features as residuals, which introduces unexplained information overlap across two different domains.
- The frequency domain is partitioned into high, mid, and low frequency bands. The hyperparameter choice of 3 frequency bands lacks ablation support: it is unclear whether 2 bands are sufficient, or whether further splitting into 4 bands would yield better performance. Experimental justification for this partition is required.
- SOAR’s benchmark performance is built upon MP1 / MeanFlow components. If baselines like DP, DP3, CP, and FlowPolicy are augmented with the same set of techniques, it remains unknown whether SOAR can still outperform them.
- In Libero experiments, SOAR uses PI05 as the model backbone. Existing results show PI05 already maintains full accuracy at NFE=1 on Libero. This critical baseline comparison is missing from the paper. Libero results must be reported with mean and standard deviation across multiple random seeds. Additional experiments on Libero-Pro or Libero-Plus are required.
Questions for authors
- Are the five selected checkpoints evaluated on the same episodes used for checkpoint selection? If yes, please report results using a disjoint held-out evaluation set and the final training checkpoint, with uncertainty estimates across multiple training seeds. This directly impacts the credibility of benchmark performance gains.
- Can you provide paired per-configuration outcomes and uncertainty metrics for both real robot experiments, especially dynamic target displacement trials? This will verify whether the observed differences in success rate and empty-grasp rate are statistically stable enough to support the claimed robustness improvement.
- What are the principled and empirical differences between SOAR and FreqPolicy, FAFM, FreqFM, especially those methods already applying supervision over DCT action structures? Could you add a matched one-step baseline equipped with a DCT-domain objective, and isolate the individual contribution of target spectrum prediction and the proposed action-space calibration?
- Could you provide full-benchmark ablations across extra training seeds: remove spectral calibration while keeping identical MeanFlow, endpoint, and dispersive objectives, and compare against a parameter- and compute-matched baseline with direct action correction? This will quantify the genuine contribution of spectral calibration, decoupled from the inherited one-step training recipe.
- Please clarify the design choice of residual addition between DCT-transformed condition and raw condition features. Why is summing features from two different domains reasonable, and how do you mitigate potential information overlap?
- Provide ablation studies to justify the hyperparameter of frequency band splitting. Compare performance under 2-band, 3-band, and 4-band frequency partitions.
- Please run control experiments: augment DP, DP3, CP, and FlowPolicy with MP1/MeanFlow components, and compare SOAR against these upgraded baselines.
- Add the critical PI05 baseline at NFE=1 for Libero. Report Libero results with mean ± standard deviation over multiple seeds. Supplement experiments on Libero-Pro or Libero-Plus.
I am open to raise my score if author solve my concerns.
5 comments
Q4
Response: Due to time and computational constraints, we could not complete full-benchmark ablations across all 53 tasks with additional training seeds. Instead, we conducted targeted ablations on 20 Adroit and Meta-World tasks with three independent training seeds, comparing SOAR without calibration and with direct action-space correction. All variants retain one-step inference and the same generation backbone and training protocol. The results support the contribution of spectral calibration beyond the shared MeanFlow training objectives.
1. Controlled variants. We evaluate:
- SOAR w/o AC: Removes spectral calibration while retaining MeanFlow consistency, inference-endpoint alignment, and Dispersive Loss.
- SOAR w/ direct correction: Replaces spectral calibration with direct action-space correction under the same backbone and training settings.
2. Quantitative comparison.
| Variant | MW Medium | MW Hard | Overall (20 tasks) |
|---|---|---|---|
| SOAR w/o AC | 72.3% | 62.2% | 70.3% |
| SOAR w/ direct correction | 72.3% | 60.9% | 70.0% |
| SOAR | 74.3% | 63.5% | 71.9% |
SOAR improves overall success by 1.6 percentage points over the no-calibration variant and 1.9 points over direct correction. The gains are particularly evident on challenging Meta-World tasks.
These results support the contribution of spectral calibration beyond the shared MeanFlow, endpoint, and dispersive objectives.
Q5
Response: SOAR does not directly add DCT coefficients to raw observation features. Instead, learned projections map each predicted frequency band into the common conditioning space, followed by gated residual addition. This preserves the original observation features while allowing the model to regulate structural guidance.
1. Feature fusion. The structure head predicts key temporal information from observations rather than independent executable actions. Each predicted frequency band is first transformed by a learnable projection into the observation-conditioning feature space, where it is residually combined with the original features.
2. Residual gating and information overlap. Since both branches originate from the same observation, information overlap is possible. We therefore use projections and gates to regulate the contribution of structural information and mitigate information overlap. Each frequency band has a learnable gate initialized to zero, allowing its structural contribution to be introduced gradually during training. The original observation features remain available through the residual connection.
See Appendix A.1.1, Eq. (8).
Q6
Response: We added frequency-partition ablations comparing two, three, and four bands across 20 tasks and three training seeds. All configurations use the same four DCT coefficients and one-step inference. The original three-band configuration achieves the highest overall success, supporting its use in the evaluated short-horizon setting.
1. Frequency partitions. All variants use K=T=4 and NFE=1, with unchanged structure prediction and calibration formulations.
| Configuration | Frequency groups |
|---|---|
| 2 bands | {0,1} / {2,3} |
| 3 bands (SOAR) | {0,1} / {2} / {3} |
| 4 bands | {0} / {1} / {2} / {3} |
All variants follow the same training and evaluation protocol with three independent seeds.
2. Results.
| Configuration | Overall success (20 tasks) |
|---|---|
| 2 bands | 70.9% |
| 3 bands (SOAR) | 71.9% |
| 4 bands | 71.2% |
The three-band configuration achieves 79.8% on Adroit, 74.3% on Meta-World Medium, and 63.5% on Hard. Its Medium-task success exceeds the two- and four-band variants by 1.48 and 0.85 percentage points, respectively, while differences on Hard are small.
These results support the three-band choice for T=4.
See Appendix A.5.5, Table 16.
Q7
Response: To examine whether SOAR's gains mainly arise from MP1/MeanFlow training components, we augmented existing policy backbones with the same training ingredients and evaluated them on the 37-task benchmark. SOAR retains the highest overall average, indicating benefits beyond the shared MP1/MeanFlow objectives.
1. MP1-augmented baselines. We retain each baseline's observation encoder and generation backbone while introducing MeanFlow/JVP training, dual-time conditioning, the guidance target from our local MP1 implementation, and Dispersive Loss (weight 0.5). All methods use NFE=1.
The adapted models include DP + MP1, DP3 + MP1, and FlowPolicy + MP1. For CP, removing the teacher and distillation objective makes its adapted student equivalent to our DP3 + MP1 implementation. We therefore report their shared result rather than treating CP as an independent fourth baseline.
2. Benchmark comparison.
| Method | Meta-World | Overall (37 tasks) |
|---|---|---|
| DP + MP1 | 81.6 ± 4.4% | 79.9 ± 4.2% |
| DP3 + MP1 | 81.9 ± 4.6% | 81.4 ± 4.3% |
| FlowPolicy + MP1 | 80.6 ± 4.6% | 81.1 ± 4.3% |
| SOAR | 84.8 ± 0.7% | 84.4 ± 0.5% |
SOAR exceeds these augmented baselines by 4.5, 3.0, and 3.3 percentage points overall.
Since all augmented baselines already incorporate the MP1/MeanFlow training ingredients, SOAR's overall gains cannot be explained solely by these shared components. The results further support using predicted temporal structure to guide action generation and calibration.
Implementation details and full results are provided in Appendix A.5.6 and Table 17.
Q8
Response: We added the critical π0.5 (NFE=1) baseline and three-seed evaluations on all 40 LIBERO tasks. We also extended the VLA comparison to RoboTwin 2.0 with longer action chunks. SOAR outperforms the reproduced one-step π0.5 baseline on both benchmarks. LIBERO-Pro and LIBERO-Plus evaluations remain incomplete due to training and benchmarking constraints.
1. LIBERO one-step and multi-seed comparisons. All models are evaluated using three independent training seeds and 50 episodes per task per seed.
| Model | NFE | Success rate |
|---|---|---|
| π0.5 | 10 | 96.50 ± 0.35% |
| π0.5 | 2 | 96.10 ± 0.43% |
| π0.5 | 1 | 95.48 ± 0.95% |
| π0.5-SOAR | 1 | 97.21 ± 0.34% |
SOAR improves success by 1.73 percentage points over the one-step π0.5 baseline and 0.71 points over the ten-step baseline, while requiring only one action-expert evaluation.
2. RoboTwin 2.0 evaluation. We further evaluated 50 tasks using 2,500 clean demonstrations and 50-step action chunks under the clean-to-clean setting.
| Model | NFE | Success rate |
|---|---|---|
| π0.5 | 10 | 70.0% |
| π0.5 | 2 | 69.1% |
| π0.5 | 1 | 65.1% |
| π0.5-SOAR | 1 | 71.7% |
Together, these results support SOAR's effectiveness across different VLA tasks and longer action horizons.
The updated comparisons are included in the Simulation Experiments with VLA Settings section. LIBERO statistics are reported in Appendix A.5.8, Table 19, and RoboTwin configurations in Appendix A.3.
We thank the reviewer for the constructive comments on evaluation reliability, real-robot uncertainty, frequency-domain methods, calibration ablations, frequency partitioning, baseline controls, and VLA generalization. We have added independent evaluations, extended ablations, and implementation details to address these concerns.
Q1
Response: To address potential checkpoint-selection bias, we evaluated the final SOAR and MP1 checkpoints on independent episodes with matched initial configurations and three training seeds. SOAR retains higher success across all task groups, suggesting that its improvements are not solely due to favorable checkpoint selection.
1. Original evaluation protocol. Following prior work, including MP1, we evaluated models every 200 epochs and averaged the five highest periodic scores per seed. These evaluations were also used for checkpoint selection and were not independent held-out tests. Initial states were not explicitly matched across checkpoints.
2. Independent final-checkpoint evaluation. We evaluated the final checkpoints without performance-based selection, using separate episodes and matched initial configurations. Results are means ± standard deviations across three training seeds.
SOAR outperforms MP1 across all groups under independent final-checkpoint evaluation. Although this does not eliminate selection bias from the original top-five results, it provides additional evidence for the reported improvements.
See Appendix A.5.2 and Table 13.
Q2
Response: To make the real-robot comparisons more transparent, we report individual outcomes for 40 Tidy-up configurations and 20 dynamic trials per policy, including success indicators and empty-grasp counts.
1. Tidy-up and dynamic success. We expanded the evaluations and added π0.5 (NFE=1) as a control.
2. Empty-grasp variability. In dynamic manipulation, SOAR records 0.35 ± 0.49 empty grasps per trial, compared with 0.95 ± 0.83 and 1.20 ± 1.01 for π0.5 at NFE=10 and NFE=1, respectively (mean ± sample standard deviation).
The individual Tidy-up outcomes and dynamic trial records are provided in Appendix A.4, Tables 9–10. These results support a descriptive improvement in dynamic performance.
Q3
Response: SOAR differs from existing frequency-domain policies by explicitly predicting an observation-conditioned target action spectrum and using it for both action generation and calibration. We clarify this distinction through comparisons with FreqPolicy, FAFM, and FreqFM, together with 20-task component ablations. A fully parameter- and compute-matched one-step baseline with a direct DCT-domain loss remains incomplete due to training constraints.
1. Differences from prior work. Frequency-domain supervision itself is not new. The main distinctions are:
Thus, SOAR focuses on predicting and utilizing temporal structure, rather than only imposing frequency-domain constraints on generated actions.
2. Component evidence. Across 20 Adroit and Meta-World tasks:
Structure supervision (SS) alone provides only a 0.2-point improvement, consistent with its role in training the structure head rather than directly improving task performance. In contrast, MC and AC yield larger gains by incorporating the predicted temporal information into generation and calibration. Removing MC or AC from SOAR reduces success to 70.7% and 70.3%, respectively.