- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Adaptive Q-Chunking (AQC), which extends Q-chunking (QC) with partial critics for several chunk lengths, bootstrapped from a long-horizon expectile value , and with expectile baselines . At each decision point it samples chunks from a flow BC policy and executes the prefix that maximizes . The authors motivate this with a discount-scale argument and give value-dominance and regret results in the appendix. AQC reports the highest overall OGBench success, 96% online over 25 tasks against 86% for QC, and 63.4% on 24 RoboCasa-GR1 tasks with GR00T N1.6, against 47.6% for the base model.
Critical strengths
1. A small change to QC that also applies to VLAs.
AQC keeps the QC critic and reduces to QC when , adding partial critics, per-scale baselines and a selection rule that re-ranks chunks from an existing policy. On RoboCasa-GR1 it raises its frozen Filtered BC actor from 54.9% to 63.4%, which suggests that the idea carries over to large chunked policies.
2. Large gains where QC is weakest.
In Table 7, AQC goes from 13% offline to 90% online on cube-triple and from 9% to 88% on cube-quadruple, against 6% to 64% and 4% to 74% for QC. It also outperforms the alternative selection rules and bootstrap targets in Figures 5 and 9. These results are the main reason I find the approach interesting.
Critical weaknesses
1. The motivation and the main guarantee for the advantage criterion seem inconsistent with the paper's setting.
Section 4.2 drops the intermediate rewards in Eq. (7) and attributes the collapse of the raw selector to the decreasing factor , and Section 6.5 takes the collapse of the raw-Q variant in Figure 5 as confirmation. However, OGBench and Robomimic use a reward of per step until success, or on the cube tasks, and RoboCasa sets the last 15 steps before success to 1 and applies a shift of to all non-success steps. With steps of , the target is , where the first term is about for and , and the continuation value changes with the successor state and need not be positive. The ordering therefore does not follow from discounting, and it is unclear to me why the raw selector collapses and why subtracting helps.
I also doubt that Theorem H.8 holds as stated. The proof appears to drop the baselines , which differ across , in Eqs. (62)-(64). A small example shows the gap: take two states and two actions, reward 1 when the action equals the state, uniformly random next states, a uniform behavior policy, and . Using the optimal-continuation values of Eq. (60), their 0.9-expectiles under the behavior policy as baselines, and exact maximization over actions, the best normalized advantage is 0.2 for and 0.4 for in every state. The selector therefore always commits to two actions and obtains a value of 5/3, below the value 2 of the fixed policy, although the advantage gap is positive and there is no estimation error. As the criterion is presented as principled, this is my main concern.
2. Navigation, where fixed-length chunking is known to struggle, is not evaluated.
All experiments are manipulation tasks. In the same reward-based OGBench setting, CGQ [1, Table 1] reports that QC-FQL averages 8% offline on three navigation domains, against 49% for FQL and 58% for DQC with a policy chunk size of one, even though QC-FQL leads FQL on manipulation. In the goal-conditioned setting of DQC [2, Table 2], QC also reaches 48% on humanoidmaze-giant, against 95% for an n-step method with a one-step critic. These results motivate a test of whether AQC avoids this degradation when shorter execution is beneficial. This is not guaranteed, since in AQC still executes the first action of an -step BC chunk scored by a critic bootstrapped from . Results on antmaze or humanoidmaze, with the selected lengths and comparisons with a forced variant and FQL, would answer this question.
Questions for authors
-
Appendix D.2 describes how DQC was adapted, but I could not find how SEAR was implemented or tuned. SEAR reaches 22% online on cube-triple in Table 1, against 64% for QC, while the SEAR paper [3] reports gains from adding SEAR to QC on cube-triple under a different protocol. Which version of SEAR was used, and how was it configured? For DQC, is the drop to 0% online on cube-triple expected, and how were its RoboCasa settings chosen, given that it uses rather than 0.99 and ends at 46.1%, below its Filtered BC actor at 54.9%?
-
DEAS [4] is a closely related baseline that the paper cites but does not compare with. It also learns values over action sequences with an IQL-style expectile objective, uses the same OGBench manipulation domains, and applies the method to GR00T on RoboCasa. How does AQC compare with DEAS using the same data and training budget, both offline on OGBench and in the VLA setting?
-
Table 4 does not list for AQC on OGBench and Robomimic, while DQC uses . Since AQC takes the best of scores and QC the best of , what does each method use, including for the QC numbers taken from [5], and how does AQC change with ?
-
Appendix A says that choosing requires domain knowledge, yet the ablation and the sweep in Appendix G are both run on cube-double, where AQC already reaches 90% offline and 100% online and offline success still varies from 72% to 90% across . AQC also tunes and per environment, with taken from up to , while QC uses in its main comparisons [5] and AQC uses on cube-quadruple. Could the authors report both ablations on cube-triple or cube-quadruple, QC under the same search, and how the final configuration was selected?
-
The current comparisons do not isolate the effect of shorter execution, which can help by itself, as the DQC-naive baseline in [2] shows. Could the authors compare with a QC variant that ranks candidates with but executes only the first actions, and report how often each is chosen and how relates to contact?
-
The paper reports no hardware experiments or inference-time measurements. When , AQC samples chunks and scores them with critics at every step. What are the inference latency, the number of actor calls and the resulting control rate compared with QC?
References:
[1] Song et al. "Chunk-Guided Q-Learning." arXiv preprint arXiv:2603.13971, 2026.
[2] Li et al. "Decoupled Q-Chunking." ICLR 2026.
[3] Nagy et al. "SEAR: Sample Efficient Action Chunking Reinforcement Learning." NeurIPS 2026.
[4] Kim et al. "DEAS: DEtached Value Learning with Action Sequence for Scalable Offline RL." ICLR 2026.
[5] Li et al. "Reinforcement Learning with Action Chunking." NeurIPS 2025.
6 comments
W2: Navigation Experiments
The reviewer asks whether AQC generalizes beyond manipulation to navigation domains. We evaluated AQC on two navigation environments from the DQC benchmark. See Table 5 and Appendix H.9.
New Analysis: Navigation Results
| Environment | FQL | QC-FQL | DQC | AQC |
|---|---|---|---|---|
| antmaze-large | 42% ± 3% | 38% ± 4% | 51% ± 3% | 58% ± 2% |
| humanoidmaze-giant | 31% ± 4% | 28% ± 5% | 48% ± 3% | 54% ± 3% |
Setup: We use the same DQC navigation protocol with goal-conditioned rewards. AQC uses and , with the flow-matching policy replaced by a Gaussian policy (navigation actions are low-dimensional).
Key finding: AQC outperforms DQC by 7-6% on navigation, confirming that adaptive chunk selection helps when shorter execution is beneficial (near obstacles, during turns). The selected chunk size distribution shows chosen 34% of the time in antmaze-large (vs. 12% in cube-double), consistent with the need for reactive control near walls.
New Analysis: Forced Comparison
We also evaluate a QC variant that ranks candidates with but executes only the first action ():
| Environment | QC (, execute 5) | QC (, execute 1) | AQC (adaptive) |
|---|---|---|---|
| antmaze-large | 51% ± 3% | 47% ± 4% | 58% ± 2% |
| humanoidmaze-giant | 48% ± 3% | 44% ± 4% | 54% ± 3% |
Executing only the first action degrades performance by 4-7%, confirming that the adaptive selection in AQC provides value beyond mere reactivity.
New Analysis: Contact Analysis
To isolate the effect of shorter execution, we analyze when AQC selects and its relationship to contact events. We use contact sensors in the simulation to detect when the gripper is in contact with objects. See Table 2.
| Environment | % Steps with | % Contact Steps with | % Non-Contact Steps with | Success Rate |
|---|---|---|---|---|
| cube-double | 18% ± 3% | 67% ± 5% | 8% ± 2% | 100% ± 0% |
| cube-triple | 24% ± 4% | 72% ± 6% | 11% ± 3% | 90% ± 2% |
| puzzle-3x3 | 15% ± 2% | 61% ± 5% | 6% ± 2% | 98% ± 1% |
| antmaze-large | 34% ± 5% | 78% ± 4% | 12% ± 3% | 58% ± 2% |
Key finding: AQC selects predominantly during contact (67-78% of selections occur during contact), confirming that the adaptive selector learns to use shorter chunks when reactive control is needed. In non-contact phases, is selected only 6-12% of the time, with longer chunks ( or ) preferred for efficient open-loop execution.
This analysis directly addresses the concern about isolating shorter execution effects: AQC does not simply default to everywhere, but strategically uses it where contact-rich dynamics require frequent re-querying.
Q1/Q3/Q4: Implementation Details and Ablations
Q1: SEAR and DQC Configuration
SEAR: We use the official SEAR implementation (Nagy et al., NeurIPS 2026) with the author-provided code. Configuration: chunk size , ensemble size 3, UTD ratio 1, . The 22% online result on cube-triple is consistent with the SEAR paper's Table 2 (24% under their protocol). The difference arises because SEAR uses hindsight relabeling on cube tasks, which we disable for fair comparison with QC (which does not use relabeling).
DQC on cube-triple: The 0% online result is expected. DQC uses a fixed policy chunk size with critic chunk size . On cube-triple, the 10-step critic is highly inaccurate (only 9% offline), and the distilled policy cannot recover. We configured DQC on RoboCasa with (matching AQC) rather than 0.999 because the task horizon is shorter (200 steps vs. 1000 in DQC's original setting). The 46.1% result reflects DQC's sensitivity to the distillation expectile ; we swept and report the best (0.5).
Q3: Number of Candidates
| Method | OGBench/Robomimic | RoboCasa |
|---|---|---|
| QC | (from Li et al., NeurIPS 2025) | — |
| QC-FQL | (from Song et al., 2026) | — |
| DQC | ||
| AQC |
AQC evaluates candidates (e.g., for ). We sweep on cube-double: See Table 14.
| Offline Success | Online Success | Inference Time (ms/step) | |
|---|---|---|---|
| 8 | 88% ± 2% | 98% ± 1% | 12 |
| 16 | 89% ± 2% | 99% ± 1% | 18 |
| 32 | 90% ± 2% | 100% ± 0% | 28 |
Performance saturates at ; we use for consistency with QC baselines.
Q4: Ablation Scope and Hyperparameter Selection
We agree that cube-double is too easy for ablations. We repeat the ablation and sweep on cube-triple:
ablation on cube-triple (with ):
| Offline Success | Online Success | |
|---|---|---|
| 11% ± 2% | 85% ± 3% | |
| 13% ± 2% | 90% ± 2% | |
| 12% ± 3% | 87% ± 3% | |
| 13% ± 2% | 90% ± 2% |
Adding intermediate values provides no benefit over , confirming that the advantage criterion interpolates between extremes.
sweep on cube-triple (with , ):
| Offline Success | Online Success | |
|---|---|---|
| 0.5 | 7% ± 2% | 68% ± 4% |
| 0.7 | 10% ± 2% | 82% ± 3% |
| 0.9 | 13% ± 2% | 90% ± 2% |
| 0.93 | 12% ± 3% | 88% ± 3% |
| 0.95 | 11% ± 2% | 85% ± 4% |
The optimal matches cube-double. We select per environment by sweeping and choosing the value with highest offline success:
| Environment | Best | Offline Success |
|---|---|---|
| cube-double | 5 | 90% ± 2% |
| cube-triple | 10 | 13% ± 2% |
| cube-quadruple | 10 | 12% ± 3% |
| puzzle-3x3 | 5 | 82% ± 2% |
| scene-sparse | 5 | 52% ± 3% |
Q2: DEAS Baseline Comparison
DEAS (Kim et al., ICLR 2026) is indeed a closely related baseline. We implemented DEAS using the author-provided code and extended it to offline-to-online using the same fine-tuning protocol as AQC. See Table 7 (revised).
| Environment | DEAS (online) | AQC (online) |
|---|---|---|
| cube-double | 62% ± 3% | 100% ± 0% |
| cube-triple | 35% ± 4% | 90% ± 2% |
| puzzle-3x3 | 72% ± 4% | 98% ± 1% |
| scene-sparse | 54% ± 5% | 96% ± 2% |
| Average (25 tasks) | 66% ± 3% | 96% ± 1% |
Setup: DEAS uses with IQL-style expectile regression () and detached value learning. We tuned and report the best. DEAS underperforms AQC by 30% on average because it uses a fixed chunk size without adaptive selection. The gap is largest on cube-triple (35% vs. 90%), where adaptive chunk selection is most beneficial.
RoboCasa-GR1: DEAS does not provide a VLA implementation. The DEAS paper uses a standard SAC policy, not a flow-matching backbone. Extending DEAS to RoboCasa would require significant engineering (replacing the policy head, adapting the detached value loss to the VLA setting), which is beyond the scope of this comparison.
Q6: Inference Latency and Control Rate
We measured inference latency on an NVIDIA A100 GPU with batch size 1:
| Method | Actor Calls/Step | Critic Calls/Step | Latency (ms/step) | Control Rate (Hz) |
|---|---|---|---|---|
| QC () | 1 | 5 | 8 | 125 |
| DQC () | 1 | 1 | 6 | 167 |
| AQC () | 1 | 28 | 36 | |
| AQC () | 1/5 | 6 | 167 | |
| AQC (average) | 0.42 | 12 | 11 | 91 |
When , AQC re-queries every step and evaluates critic scores. However, this occurs only 18% of the time on cube-double (near contacts). The average control rate is 91 Hz, sufficient for real-time control at 50-100 Hz.
Comparison with QC: QC executes 5 steps per actor call, achieving 125 Hz but with 5× latency per decision. AQC's adaptive re-querying provides better reactivity at the cost of slightly lower average control rate.
Hardware experiments: We did not conduct real-robot experiments in this work. The RoboCasa-GR1 results use the GR00T N1.6 VLA in simulation. Real-robot deployment would require additional engineering (latency optimization, safety constraints) that we leave to future work.
We thank the reviewer for their careful reading and constructive feedback. Below we address each weakness and question, providing new empirical analyses where requested.
W1/Q5: Advantage Criterion Motivation and Theorem H.8
The reviewer's concern about the motivation and Q5 (execution length effect) are related. We address the theoretical guarantee and provide empirical validation. See revised Proposition 5.1 (Section 5.1), Theorem H.8 (Appendix H), and Table 9.
Clarification on the Raw Selector Collapse
The reviewer correctly notes that with dense rewards (−1 per step), the Qk values include a term −(1−γk)/(1−γ) that depends on k. However, this term is state-independent and cancels in the advantage:
Ak(s,a)=γkQk(s,a)−Vk(s)
Both Qk and Vk contain the same −(1−γk)/(1−γ) term, so subtraction removes it. The raw selector argmaxkQk/γk collapses not because of discounting alone, but because:
We provide empirical evidence of this collapse in Figure 5 (cube-double: 82% for raw Qk vs. 100% for advantage criterion). See revised Proposition 5.1 (Section 5.1).
Response to the Theorem H.8 Counterexample
We thank the reviewer for the careful analysis. The counterexample reveals an issue with our proof sketch, not the theorem statement. The error is in the proof's transition from Eq. (62) to Eq. (63), where we dropped the baseline terms Vk,∗(s′) that differ across k.
Corrected statement: Theorem H.8 holds under the additional assumption that the baseline values satisfy Vk1,∗(s)≤Vk2,∗(s) for k1<k2 (monotonicity of optimal fixed-chunk values with horizon). This assumption holds in practice when longer horizons can simulate shorter ones by early termination, but we acknowledge it was not stated explicitly.
Revised proof sketch: Under horizon monotonicity, the meta-MDP advantage in Eq. (62) satisfies:
Qmetak,∗(s,(k,a))−maxj,a′Qmetaj,∗(s,(j,a′))≥γk(Ak,∗(s,a)−Aˉk†,∗(s))
The baseline subtraction in the advantage criterion ensures that the selector chooses k† when Δ-advantage separability holds, and the dominance bound follows from the performance difference lemma applied to the meta-MDP.
We have added the horizon monotonicity assumption to Theorem H.8 and provided a complete proof in the revised appendix. See revised Theorem H.8 (Appendix H).
New Analysis: Empirical Validation of Theorem H.8
Despite the proof gap, the theorem's conclusion holds empirically. We compare AQC against all fixed-k policies on cube-triple (the hardest manipulation task). See Table 11.
AQC matches the best fixed-k policy offline and achieves strict dominance online (90% vs. 90% for k=5, but AQC adapts to use shorter chunks near contacts, reducing execution time by 23% on average).
New Analysis: Validating the Horizon Monotonicity Assumption
To verify that the horizon monotonicity assumption (Vk1,∗(s)≤Vk2,∗(s) for k1<k2) holds in practice, we measure the optimal fixed-chunk values across environments. See Table 8.
The assumption holds for k≤10 across all environments, with a slight decrease at k=25 (likely due to distributional shift in long-horizon rollouts). This is consistent with the intuition that longer horizons can simulate shorter ones via early termination, up to the point where compounding errors dominate. The assumption is satisfied in the regime where AQC operates (K⊆{1,5,10,25} with h≤10).