acceptodds
Under review as a conference paper at ICLR 2027

LoopQuant: Loop-Aware Post-Training Quantization for Vision-Language-Action Models

Abstract

Vision-language-action (VLA) models achieve strong manipulation performance, yet their substantial memory footprint and latency hinder real-time execution on robot accelerators. In practice, a VLA operates across two nested loops: an inner loop over denoising steps and an outer control loop that initiates each re-planning request from a new observation. Under W4A8 post-training quantization (PTQ), static four-bit weights cause reconstruction errors to recur at every denoising step and compound across consecutive requests through executed actions. Meanwhile, dynamic activations shift across denoising steps, and we find that their channel peaks drift across observations, requiring offline calibration thresholds to cover both loops. We propose **LoopQuant**, a training-free W4A8 PTQ framework tailored to these nested loops. **Request-aware multi-observation activation extremal calibration** (RMAEC) records per-channel token percentiles at every denoising step, aggregates their maxima within each request and across diverse observations, and freezes the resulting scales. An extreme-value model predicts that these scales grow sub-linearly with the number of observations. To reduce weight error, **exact non-uniform codebook construction** aggregates repeated weight values to make dynamic programming tractable at layer scale, yielding globally optimal levels for fixed row scales. At runtime, **fused packed execution** decodes four-bit weights in a Triton kernel for half-precision matrix multiplication and executes the expert core with a single launch per denoising step. Averaged across four LIBERO suites, LoopQuant achieves closed-loop success on 0.5 and on GR00T-N1.5, outperforming the strongest quantized baseline by and percentage points, respectively. The packed 0.5 runtime serves requests faster than the unquantized BF16 model, requiring only of its resident GPU memory.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Quantization, Model Compression, Vision-Language-Action Model

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_phKYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free W4A8 PTQ framework with nested loop format that can be applied for VLA models, named as LoopQuant. LoopQuant adapted an activation calibration across multiple observations with denoising steps and coodbooks. LoopQuant was validated in LIBERO suites, with improvement over other methods on GR00T-N1.5, although a gap to FP16 remains. The authors also reported a 1.88x speedup with a GPU memory reduction to 59% on an RTX 5090 setting.

Critical strengths
  1. This paper's proposed method was evaluated in a closed-loop task environment, where numerical reconstruction error itself doesn't show whether a quantized policy preserves robot behavior. The authors tested their methods in two models environments and the results backed their hypothesis and system design.
  2. The calibration experiments are good. Especially in the Table 3, RMAEC outperforms other calibration settings with large gaps.
  3. The authors also showed the real-world implementation of GPU memory use case, where the ablations and detailed appendix experiments helped readers to understand the reproducibility and limits of this evaluation.
Critical weaknesses
  1. Although Tables 2 and 3 provide useful controlled experiments, Table 1 does not establish whether LoopQuant’s advantage over competing methods remains under matched calibration data and denoising-step coverage.
  2. The benefit of exact codebook optimization over a strong approximate solver is not established. The comparison with uniform grids demonstrates the value of fitted non-uniform levels, but does not isolate the value of finding the exact optimum.
  3. Although the authors provided numerous ablation studies and comparison experiments, still, the main evaluation supports a narrower scope than broad generalizability across VLAs or deployment settings. It includes two models, one simulation benchmark, and one evaluation seed per configuration. Hardware measurements are limited to a desktop GPU.
  4. Several findings would benefit from deeper interpretation. For example, single-frame calibration substantially reduces clipping without improving task success, and the codebook contributes differently across the two models.
  5. In case of the writing, the conclusion section mainly focuses the components and results. It would be good if the author can restate/introduce the problem, and higher conclusion (e.g. what these findings teach us about quantizing iterative robot policies)
Questions for authors
  1. How does LoopQuant compare with a BF16 model using equivalent dispatch optimization? Could you separate the latency effects of packed quantization and single-launch execution?
  2. Would the accuracy advantage remain if compatible baselines used the same 32 calibration observations and denoising-step coverage? I acknowledge the controls in Tables 2 and 3, but a matched comparison with competing methods would strengthen Table 1.
  3. Why does the codebook provide little additional benefit after RMAEC on π0.5 but a larger improvement on GR00T? Similarly, what explains the weak relationship between the clipping reduction from single-frame calibration and task success?
  4. Could you provide evidence from another architecture, such as SmolVLA, or a second benchmark? Either would help clarify how well the findings transfer beyond the current setting.
2 comments
AuthorsOpenReview ↗

Response to Reviewer phKY (1/2)

We thank the reviewer for the constructive comments and for the positive remarks on the closed-loop evaluation and the calibration experiments.

Q1. "How does LoopQuant compare with a BF16 model using equivalent dispatch optimization? Could you separate the latency effects of packed quantization and single-launch execution?"

Answer. Thank you for raising this; another reviewer asked the same question, so we measured BF16 with equivalent dispatch. With the same single-launch core and graph capture on both sides, LoopQuant is 1.87× faster than BF16 (86.67 vs. 162.01 ms per request; Table R1). Since dispatch is identical in this row, the 1.87× comes from packed quantization. In the Table 4 row, by contrast, only LoopQuant used the single-launch core, so its 1.88× mixes both effects. We did not measure the single-launch core in isolation.

Table R1. Request latency under the two dispatch settings (new entries: mean ± std over 30 requests after 5 warm-up requests).

SettingBF16LoopQuantSpeedup
Table 4 (single-launch core for LoopQuant only, no graph capture)290 ms154 ms1.88×
Same single-launch core and graph capture on both sides (new)162.01 ± 1.49 ms86.67 ± 0.72 ms1.87×

Where the quantization speedup comes from. In the new setting we also varied only the decoding path. Decoding into a buffer before the GEMM already gives 93.61 ms (1.73×), and fusing the decoding steps brings it to 86.67 ms (−6.94 ms); both paths compute the same decoded weights and match the reference bit for bit (App. C.1). Apart from latency, 4-bit storage lowers resident memory to 4.100 GiB (0.59× of BF16). We will report both settings in Table 4 and make the abstract's speedup refer to the matched setting.

W1 / Q2. "Would the accuracy advantage remain if compatible baselines used the same 32 calibration observations and denoising-step coverage?"

Answer. Thank you for this question. Table 2 of the paper provides this comparison: its base row is the QuantVLA baseline of Table 1 (82.4% / 72.8%), and all other W4A8 settings are fixed while only the calibration changes. Given the same 32 observations and full step coverage, QuantVLA rises to 97.1% on π0.5 and 81.0% on GR00T-N1.5, yet LoopQuant still leads it by 0.2 pp and 3.3 pp, respectively (Table R2). The advantage thus remains under matched calibration, although much of it comes from the calibration itself, which is the component we propose.

Table R2. Effect of RMAEC with all other W4A8 settings fixed (from Table 2).

Changeπ0.5GR00T-N1.5
Uniform W4: first-forward → RMAEC82.4% → 97.1%72.8% → 81.0%
Exact codebook: first-forward → RMAEC90.8% → 97.3%80.5% → 84.3%

For DuQuant, however, a matched version cannot be built without changing the method. Its calibration data only locate outlier channels for its rotations (DuQuant, App. E.4), and its activation quantizer is per-token, asymmetric, and recomputed from each input, so plugging in RMAEC's per-channel static thresholds would change granularity, symmetry, and dynamic vs. static scales at once. Table 2 avoids this, as both arms share the same static per-channel quantizer.

W2. "The benefit of exact codebook optimization over a strong approximate solver is not established."

Answer. Thank you for this suggestion. We added weighted Lloyd, a strong approximate solver for the same objective. Exact DP lowers the total reconstruction objective by 0.374% relative to Lloyd and is no worse on any of the 252 projections (Table R3). In closed loop it is 0.3 pp higher without RMAEC and 0.2 pp higher with RMAEC (Table R4). Exact DP thus stays ahead of a strong approximate solver at both levels, and exactness further gives a global-optimality guarantee and deterministic solving without initialization or restarts.

Table R3. Weighted reconstruction objective of Sec. 3.2 over the 252 projections of π0.5 (same row scales, activation second moments, 16 levels; lower is better).

SolverTotal objectiveGap to DP
Weighted Lloyd600345.48+0.374%
Exact DP598106.790

Table R4. Closed-loop W4A8 success on π0.5 (Lloyd is new; the rest is from Table 2).

Activation calibrationUniform W4Weighted LloydExact DPDP − Lloyd
First-forward82.4%90.5%90.8%+0.3 pp
RMAEC97.1%97.1%97.3%+0.2 pp
AuthorsOpenReview ↗

Response to Reviewer phKY (2/2)

Q3 (first part). "Why does the codebook provide little additional benefit after RMAEC on π0.5 but a larger improvement on GR00T?"

Answer. The difference lies in how much headroom RMAEC leaves, rather than in the codebook itself. On π0.5, uniform W4 + RMAEC already matches BF16, so there is little left to recover; on GR00T-N1.5 a 6 pp gap remains, and the codebook recovers about half of it (Table R5). Consistent with this, the codebook's reconstruction gain is similar on both models (median per-projection reduction of 39.3% vs. 38.5%, Sec. 4.3), and on π0.5 it still beats uniform W4 + RMAEC on all three seeds (Table R5 in our response to Reviewer gt7p).

Table R5. Headroom after RMAEC.

ModelUniform W4 + RMAEC+ exact codebookUnquantizedHeadroom
π0.597.1%97.3% (+0.2 pp)BF16 97.1%0 pp
GR00T-N1.581.0%84.3% (+3.3 pp)FP16 87.0%6 pp

W4 / Q3 (second part). "What explains the weak relationship between the clipping reduction from single-frame calibration and task success?"

Answer. Single-frame calibration removes most clipping on average, but not the clipping that matters. It covers the denoising steps of one request, while activations from other observations still exceed its thresholds, and success only rises once the thresholds also cover those observations, as in RMAEC (Table R6).

Table R6. Clipped mass (32 unseen episodes, 960 requests) and success (800 episodes) on π0.5.

CalibrationAction-expert clipped massSuccess
First-forward baseline29.78%82.4%
Single-frame envelope6.98%82.5%
RMAEC2.14%97.1%

Two further mechanisms plausibly add to this mismatch: errors concentrated in particular channels or moments, and closed-loop feedback, in which one poor action changes later observations.

W3 / Q4. "Could you provide evidence from another architecture, such as SmolVLA, or a second benchmark?"

Answer. Thank you for suggesting SmolVLA. We evaluated it on the four LIBERO suites against ZSQ, the strongest baseline on π0.5 in Table 1. LoopQuant reaches 50.25%, against 67.63% for FP16 and 22.13% for ZSQ, and is above ZSQ on every suite (Table R7). In addition, the held-out-task test in the appendix shows transfer to tasks unseen in calibration: 94.4% on LIBERO tasks 8–9 vs. 98.0% on tasks 0–7.

Table R7. SmolVLA success on LIBERO (W4A8 for ZSQ and LoopQuant).

SmolVLASpatialObjectGoal10Avg.
FP1663.00%73.50%79.00%55.00%67.63%
ZSQ28.00%13.00%20.00%27.50%22.13%
LoopQuant52.50%62.50%37.50%48.50%50.25%

The gap to FP16 is concentrated on LIBERO-Goal, while on the other suites it is 6.5–11.0 pp. LoopQuant applies to SmolVLA because it shares the dual-loop flow: a cached vision-language prefix and a flow-matching action expert called repeatedly per request (SmolVLA, Sec. 3.1). We will add these results to Sec. 4.

W5. "... the conclusion section mainly focuses the components and results ... restate/introduce the problem, and higher conclusion ..."

Answer. Thank you for this suggestion. We rewrote Sec. 5 so that it opens with the problem and ends with what the findings teach about quantizing iterative robot policies:

In this work, we traced W4A8 quantization errors through the two nested loops of VLA inference, where the same quantized weights and activation thresholds serve every denoising step and every request. We showed that channel activation peaks drift across successive requests, and that an extreme-value model captures the sub-linear threshold growth with calibration frames. Grounded in this analysis, we introduced LoopQuant, a W4A8 PTQ framework tailored to these loops. For activations, RMAEC aggregates diverse observations while preserving per-request extrema, with ablations confirming that both broad coverage and extremal aggregation are critical to closed-loop success. For weights, an exact non-uniform codebook determines globally optimal four-bit levels under fixed row scales. Evaluations on π0.5 and GR00T-N1.5 show that LoopQuant nearly matches the closed-loop success of unquantized models. At runtime, packed storage reduces resident weight memory and weight traffic, so that under the same single-launch execution LoopQuant serves requests 1.87× faster than the unquantized BF16 model. Our work stresses the need to calibrate quantized robot policies across both loops and to judge them by closed-loop task success.

Reviewer_gt7pOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies W4A8 post-training quantization of diffusion/flow-based VLA policies (π0.5, GR00T-N1.5), framing inference as two nested loops: denoising steps within a request and successive requests across observations. It observes (Fig. 1b) that per-channel activation peaks drift across requests, so static thresholds calibrated on a single forward pass clip heavily. LoopQuant has three parts: (i) RMAEC, a static activation calibration that takes the 99.9th token percentile per denoising step, the max over steps, then the max over M=32 calibration frames from diverse tasks. (ii) an exact non-uniform 16-level weight codebook per projection, solved by SMAWK-accelerated dynamic programming after aggregating the few thousand distinct normalized weight values, with per-row scales fixed in advance. (iii) a packed INT4 runtime with a fused Triton decode kernel and one launch of the expert core per denoising step. On the four LIBERO suites (800 episodes/config, single seed) it reports 97.3% average success on π0.5 (BF16: 97.1%) and 84.3% on GR00T-N1.5 (FP16: 87.0%), ahead of ten quantized baselines, with 1.88× lower request latency and 0.59× resident memory versus BF16 on an RTX 5090.

Critical strengths
  • Organizing VLA quantization error around the two nested loops (denoising steps within a request, observations across requests) is a clear framing, and the design decisions follow from it.
  • The paper identifies and fixes a calibration problem specific to VLAs. Prior VLA quantization work inherits the LLM recipe and sets static activation thresholds from a small buffer or the first forward pass. This paper shows that activation peaks in the diffusion expert vary both across denoising steps and across observations, so static thresholds need the per-request extremal envelope maximized across observations, and that a pooled percentile does not work (Table 3). This finding has no counterpart in LLM quantization and diffusion-model quantization only handles the timestep dimension. It is simple, effective (this one change lifts the base from 82.4 to 97.1 in Table 2), and directly adoptable by other methods.
  • The paper builds a deployable runtime (Triton decode kernel plus single-launch dispatch) verified bit-exact against the simulation build. The 0.59× memory reduction is a real gain.
Critical weaknesses
  1. [W1 ]The 1.88× speedup is not attributable to quantization (Table 4, Table 9, pp. 9, 21, Fig. 1c). The entire saving comes from the per-step time (20.3 → 5.74 ms), while the prefix is 31.9 ms slower than BF16. The single-launch core is a CUDA-graph-style dispatch optimization that applies equally to the BF16 expert. Moreover, the runtime decodes to FP16 and runs an FP16 GEMM, so quantization by itself cannot make a step faster. Without a "BF16 + single-launch core" row, the abstract's "1.88× faster than BF16" does not hold.
  2. [W2 ]Calibration data is confounded with method (Table 1 caption, §4.1, p. 7, and Table 2, p. 8). Baselines keep their native calibration while LoopQuant calibrates on 32 task-stratified frames. Table 2 shows RMAEC alone lifts the QuantVLA base from 82.4 to 97.1, so almost the whole gap to the baselines is closed by the calibration protocol, which any W4A8 baseline could adopt unchanged. At minimum DuQuant + RMAEC is needed (LoopQuant already uses DuQuant's rotations, and DuQuant is at 94.6/80.3).
  3. [W3] All results use a single seed (Table 1, C.5), while several margins in Table 1 are around 1 pp, comparable to the authors' own confidence half-width. Multi-seed results or paired tests for LoopQuant, the strongest baseline, and the unquantized model would help.
Questions for authors
  1. [Q1] What is the request latency of the BF16 model with the same single-launch (graph-captured) expert core? Please add this row to Table 4, since it decides whether the speedup claim stands.
  2. [Q2] What success does DuQuant W4A8 reach when calibrated with RMAEC's 32 frames and extremal aggregation? More generally, which baselines were given access to the 32 calibration frames at all?
  3. [Q3] What does the pooled maximum over the same 32×10 calls give in Table 3?
  4. [Q4] Can you report at least 3 seeds for LoopQuant, the strongest baseline, and the unquantized model, or run a significance test on the per-episode paired data you already have?
3 comments
AuthorsOpenReview ↗

Response to Reviewer gt7p (1/2)

We thank the reviewer for the careful reading and for noting that the calibration finding has no counterpart in LLM quantization and can be adopted directly by other methods.

W1 / Q1. "The 1.88× speedup is not attributable to quantization ... What is the request latency of the BF16 model with the same single-launch (graph-captured) expert core? Please add this row to Table 4."

Answer to Q1. Thank you for this request; we have measured this row. With the same single-launch expert core and graph capture, BF16 takes 162.01 ms per request, while LoopQuant takes 86.67 ms in the identical setting (Table R1). LoopQuant thus stays 1.87× faster once BF16 receives the same dispatch optimization, so the speedup does not come from the single-launch core.

Table R1. Request latency in the submission setting and in the requested matched setting (new entries: mean ± std over 30 requests after 5 warm-up requests).

SettingBF16LoopQuantSpeedup
Table 4 (single-launch core for LoopQuant only, no graph capture)290 ms154 ms1.88×
Requested row: same single-launch core and graph capture on both sides162.01 ± 1.49 ms86.67 ± 0.72 ms1.87×

We will add this row to Table 4 and make the abstract's speedup refer to the matched setting.

Answer to W1.

  1. "The single-launch core applies equally to the BF16 expert." We agree, and this is why we added the matched row: BF16 now receives the same optimization, so dispatch is held constant and does not contribute to the 1.87×.
  2. "The entire saving comes from the per-step time, while the prefix is 31.9 ms slower." This is correct, and it is what we expect from 4-bit weight storage (App. C.4: prefix 63.3 → 95.2 ms, per step 20.3 → 5.74 ms). The prefix processes several hundred tokens in one pass and is compute-bound, so 4-bit storage does not reduce its GEMM work, and decoding adds overhead. Each denoising step, in contrast, processes only a few action tokens and is bound by weight reads, which is where 4-bit indices pay off. Since the expert runs ten times per request and the prefix once, the per-step saving outweighs the slower prefix.
  3. "The runtime decodes to FP16 and runs an FP16 GEMM, so quantization by itself cannot make a step faster." This would hold if the step were compute-bound, but our measurements show it is bound by weight reads. Keeping the single-launch core and graph capture fixed, we varied only the decoding path (Table R2).

Table R2. Request latency by decoding path (same setting as the requested row).

How quantized weights reach the half-precision GEMMLatency (ms)Speedup vs. BF16
BF16 weights, no quantization (reference)162.01 ± 1.491.00×
LoopQuant, decoded (unfused) into a buffer, then the GEMM93.61 ± 0.421.73×
LoopQuant, decoded (fused) into a buffer, then the GEMM86.67 ± 0.721.87×

Even when the decoded weights are written to a buffer and passed to a separate half-precision GEMM, the request is already 1.73× faster than BF16, and fusing the decoding steps brings this to 1.87× (−6.94 ms). The weights are read from memory as 4-bit indices, and the decoded half-precision values occupy only a bounded buffer rather than full weight matrices. Both quantized rows compute the decoded weights W^rc=αrγq(r,c)\widehat{W}_{rc}=\alpha_r\gamma_{q(r,c)} of Eq. (6) and match the reference decoding bit for bit (App. C.1).

Additional result. Going one step further, we implemented an in-GEMM decoding kernel that decodes the 4-bit indices inside the GEMM tiles. It removes the decode buffer and brings the request down to 84.71 ± 0.53 ms in the same setting (1.91× over BF16), with the same decoded weights bit for bit. We will add it to the paper.

AuthorsOpenReview ↗

Response to Reviewer gt7p (2/2)

W2 / Q2. "Calibration data is confounded with method ... What success does DuQuant W4A8 reach when calibrated with RMAEC's 32 frames and extremal aggregation? ... which baselines were given access to the 32 calibration frames at all?"

Answer to Q2 (calibration frames). To clarify the setup first, no baseline in Table 1 was calibrated on the 32 frames. As stated in the Table 1 caption, each baseline keeps its native activation quantizer and calibration settings, so Table 1 compares complete quantization schemes.

Answer to Q2 (DuQuant + RMAEC). We understand the motivation for this control, but RMAEC cannot be plugged into DuQuant without replacing its activation quantizer. DuQuant uses calibration data only to identify outlier channels for its rotations and zigzag permutation (DuQuant, App. E.4). Its activation quantizer is per-token and asymmetric, recomputes scale and zero point from the current input at every forward pass, and uses a fixed clipping ratio of 0.9 (DuQuant, Sec. 3.1; official quantizer). RMAEC's per-channel symmetric thresholds are frozen at inference, so inserting them would change granularity, symmetry, and dynamic vs. static scales at once, and the result would no longer be DuQuant. For the same reason, LoopQuant adopts DuQuant's rotations but not its online activation quantizer.

Answer to W2. We agree that most of the gain comes from the calibration; this is in fact our point, since RMAEC is the calibration we propose. To separate it from the rest of the method, Table 2 (summarized in Table R3) keeps the same static, per-channel, symmetric 8-bit quantizer and changes only which observations set the thresholds and how per-call statistics are aggregated.

Table R3. Effect of RMAEC with all other W4A8 settings fixed (from Table 2).

Changeπ0.5GR00T-N1.5
Uniform W4: first-forward → RMAEC82.4% → 97.1% (+14.7 pp)72.8% → 81.0% (+8.2 pp)
Exact codebook: first-forward → RMAEC90.8% → 97.3% (+6.5 pp)80.5% → 84.3% (+3.8 pp)

RMAEC gives a large gain under both weight formats, and the two arms separate the calibration contribution from the codebook contribution.

Q3. "What does the pooled maximum over the same 32×10 calls give in Table 3?"

Answer. Thank you for suggesting this control. We ran it on π0.5 with uniform W4 and static A8, taking the per-channel maximum over the raw activations of the same 32×10 calls. It reaches 96.8%, close to RMAEC (97.1%) and far above the pooled 99.9th percentile (85.0%), as shown in Table R4.

Table R4. Aggregation rules over the same 32×10 calls (π0.5, uniform W4, static A8).

Threshold ruleSuccess
Pooled 99.9th percentile (Table 3)85.0%
Pooled raw maximum (new)96.8%
RMAEC: per-call 99.9th percentile, then maximum97.1%

We use raw activations because taking per-call percentiles before the maximum would reproduce RMAEC. Under the raw maximum, isolated outliers set the threshold, but with 8-bit activations the larger step still leaves fine resolution, so the loss is small. What matters most is thus whether the threshold keeps the extreme range of individual calls; filtering outliers within each call adds a small further gain. We will add this row to Table 3.

W3 / Q4. "Several margins in Table 1 are around 1 pp ... Can you report at least 3 seeds for LoopQuant, the strongest baseline, and the unquantized model, or run a significance test ...?"

Answer. Thank you for this suggestion. We ran three evaluation seeds (7, 17, 27; 800 episodes each) on π0.5 for the three models you asked for, plus uniform W4 + RMAEC (Table R5). LoopQuant beats the strongest baseline ZSQ by about 4 pp on every seed and stays within 0.1 pp of BF16 on average.

Table R5. Three-seed success on π0.5.

ConfigurationSeed 7Seed 17Seed 27Mean ± std
BF16 reference96.375%97.250%97.375%97.000% ± 0.545 pp
ZSQ (strongest baseline)92.500%93.250%93.375%93.042% ± 0.473 pp
Uniform W4 + RMAEC96.125%96.125%96.875%96.375% ± 0.433 pp
LoopQuant96.500%97.500%97.250%97.083% ± 0.520 pp
  • LoopQuant vs. ZSQ. LoopQuant is higher on every seed: +4.000, +4.250, +3.875 pp.
  • LoopQuant vs. BF16. LoopQuant averages 97.083%, close to BF16 at 97.000%.
  • Exact codebook vs. uniform W4. With the same static A8 activations, the codebook is higher on all three seeds (mean +0.708 pp).

Thank you for the detailed rebuttal and additional experiments. The responses address my main concerns, and I appreciate the authors' efforts. I am happy to raise my score.

Reviewer_kCjPOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduce LoopQuant, a training-free W4A8 post-training quantization framework for vision-language-action models. The key insight is that VLA inference contains two nested loops: denoising steps within each action request and repeated requests driven by new observations. LoopQuant addresses the resulting quantization errors through three components: RMAEC, which calibrates static activation thresholds across both denoising steps and diverse observations; an exact non-uniform 4-bit codebook, optimized by dynamic programming for fixed row scales; and a fused packed runtime that reduces weight memory and kernel-launch overhead

Critical strengths
  • The analysis and motivation are reasonable. The dual-loop analysis leads to a concrete calibration method rather than serving only as motivation. The calibration ablation is compelling: covering all denoising steps on one observation is insufficient, while using diverse observations with the request-wise extremal envelope recovers a large amount of closed-loop success.

  • Strong closed-loop performance with practical memory and latency evidence. The authors provided an exact dynamic-programming construction for the shared 16-level codebook under fixed row scales, verifies the accelerated solver against the reference, and provided a packed runtime with bit-exact decoding relative to its reference path. The combination of algorithmic improvements and an efficient runtime strengthens the practical significance of the work.

Critical weaknesses
  • With the performance and deployment evidence being said, all evaluation are conducted in the LIBERO simulator, and the runtime measurements are performed on an RTX 5090. Discussions on how the accuracy and latency advantages will generalize or not to other platforms or even physical robots with resource-limited accelerators would be beneficial.
Questions for authors

Could the authors provide a latency ablation separating the effects of packed weight decoding, fused execution, and single-launch expert dispatch? In particular, how would the BF16 baseline perform with an equivalent single-launch execution strategy?

This comparison would clarify the net acceleration attributable to quantization rather than differences in runtime implementation.

2 comments

We thank the reviewer for the positive assessment of the dual-loop analysis, the calibration ablation, and the packed runtime, and respond to the two comments below.

Q1. "Could the authors provide a latency ablation separating the effects of packed weight decoding, fused execution, and single-launch expert dispatch? ... how would the BF16 baseline perform with an equivalent single-launch execution strategy?"

Answer. Thank you for this suggestion, which helps show where the speedup comes from. We re-measured with BF16 given the same single-launch expert core as LoopQuant and with graph capture on both sides. In this matched setting, BF16 takes 162.01 ms per request and LoopQuant 86.67 ms, so LoopQuant remains 1.87× faster. Table R1 breaks this down along the three factors you list:

  • Single-launch expert dispatch. Every row, BF16 included, uses the same single-launch core, which submits the expert Transformer core once per denoising step. Since it is held constant, it contributes nothing to the 1.87×; we did not measure its effect in isolation.
  • Packed weight decoding. Even the unfused packed row, which decodes 4-bit weights into a temporary buffer before the GEMM, is already 1.73× faster than BF16.
  • Fused execution. Fusing unpacking, codebook lookup, and row scaling into one pass saves a further 6.94 ms (7.41%).

Table R1. Request latency with the same single-launch core and graph capture in every row (mean ± std over 30 requests after 5 warm-up requests).

ConfigurationWeight decoding pathLatency (ms)Speedup vs. BF16
BF16BF16 weights used directly162.01 ± 1.491.00×
LoopQuant, unfusedunpack, lookup, row scaling as separate steps into a buffer, then GEMM93.61 ± 0.421.73×
LoopQuant, fusedthe same steps fused into one pass into a buffer, then GEMM86.67 ± 0.721.87×

The two quantized rows share the same operator optimizations and differ only in whether decoding is fused. Both compute the decoded weights W^rc=αrγq(r,c)\widehat{W}_{rc}=\alpha_r\gamma_{q(r,c)} of Eq. (6) and match the reference decoding bit for bit (App. C.1).

Relation to the submission. Table 4 in the submission was measured without graph capture: BF16 takes 0.290 s and LoopQuant 0.154 s (1.88×), with 4.100 GiB resident memory (0.59× of BF16). We will add Table R1 to the paper, state the execution setting of each measurement, and make the abstract's speedup refer to the matched setting.

Additional result. While preparing this response, we also implemented an in-GEMM decoding kernel that decodes the 4-bit indices inside the GEMM tiles. It removes the decode buffer and brings the request down to 84.71 ± 0.53 ms in the same setting (1.91× over BF16), with the same decoded weights bit for bit. We will add it to the paper.

W1. "... all evaluation are conducted in the LIBERO simulator, and the runtime measurements are performed on an RTX 5090. Discussions on how the accuracy and latency advantages will generalize or not to other platforms or even physical robots ..."

Answer. We agree that this matters for deployment. Beyond simulation, we ran an experiment on a physical SO-ARM101 arm with an RTX 4090. On the real arm, the behavior of the FP16 and quantized policies differed somewhat from what we observe in simulation. We attribute this mainly to three sources: the latency of the real arm does not match that of the simulator, data transmission adds further delay, and the physical precision of the arm differs from the simulated one. In practice, real-robot deployment usually also requires collecting data on the robot for a second calibration.

What carries over. The memory saving (4.100 GiB, 0.59× of BF16) comes from 4-bit storage and holds on any platform with a matching decoder, which matters directly for memory-constrained accelerators. The latency ratio, in contrast, depends on device bandwidth, decoding cost, and half-precision GEMM throughput, so the 1.88× is an RTX 5090 measurement and must be re-measured on each device.

Calibration on a real robot. On a real robot, camera observations, robot state, and actuation all shift, which changes how well the calibration samples cover deployment. Our real-arm observation is consistent with this, so for deployment we would recalibrate on frames collected from the target robot; RMAEC only needs calibration frames, so the procedure is unchanged. Inside LIBERO, the held-out-task test in the appendix gives a first indication of transfer: with calibration frames from tasks 0–7 only, LoopQuant reaches 94.4% on the held-out tasks 8 and 9, against 98.0% on tasks 0–7. We will add this discussion to the limitations.

Thanks to the authors for the reply. It was quite informative and most of my concerns are addressed. I will maintain my original positive position. I hope the additional discussion could be added in future versions to make the submission more comprehensive and complete.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary for Submission 10602

Dear Area Chair and Reviewers,

Thank you for overseeing and evaluating our submission. All three reviewers give positive opinions on our work, and they credit the dual-loop analysis, the calibration finding, and the closed-loop evaluation. Below we summarize the main concerns and how we addressed them.

Key Concerns Raised by Reviewers and Our Responses

Common concern: source of the speedup (kCjP, gt7p, phKY)

All three reviewers are concerned with how much of the speedup comes from quantization and how much from the single-launch execution. In the submission, only LoopQuant used the single-launch expert core, so we re-measured BF16 with the same core and execution setting. LoopQuant takes 86.67 ms per request and BF16 takes 162.01 ms, so LoopQuant remains 1.87× faster, and this speedup comes from packed 4-bit weights. Unfused decoding already gives 1.73×, fused decoding gives 1.87×, and an in-GEMM decoding kernel reaches 1.91×, all matching the reference decoding bit for bit. We will add the matched row to Table 4 and make the abstract refer to this setting.

kCjP (3, keeps positive score)

We thank Reviewer kCjP for engaging with our response and for keeping a positive assessment.

  • Latency ablation. See the common concern above.
  • Other platforms and physical robots. We report an experiment on a physical SO-ARM101 arm with an RTX 4090 and explain why its behavior differs from simulation. The memory saving (0.59× of BF16) holds on any platform with a matching decoder, while the latency ratio must be re-measured per device. On a real robot we would recalibrate on frames from that robot, and the RMAEC procedure stays the same. We will add this discussion to the paper as the reviewer suggested.

gt7p (3 → 4)

We thank Reviewer gt7p for the careful review and for raising the score after the rebuttal.

  • Speedup attribution (W1/Q1). We added the requested "BF16 + single-launch core" row (1.87×). The slower prefix and faster per-step time follow from the compute-bound prefix and the weight-read-bound denoising steps.
  • Calibration and method (W2/Q2). No Table 1 baseline used our 32 frames. DuQuant's per-token dynamic quantizer cannot take RMAEC's static per-channel thresholds without becoming a different method. Table 2 isolates RMAEC under a fixed quantizer (+14.7/+8.2 pp under uniform W4, +6.5/+3.8 pp under the exact codebook); RMAEC is the calibration we propose, so this gain is part of our contribution.
  • Pooled maximum (Q3). The new control reaches 96.8%, against 97.1% for RMAEC and 85.0% for the pooled percentile.
  • Seeds (W3/Q4). Over three seeds on π0.5, LoopQuant averages 97.08%, against 97.00% for BF16 and 93.04% for ZSQ, and beats ZSQ on every seed by 3.9 to 4.3 pp.

phKY (3, "most of my concerns were resolved")

We thank Reviewer phKY for the constructive suggestions and for acknowledging that most concerns were resolved.

  • Matched dispatch latency (Q1). See the common concern above.
  • Matched calibration (W1/Q2). With the same 32 observations and full step coverage, QuantVLA rises to 97.1%/81.0%, and LoopQuant still leads by 0.2/3.3 pp.
  • Exact DP vs. approximate solver (W2). Against weighted Lloyd, exact DP lowers the objective by 0.374%, is no worse on any of the 252 projections, and is 0.2 to 0.3 pp higher in closed loop.
  • Interpretation (W4/Q3). The codebook's gain depends on the headroom left after RMAEC (0 pp on π0.5, 6 pp on GR00T-N1.5). Single-frame calibration lowers average clipping but leaves the cross-observation clipping that limits success.
  • Broader evidence (W3/Q4). On SmolVLA, LoopQuant reaches 50.25% against 22.13% for ZSQ and leads on every LIBERO suite; a held-out-task test reaches 94.4% on tasks unseen in calibration.
  • Conclusion (W5). We rewrote Sec. 5 to open with the problem and close with what the findings teach about quantizing iterative robot policies.

Conclusion

Our rebuttal answers each concern with new measurements: matched-execution latency, three-seed results, a pooled-maximum control, a weighted-Lloyd comparison, SmolVLA results, and a real-arm discussion. All will be added to the revised manuscript. We are grateful to all three reviewers for their timely replies during the discussion phase.

We hope these revisions clearly convey LoopQuant's contributions to the ICLR community, and we thank you all for your time and effort in evaluating our work.

Best regards,

Submission10602 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.