- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies which quantization-error directions should receive a limited low-rank correction budget to preserve a full-precision VLM teacher's answers. OQER weights the residual of a fixed quantized checkpoint by both input-activation and output-gradient moments, then constructs a training-free correction using truncated SVD. On LLaVA-1.5-7B at 3 bits and rank 64, net recovery on POPE-adversarial/MME increases from 34.3%/16.6% for activation weighting to 57.5%/33.1% (Table 2). Experiments across quantizers, backbones and short generated answers provide further support. The contribution is primarily the post-hoc correction setting and fidelity evidence, rather than a new weighted-SVD closed form.
Critical strengths
-
Direct, controlled validation of fidelity repair at equal correction storage. With the same quantized checkpoint, rank, calibration questions and factor precision, OQER achieves 57.5%/33.1% net recovery on POPE-adversarial/MME versus 34.3%/16.6% for activation weighting; rank 32 still reaches 46.0%/29.6%. Table 10 implies mean remaining disagreements of 57 versus 88 and approximately 187 versus 233. Paired statistics and correction-strength controls support the difference (Tables 2, 10, 13; Appendices A, H). This establishes a useful application of known weighting principles to fixed-residual repair and is the main acceptance argument; it does not establish a unique margin-gradient benefit.
-
Structure controls and cross-setting results give the empirical contribution substance beyond one main table. The diagonal, sign and permutation controls support the role of output-coordinate structure and alignment (Table 6; Appendices I, K). Tables 16-19 extend the fidelity evidence across quantizers, backbones and short generated answers. Full output moments are not new, but the repair experiments provide concrete evidence of their usefulness. Some Qwen MME effects remain unresolved, so the finding should remain limited to the tested settings.
Critical weaknesses
-
Limited methodological differentiation. The formulation draws on QERA-style reconstruction of a fixed quantization residual and LASER-style two-sided output-gradient weighting. In Table 7, whole-prompt loss weighting achieves most of the improvement over activation weighting, while the additional margin advantage is not statistically significant. The authors acknowledge this overlap, but the extent of new knowledge beyond these established ingredients remains a principal limitation. The acceptance case therefore depends primarily on the significance of the controlled fidelity and structure findings.
-
The basis for the default configuration and its supported scope require clarification. Gradient rankings vary with the evaluated setting: Table 11 includes a lower OQER point estimate in a partial-layer, rank-8 configuration, without a significance mark, and the ScienceQA pilot in Appendix I favors likelihood over nearest-rival margin weighting. Appendix O explicitly acknowledges the absence of a general selection rule. These observations do not invalidate the main all-layer, rank-64 results, but leave uncertainty about how the recommended configuration is selected and the budget/readout regimes to which the reported benefits should be generalized.
-
Incomplete evidence for the deployment trade-off. Accuracy and teacher fidelity are evaluated with simulated quantization, whereas actual kernels are measured only for resource use. Table 8 reports similar peak memory for corrected 3-bit and the lowest-memory 4-bit reference, while Table 17's simulated MME results favor 4-bit in teacher fidelity. Since these measurements do not jointly characterize actual execution, they do not yet establish the practical fidelity-memory-latency trade-off under the intended deployment constraints. This is an evidence gap, rather than a demonstrated failure of the real-kernel implementation.
Questions for authors
-
Could the authors delineate the contribution relative to QERA-style residual reconstruction and LASER-style output-gradient weighting? Given that Table 7 does not establish a statistically significant advantage of the margin gradient over whole-prompt loss, which methodological distinction or controlled empirical finding constitutes the principal advance beyond these existing approaches? Please also contextualize this advance relative to other residual reconstruction methods like Quant Experts and SVDQuant.
-
How were the output-gradient objective and correction rank selected for the main experiments? Table 11 reports a lower POPE net-recovery point estimate for OQER at rank 8 when only layers 0-11 are corrected; the ScienceQA results in Appendix I also favors likelihood over nearest-rival margin weighting. Could the authors clarify whether the margin/rank-64 defaults were specified a priori or selected using calibration or development data, and delineate the budget and readout settings supported by the current evidence?
-
Using the existing evaluation items at the same correction budget, please report teacher-disagreement rates by teacher-margin bin for activation weighting, whole-prompt-loss weighting and OQER. Given the limitations in Appendices P/Q, which differences between these corrections does the flip analysis explain?
-
Could the authors characterize the fidelity-memory-latency trade-off of 3-bit plus OQER (ranks 32/64) relative to the 3 or 4-bit reference under comparable evaluation conditions? Indicate when adding a correction is preferable to increasing bitwidth at comparable memory or latency budgets.
If the authors satisfactorily address these concerns, I would be willing to raise my rating.
7 comments
[2/4] Response to Reviewer z7Jm
(Continued from Previous Response)
Q2. How the gradient and the rank were selected, and the supported settings
The answer margin was not chosen from results, but rank 64 was first compared on the test sets. An independent selection on held-out data is consistent with the same configuration, and a new all-layer sweep states for which ranks the gain holds.
The two defaults were not chosen in the same way. The answer margin was the first output gradient we ran. It follows from the flip analysis of Section 4, and the other gradient sources of Table 7 were implemented afterwards as controls, so no result was used to prefer it. The rank was not fixed in advance. Ranks 8, 32 and 64 were first compared on POPE-adversarial and MME (layers 0 to 11, seed 0, the runs of Table 12) before the all-layer rank-64 experiments, so the choice of rank 64 is not free of test-set selection.
To test whether the choice holds without the test sets, we ran an independent selection on held-out data. We used the 3,000 COCO dev questions of Appendix H (1,500 images). None of these images is in POPE or in the calibration sets. Six of them also appear in the COCO-based subtasks of MME, and the 12 questions on these images are excluded, which leaves 2,988 questions. Teacher disagreement needs no labels. In Table 28 the dev columns are measured on a re-quantized checkpoint (checkpoint C of Appendix R) at ranks 32 and 64, and the test column repeats Table 7 (the checkpoint of the submission). The activation-weighted and the answer-margin rows of one seed share one job and one set of quantized weights.
| Output moment built from | Dev disagreement (%), rank 32 | Dev disagreement (%), rank 64 | Test net recovery at rank 64, POPE-adversarial / MME (Table 7) |
|---|---|---|---|
| nothing (activation-only) | 2.35 | 2.29 | 34.3% / 16.6% |
| LASER-inspired last-24-token loss | 2.06 | 1.86 | 45.5% / 26.3% |
| whole-prompt loss | 2.01 | 1.86 | 52.5% / 28.6% |
| teacher-response likelihood | 1.65 | 1.55 | 54.0% / 28.0% |
| answer likelihood | 1.70 | 1.45 | 53.7% / 30.0% |
| answer margin (OQER default) | 1.74 | 1.45 | 57.5% / 33.1% |
At rank 64 the answer margin and the answer likelihood tie for the lowest dev disagreement (1.45% each, difference 0.00 points with a 95% interval of [−0.19, +0.19]). At rank 32 the teacher-response likelihood is lowest (1.65%) and the answer margin is close behind (1.74%). Without a correction the disagreement is 3.05%. The candidates of this selection are the six gradient sources at ranks 32 and 64. Among them, rank 64 with the answer margin or the answer likelihood gave the lowest observed dev disagreement, and in the same-checkpoint comparison of Table 7 these two do not differ significantly on the test sets. Four of the gradient sources ran as separate jobs (Appendix R), so we read this as consistent with the default and not as a ranking of all six. It does not make the original choice of the rank independent of the test sets.
Supported settings. Table 27 adds an all-layer rank sweep over three seeds on LLaVA-1.5-7B with AWQ-style 3-bit weights. The cells give the teacher disagreement of the activation-weighted correction minus that of OQER, in points (uncorrected model: 4.83% on POPE-adversarial, 10.87% on MME). For each seed the two corrections were built and evaluated in one job on the same quantized weights, so the comparison is paired.
| Rank (extra bits/weight) | 8 (+0.05) | 16 (+0.10) | 32 (+0.20) | 64 (+0.40) | 128 (+0.79) |
|---|---|---|---|---|---|
| POPE-adversarial | +1.29† | +1.74† | +0.73† | +1.04† | +0.81† |
| MME | −0.56 | +0.63 | +0.55 | +2.11† | +2.20† |
- Supported on LLaVA-1.5-7B with AWQ-style 3-bit weights: all-layer correction from rank 8 on POPE and from rank 64 on MME (Table 27). On MME the difference at ranks 16 and 32 has the same sign but is not significant, and at rank 8 OQER is not ahead.
- Supported at rank 64: binary readouts on three backbones and with three quantizers at 3 bits (Tables 3 and 4), and short generated answers on the three backbones with AWQ-style 3-bit weights (Table 17, seed 0).
- Benchmark-dependent: rank 8 on layers 0 to 11. In Table 12 the difference to activation weighting is −10.4 points on POPE (not significant) and +8.6† on MME, so we do not claim a gain for such budgets.
- Mixed: multiple-choice readouts. On MMBench OQER is ahead of activation weighting in fidelity and in accuracy (+2.18 pp†), and on the full ScienceQA-IMG test set the two are level (Table 30). In the ScienceQA pilot a candidate-normalised likelihood is ahead of the fixed-rival margin (43.2% against 32.6%, and 24.2% for activation-only weighting), because a K-way answer can flip to any rival. We therefore claim no general gain for K-way readouts and give no rule for the gradient there.
(Continued in Next Response)
[3/4] Response to Reviewer z7Jm
(Continued from Previous Response)
Q3. Teacher disagreement by teacher-margin bin, and what the flip analysis explains
The binned rates follow the prediction of the flip analysis, and OQER has the lowest disagreement in every low-margin bin of POPE. The margin model explains where the flips occur and the gap between activation-only and output weighting, but not the order among the output-weighted variants.
Table 31 (Appendix W), LLaVA-1.5-7B, rank 64, three-seed means, teacher disagreement in %. All columns come from the stored outputs of the main experiments and share one teacher and one uncorrected model. POPE-adv. is the adversarial split (3,000 questions), the POPE split that the whole-prompt run covers.
| Benchmark | Teacher margin (questions) | 3-bit | Activation-weighted | Whole-prompt loss | OQER |
|---|---|---|---|---|---|
| POPE-adv. | below 0.5 (263) | 31.18 | 25.60 | 20.41 | 19.90 |
| POPE-adv. | 0.5 to 1 (266) | 15.41 | 7.14 | 3.26 | 1.50 |
| POPE-adv. | 1 to 2 (537) | 2.05 | 0.31 | 0.25 | 0.12 |
| POPE-adv. | above 2 (1,934) | 0.00 | 0.00 | 0.00 | 0.00 |
| MME | below 0.5 (504) | 36.71 | 31.61 | 29.96 | 27.78 |
| MME | 0.5 to 1 (447) | 13.65 | 11.33 | 8.65 | 9.17 |
| MME | 1 to 2 (642) | 4.67 | 3.12 | 1.35 | 0.73 |
| MME | 2 to 4 (634) | 0.47 | 0.42 | 0.16 | 0.16 |
| MME | above 4 (147) | 0.00 | 0.00 | 0.00 | 0.00 |
Bold marks the lowest value in each row. Most disagreements concentrate below a teacher margin of 1, and no model disagrees above a margin of 2 on POPE. The lead of OQER over the whole-prompt loss is largest in the 0.5 to 1 bin of POPE (1.76 points), where the 3-bit model disagrees on 15.41% of the questions, the activation-weighted correction on 7.14%, the whole-prompt loss on 3.26% and OQER on 1.50%. The whole-prompt loss stays close to OQER (20.41% against 19.90% in the lowest bin) and is ahead in one MME bin, in line with Table 7.
Fit of the margin model to each corrected model (POPE-adversarial, seed 0). The numbers are those of Table 24, which has a new row for the whole-prompt loss.
| Correction | Slope β | Low-margin residual σ_Q1 | Flips predicted | Flips observed |
|---|---|---|---|---|
| Activation-weighted | 0.894 | 0.445 | 3.8% | 2.8% |
| Whole-prompt loss | 0.870 | 0.376 | 2.8% | 2.2% |
| OQER | 0.921 | 0.354 | 2.7% | 2.0% |
| Likelihood variant | 0.915 | 0.359 | 2.8% | 1.8% |
| Diagonal output moment | 0.876 | 0.427 | 3.5% | 3.6% |
On MME the model predicts 11.9% of flips for the activation-weighted correction and 9.9% for OQER, and 10.0% and 8.0% are observed (Table 24).
- Explained: where disagreements arise (small teacher margins) and the difference between activation-only and full output weighting (lower σ_Q1, fewer predicted and fewer observed flips).
- Not explained: the order among the output-weighted variants (the likelihood variant has more predicted and fewer observed flips than OQER) and the diagonal arm (fewer predicted than activation-only, more observed).
- We therefore use the model as motivation and as a description of the corrected models, not as the mechanism that ranks every arm (Appendices Q and W).
(Continued in Next Response)
[1/4] Response to Reviewer z7Jm
Thank you for the precise review. Your summary states the contribution as we intend it. We answer the four questions in order, and Q1, Q2 and Q4 also address the three weaknesses. Table numbers below refer to the revised PDF. Our summary comment lists all changes and the renumbering.
Q1. Contribution relative to QERA and LASER, and Quant Experts and SVDQuant
The principal advance is empirical. The paper asks whether, under a limited correction budget, minimizing the reconstruction error of a quantized VLM preserves its answers. Three controlled findings show that at the same budget a correction with a lower reconstruction error can preserve fewer of them.
Over three seeds the net recovery is 34.3% / 16.6% against 57.5% / 33.1% (Table 2), and the activation-weighted correction also captures more of the layer-output error than OQER (0.609 against 0.539, Table 6). It is already the exact optimum for its criterion, so the limit lies in the criterion. Table 32 describes the difference per layer for one seed. Of the residual energy weighted by the margin-gradient moment, the activation-weighted correction captures 61% and OQER 91%. In the attention projections of layers 22 to 31 the output moment has an effective rank of 1.6 to 3.4 against 14.6 to 16.1 for the input moment, and there the activation-weighted correction captures only 49 to 58%. OQER is ahead under its own metric by construction, so the numbers of Table 32 characterize the two corrections and are not an independent proof.
Ingredients and prior work. From QERA we take the object (the residual of a fixed quantized checkpoint) and from LASER the two-sided weighting. Neither is claimed. While preparing the revision we also found OBD-LLM (arXiv, April 2026), which is closer than either. As an extension it fits a low-rank adapter to the residual of a GPTQ-quantized LLM with the same two-sided whitening, using the output-gradient moment of the next-token loss, and reports perplexity. We now cite it as the closest prior work (Section 2, Appendix Y), and the whole-prompt loss arm of Table 7 is the counterpart of that weighting in our setting. The contribution is therefore the evidence for quantized VLMs with teacher fidelity as the target.
We do not rely on the margin gradient. Its lead over the whole-prompt loss is within resampling noise (Table 7, +5.0 / +4.5), as Section 6.3.2 states, while its lead over the LASER-inspired last-24-token loss is significant (+11.9† / +6.8†).
Quant Experts and SVDQuant (Appendix Y, with SVDQuant also in Table 33). SVDQuant keeps the leading singular components of the weight in a 16-bit branch and quantizes the remainder afterwards, without weighting. It belongs to the preserve-then-quantize family that SRR represents in Table 18. Quant Experts compensates weight and activation quantization of VLMs with a shared adapter and token-routed experts, weighting the input side. OQER fits one static term to the residual of a checkpoint that is already quantized and weights the output side as well, so it is complementary to both. We compare with both at the level of the objective and did not run them on our checkpoints.
(Continued in Next Response)