acceptodds
Withdrawn from ICLR 2027

Calibrate the Decisions That Change the Future: On-Policy Post-Training Quantization for Multimodal Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Post-training quantization (PTQ) lowers deployment cost for multimodal large language models, but calibration typically reconstructs fixed sequences with local objectives. This overlooks autoregressive feedback: a quantization-induced token change redirects the prefix and changes future states. Yet on-policy coverage alone is insufficient because many decision mismatches barely affect future generation. We propose OnPTQ, an on-policy framework that calibrates on trajectories visited by the current quantized policy. On shared prefixes, OnPTQ identifies quantization-eroded boundaries, evaluates competing tokens through short counterfactual rollouts, and combines current discrepancy with branch consequence into a Decision–Consequence risk. The risk prioritizes critical states, while context anchoring and trajectory refresh preserve multimodal behavior and keep calibration aligned with the updated policy. We further derive a Decision–Consequence bound linking behavioral deviation to current policy discrepancy and action-conditioned future-value span. Across vision–language and omni-modal Qwen models under multiple low-bit settings, OnPTQ improves downstream performance and yields fewer correctness flips against the corresponding Dense/FP16 references, without changing the deployed inference graph.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Post-Training Quantization, On-Policy Learning, Vision-Language Models, Behavioral Robustness, Counterfactual Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_y1r8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

ONPTQ calibrates multimodal models on prefixes generated by the current quantized model. It combines decision-boundary erosion and short counterfactual branch divergence to prioritize calibration states, with context anchoring and iterative trajectory refresh. Experiments on Qwen models show the clearest gains.

Critical strengths
  1. Well-motivated. On-policy prefixes and counterfactual branches address autoregressive distribution shift beyond static reconstruction.

  2. W4A6 average accuracy improves over the strongest listed baseline by 2.8 and 1.8 points on the two VLM backbones (Table 1), without changing the deployed inference graph.

Critical weaknesses
  1. Practical advantage and robustness remain unclear.

    • Added offline calibration cost is not quantified in GPU-hours or relative compute.
    • Evaluation is limited to 3B–8B Qwen-family models.
    • Five refreshed rounds reduce MMMU/ScienceQA accuracy from 48.8/89.7 to 44.2/86.1 (Table 4), raising concerns about stopping and stability.
    • Qwen2.5-VL baselines are imported while ONPTQ is evaluated separately (Appendix C.1); higher-precision gains are small or inconsistent.
  2. The benefit of combining the risk signals remains unclear. Consequence-only selection achieves a lower joint-critical flip rate in Table 9, while ONPTQ improves erosion. A matched downstream comparison would clarify the value of the combined risk.

Questions for authors
  1. What are the calibration GPU-hours, base quantizer, trainable parameters, and optimization schedule? Does the gain persist under equal compute?
  2. How were the imported Qwen2.5-VL baselines verified to match ONPTQ's evaluation protocol?
  3. Why does five-round refresh degrade performance in Table 4? Was the total update budget controlled, and how was the default round count selected?
  4. Does ONPTQ outperform consequence-only selection under matched downstream calibration? Please explain the flip-rate trade-off in Table 9.
Reviewer_A8VaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a new post-training quantization method that uses on-policy model trajectories/traces generated by the quantized model as the calibration data. It introduces the loss function that weighs individual prefix losses by their corresponding risks. The risk is determined by the current token discrepancy between the dense and quantized model and an estimate of the future discrepancies. The loss also includes a reconstruction loss on the input context making sure the dense and quantized models preserve similar activations there.

Critical strengths

The proposed quantization method is well motivated by how quantization errors can propagate through the whole response generation. Although the method is fairly complicated, the methods section is well written. The empirical results are promising. Splitting the risk into components on token discrepancy (e_t), future discrepancy (A_t), and distribution discrepancy (j_t) is interesting.

Critical weaknesses

Missing standard baselines. Some very standard baselines for weight-only quantization such as AWQ and GPTQ are missing as well as some baselines that are known in the literature to improve over the reported ones, such as OmniQuant, which improves over SmoothQuant. AWQ baseline is mentioned as used in the main text but the results are not reported in the table.

On-policy calibration data justification. The motivation for the quantized and not dense model generation of the calibration data is missing. The current procedure that generates from the quantized model involves weighting and then the iterative training procedure is fairly involved so it would be good to get additional results why it is empirically better than generating from the dense model. Concretely, what happens to the baseline’s quantization methods, optimization in eq (2), and to OnPTQ once the calibration data is sampled from the dense model itself and not a static dataset? Some works in quantization literature show the full model responses provide improvements over pre-specified calibration: Self-calibration for Language Model Quantization and Pruning Williams et al (2025) (https://aclanthology.org/2025.naacl-long.509.pdf). Somewhat related LLM pruning literature also uses the full/unpruned model responses to generate calibration data.

Additional ablations. The risk formula in eq (9) is fairly complex, so justifying that with an additional ablation where only A_t is used for the risk would be helpful across datasets.

Compute cost and scalability. The compute cost of the proposed method is not discussed. It would be good to get some numbers on the total generated tokens by the proposed procedure.

Questions for authors

See the weakness and the additional questions below.

  1. How is the proposed method quantized model initialized and how does initialization change the performance?

  2. What data was used to select the optimal hyperparameters? Table 8 reports the results on MMMU and ScienceQA, which are also eval datasets, so it’s unclear if Table 8 used a separate validation set or actual eval dataset.

  3. Can the authors add one more larger model to show the compute of the proposed method scales and the gains compared to the baselines (and additional baselines such as AWS and GPTQ) are still present?

Reviewer_sP99OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces OnPTQ, an on-policy post-training quantization framework for multimodal LLMs designed to mitigate error compounding during autoregressive generation. Rather than relying on static calibration data, OnPTQ samples trajectories directly from the quantized model and runs short counterfactual rollouts at decision points where dense and quantized predictions diverge. These rollouts, together with distributional and decision-boundary shifts, form a composite risk score that selectively prioritizes critical states during calibration. Experiments on several multimodal Qwen models show consistent gains over standard PTQ baselines, with the advantage being most noticeable in challenging low-bit settings.

Critical strengths

1. The motivation for this calibration design is quite intuitive.

A quantized model’s predictions affect its future inputs, so calibration on its own prefixes makes sense. The paper goes a step further by asking which decisions are worth correcting, rather than treating all visited states equally. The additional computation stays within calibration and does not change the inference graph.

2. The W4A6 results are encouraging.

On Qwen3-VL, average accuracy improves from 71.6 for the strongest compared baseline to 73.4 under a common evaluation pipeline. The ablations also support the usefulness of consequence information, context anchoring, and trajectory refresh. Together, these results suggest that the approach offers a meaningful improvement in the tested low-precision setting.

Critical weaknesses
  1. The calibration update needs more detail. Algorithm 1 uses BasePTQUpdate, but the base quantizer, trainable parameters, optimizer, and update schedule are not clearly specified. Without these details, it is hard to reproduce the results or judge how much of the improvement comes from the proposed calibration strategy.

  2. The advantage of the combined risk is not yet clear. In Table 9, consequence-only selection has a lower joint-critical margin flip rate than OnPTQ (43.59% vs. 53.85%), while OnPTQ performs better on erosion. However, consequence-only is not included in Table 3 or the downstream ablations. I would like to see its downstream results under the same calibration budget before concluding that combining the signals is better than using consequence alone.

Questions for authors
  1. The comparison of rollout lengths only covers one model and one quantization setting. Is H=4H=4 also a good choice for other models, modalities, and bit widths? It would be helpful to see how changing HH affects both performance and calibration time.

  2. What are the base quantizer, trainable parameters, and exact optimization schedule used by OnPTQ? Please also report the calibration time relative to the baseline.

  3. How does a consequence-only baseline hold up across downstream tasks when held to the same data, initialization, and compute budget? Also, for the “all on-policy states” ablation, are the loss formulations and the effective number of updates strictly matched?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.