- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper argues that in feed-forward geometry Transformers, a high-leverage special token (MapAnything's metric-scale token) degrades disproportionately under quantization because it shares one activation grid with many patch tokens, and that token-blind transforms such as rotation and smoothing cannot fix this. It proposes QMapAnything, which adds a scale-token-specific bias correction on top of rotation and standard bias correction, and reports near-lossless low-bit accuracy and substantial memory savings on MapAnything.
Critical strengths
- Practical pipeline for MapAnything. The paper offers a complete, low-overhead PTQ recipe for a metric feed-forward reconstructor, which is useful for deploying MapAnything on memory-constrained hardware.
- Strong results. Accuracy is near-lossless at W4A8, the method clearly outperforms standard PTQ baselines at W4A4, and the W8A8 version achieves substantial memory savings and speedups on real hardware.
- Careful evaluation. Reporting metric outputs separately from scale-aligned ones exposes degradation that standard metrics would hide, a useful practice for future work.
- Insightful observation. App. D shows that token-averaged reconstruction losses are nearly blind to errors on an output-critical token, a finding that may matter beyond this model.
Critical weaknesses
Limited contribution beyond . Apart from , every component of QMapAnything is an existing technique:
- randomized Hadamard rotation (QuaRot; Ashkboos et al., 2024);
- token-uniform bias correction (Banner et al., 2019; Nagel et al., 2019);
- gradient refinement of the biases in the style of block-wise reconstruction (BRECQ, Li et al., 2021; OmniQuant, Shao et al., 2024);
- standard static per-tensor activation and per-channel weight quantization.
Without , the method reduces to QuaRot plus bias correction applied to MapAnything. PTQ for feed-forward geometry Transformers has already been studied (QuantVGGT, QVGGT, FGQ).
The analytical contributions do not stand on their own either. Props. 1–2 and Lemma 1 are elementary facts about orthogonal invariance, singular-value bounds, and permutation equivariance. Their role in the paper is to argue that an out-of-group term is required. exists only to decide whether to insert . Among the structural properties, S3 holds for essentially all Transformers, and S4 only determines where is placed. Once is set aside, little of independent interest remains.
Limited value of itself. only helps when a model has a special token that multiplicatively controls the global output, as MapAnything's scale token does. VGGT and DA3 have no such token, which is why extending to them yields no benefit (Table 5). This suggests that addresses a property of one architecture rather than of feed-forward geometry Transformers in general. Moreover, even when such a token exists, it occupies only a single row of the sequence. Handling it separately, or simply leaving it unquantized, would therefore add negligible overhead and avoid the grid disadvantage altogether, making the calibrated correction unnecessary.
Suggestion: (i) Compare against keeping the scale-token row in FP16 or INT8 activations and against per-token activation scaling, reporting accuracy and measured overhead. (ii) Identify other models with , or narrow the claims of generality.
Questions for authors
Please check the suggestions in the weakness.
2 comments
Thank you for your further elaboration. I am willing to raise my score to 3.
Official Comment by Authors
We thank the reviewer for the careful assessment of the method’s novelty, theoretical analysis, and practical overhead. Below, we clarify the contribution of special-row residual correction and the limits of the analysis, supported by controlled ablations, MoGe-2 experiments, and deployment measurements.
R3.1 Question: What is the independent contribution beyond existing PTQ components?
Response: Our contributions beyond existing PTQ components are threefold.
(1) Risk diagnosis with Γ. We combine a special row’s quantization disadvantage and output sensitivity to screen for task-critical quantization risk. On MoGe-2, the median Γ is 25.651 across 1,536 valid scene-layer records.
(2) Correctability analysis with η. We quantify the systematic component of the special-row residual remaining after token-uniform bias correction. On MoGe-2, fitting bs on one of two disjoint scene folds and evaluating it on the other reduces residual energy by 24.57% and 24.21% in the two directions.
(3) Row-selective correction with bs. We compensate for this residual while preserving the shared activation grid and integer GEMM path. With the rotation, quantized weights, and quantization scales fixed, adding bs improves AUC@30 by 1.04 and 2.46 percentage points under closed-form calibration and after refinement, respectively (Table 8, Appendix F.1).
Table 8. W4A4 comparison with indoor calibration and outdoor evaluation.
R3.2 Question: What is the independent value of the theoretical analysis and Γ?
Response: The analysis provides three practical insights.
(1) Whole-block reconstruction loss can obscure errors in an output-critical row. In the original appendix, a 34.2% reduction in held-out special-row MSE corresponds to only a 0.010% reduction in whole-block MSE, motivating separate inspection of critical-row residuals.
(2) Quantization risk and residual correctability require separate assessments. Γ screens for task-critical quantization risk, while η measures the systematic residual component recoverable by bias correction.
(3) The theory motivates residual checks after shared channel transformations. Orthogonal transforms preserve relative token SNR proxies under a shared quantization step. General channel transformations can change these ratios, subject to a condition-number bound. The row-specific term bs targets the systematic residual remaining after these transformations and token-uniform bias correction.
R3.3 Question: Is bs effective only for MapAnything?
Response: No. On MoGe-2, adding bs reduces metric-depth AbsRel from 0.225 ± 0.006 to 0.198 ± 0.003 and absolute log-scale drift from 0.137 ± 0.001 to 0.076 ± 0.003 across three calibration seeds (Table 6, Section 4.6). MoGe-2 updates its CLS token in every encoder block, providing a positive case with a different update structure from MapAnything. The method targets output-critical special rows with correctable systematic residuals under a shared activation quantization grid.
Table 6. MoGe-2 W4A4 QDQ on 462 images (quantized rows: three-seed mean ± SD).
R3.4 Question: What are the measured overheads of higher precision for special rows and per-token quantization?
Response: In the evaluated W8A8 implementation, special-token FP16 increases latency and peak memory by 2.90% and 12.24% relative to QMapAnything, while dynamic per-token INT8 increases them by 9.78% and 0.90% (Table 9, Appendix F.2). Adding bs to the baseline increases latency by only 0.19 ms, with no change in measured peak memory. The dynamic per-token implementation adds row-wise scale computation, quantization and dequantization, INT32 intermediate reads and writes, and additional kernel dispatch. Both QMapAnything and this implementation execute 64 INT8 GEMMs.
Table 9. W8A8 deployment with integer computation on L4.