acceptodds
Under review as a conference paper at ICLR 2027

Rotation Is Token-Blind: A Structural Criterion for Post-Training Quantization of Feed-Forward Geometry Transformers

Abstract

Feed-forward 3D models use Transformers to predict camera poses, depth, and structure in one pass, avoiding iterative multi-view optimization. Long sequences and deep attention incur high compute and memory costs, motivating Post-Training Quantization (PTQ) for deployment. Existing PTQ methods mainly suppress channel-wise outliers by smoothing or orthogonal rotation. Geometry Transformers introduce a different challenge: heterogeneous token types share one activation quantization grid. A few special tokens may exert high leverage over global outputs and be updated by only a subset of blocks. Token-blind transforms, which apply the same transformation to every token, reduce overall quantization error but cannot remove differentiated degradation across token types. We propose QMapAnything, a structure-aware PTQ framework. We cast floating-point-preserving rotation and smoothing as a token-blind reparameterization group, and prove that they yield common-mode gains while leaving relative token SNR unchanged. We introduce a statistic that combines a special token's quantization disadvantage and output leverage to predict whether differentiated bias correction is needed beyond rotation. Experiments on geometric benchmarks show that W4A8 closely matches full-precision performance. Under W8A8, peak memory falls by - versus FP32, with up to speedup on real hardware. These results provide a principled and efficient approach to deploying feed-forward 3D reconstruction models in resource-constrained edges.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
post-training quantization, model compression, 3D reconstruction, feed-forward transformers, geometry transformers, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_u4nPOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that in feed-forward geometry Transformers, a high-leverage special token (MapAnything's metric-scale token) degrades disproportionately under quantization because it shares one activation grid with many patch tokens, and that token-blind transforms such as rotation and smoothing cannot fix this. It proposes QMapAnything, which adds a scale-token-specific bias correction on top of rotation and standard bias correction, and reports near-lossless low-bit accuracy and substantial memory savings on MapAnything.

Critical strengths
  • Practical pipeline for MapAnything. The paper offers a complete, low-overhead PTQ recipe for a metric feed-forward reconstructor, which is useful for deploying MapAnything on memory-constrained hardware.
  • Strong results. Accuracy is near-lossless at W4A8, the method clearly outperforms standard PTQ baselines at W4A4, and the W8A8 version achieves substantial memory savings and speedups on real hardware.
  • Careful evaluation. Reporting metric outputs separately from scale-aligned ones exposes degradation that standard metrics would hide, a useful practice for future work.
  • Insightful observation. App. D shows that token-averaged reconstruction losses are nearly blind to errors on an output-critical token, a finding that may matter beyond this model.
Critical weaknesses

Limited contribution beyond bsb_s. Apart from bsb_s, every component of QMapAnything is an existing technique:

  • randomized Hadamard rotation (QuaRot; Ashkboos et al., 2024);
  • token-uniform bias correction β\beta (Banner et al., 2019; Nagel et al., 2019);
  • gradient refinement of the biases in the style of block-wise reconstruction (BRECQ, Li et al., 2021; OmniQuant, Shao et al., 2024);
  • standard static per-tensor activation and per-channel weight quantization.

Without bsb_s, the method reduces to QuaRot plus bias correction applied to MapAnything. PTQ for feed-forward geometry Transformers has already been studied (QuantVGGT, QVGGT, FGQ).

The analytical contributions do not stand on their own either. Props. 1–2 and Lemma 1 are elementary facts about orthogonal invariance, singular-value bounds, and permutation equivariance. Their role in the paper is to argue that an out-of-group term is required. Γ\Gamma exists only to decide whether to insert bsb_s. Among the structural properties, S3 holds for essentially all Transformers, and S4 only determines where bsb_s is placed. Once bsb_s is set aside, little of independent interest remains.

Limited value of bsb_s itself. bsb_s only helps when a model has a special token that multiplicatively controls the global output, as MapAnything's scale token does. VGGT and DA3 have no such token, which is why extending bsb_s to them yields no benefit (Table 5). This suggests that bsb_s addresses a property of one architecture rather than of feed-forward geometry Transformers in general. Moreover, even when such a token exists, it occupies only a single row of the sequence. Handling it separately, or simply leaving it unquantized, would therefore add negligible overhead and avoid the grid disadvantage altogether, making the calibrated correction bsb_s unnecessary.

Suggestion: (i) Compare against keeping the scale-token row in FP16 or INT8 activations and against per-token activation scaling, reporting accuracy and measured overhead. (ii) Identify other models with Γ>0\Gamma > 0, or narrow the claims of generality.

Questions for authors

Please check the suggestions in the weakness.

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for the careful assessment of the method’s novelty, theoretical analysis, and practical overhead. Below, we clarify the contribution of special-row residual correction and the limits of the analysis, supported by controlled ablations, MoGe-2 experiments, and deployment measurements.

R3.1 Question: What is the independent contribution beyond existing PTQ components?

Response: Our contributions beyond existing PTQ components are threefold.

(1) Risk diagnosis with Γ\Gamma. We combine a special row’s quantization disadvantage and output sensitivity to screen for task-critical quantization risk. On MoGe-2, the median Γ\Gamma is 25.651 across 1,536 valid scene-layer records.

(2) Correctability analysis with η\eta. We quantify the systematic component of the special-row residual remaining after token-uniform bias correction. On MoGe-2, fitting bsb_s on one of two disjoint scene folds and evaluating it on the other reduces residual energy by 24.57% and 24.21% in the two directions.

(3) Row-selective correction with bsb_s. We compensate for this residual while preserving the shared activation grid and integer GEMM path. With the rotation, quantized weights, and quantization scales fixed, adding bsb_s improves AUC@30 by 1.04 and 2.46 percentage points under closed-form calibration and after refinement, respectively (Table 8, Appendix F.1).

Table 8. W4A4 comparison with indoor calibration and outdoor evaluation.

MethodScale rel ↓AUC@30 (%) ↑Depth rel ↓
Original FP (BF16)0.0798.570.09
Baseline-CF0.3087.630.14
QMapAnything-CF0.2688.670.13
Baseline-Ref0.1791.140.14
QMapAnything-Ref0.1593.600.13
Special-token FP160.2588.940.13
QuantVGGT0.1592.960.13

R3.2 Question: What is the independent value of the theoretical analysis and Γ\Gamma?

Response: The analysis provides three practical insights.

(1) Whole-block reconstruction loss can obscure errors in an output-critical row. In the original appendix, a 34.2% reduction in held-out special-row MSE corresponds to only a 0.010% reduction in whole-block MSE, motivating separate inspection of critical-row residuals.

(2) Quantization risk and residual correctability require separate assessments. Γ\Gamma screens for task-critical quantization risk, while η\eta measures the systematic residual component recoverable by bias correction.

(3) The theory motivates residual checks after shared channel transformations. Orthogonal transforms preserve relative token SNR proxies under a shared quantization step. General channel transformations can change these ratios, subject to a condition-number bound. The row-specific term bsb_s targets the systematic residual remaining after these transformations and token-uniform bias correction.

R3.3 Question: Is bsb_s effective only for MapAnything?

Response: No. On MoGe-2, adding bsb_s reduces metric-depth AbsRel from 0.225 ± 0.006 to 0.198 ± 0.003 and absolute log-scale drift from 0.137 ± 0.001 to 0.076 ± 0.003 across three calibration seeds (Table 6, Section 4.6). MoGe-2 updates its CLS token in every encoder block, providing a positive case with a different update structure from MapAnything. The method targets output-critical special rows with correctable systematic residuals under a shared activation quantization grid.

Table 6. MoGe-2 W4A4 QDQ on 462 images (quantized rows: three-seed mean ± SD).

MethodMetric depth AbsRel ↓Scale-aligned depth AbsRel ↓Absolute log-scale drift ↓
FP0.1930.072—
Baseline (R + β)0.225 ± 0.0060.100 ± 0.0030.137 ± 0.001
QMapAnything0.198 ± 0.0030.085 ± 0.0010.076 ± 0.003

R3.4 Question: What are the measured overheads of higher precision for special rows and per-token quantization?

Response: In the evaluated W8A8 implementation, special-token FP16 increases latency and peak memory by 2.90% and 12.24% relative to QMapAnything, while dynamic per-token INT8 increases them by 9.78% and 0.90% (Table 9, Appendix F.2). Adding bsb_s to the baseline increases latency by only 0.19 ms, with no change in measured peak memory. The dynamic per-token implementation adds row-wise scale computation, quantization and dequantization, INT32 intermediate reads and writes, and additional kernel dispatch. Both QMapAnything and this implementation execute 64 INT8 GEMMs.

Table 9. W8A8 deployment with integer computation on L4.

MethodScale AbsRel ↓Unaligned depth AbsRel ↓AUC@30 (%) ↑Latency (ms) ↓Peak memory (GB) ↓
Baseline (R + β)0.0710.09598.47218.723.563
QMapAnything: R + β + bₛ0.0710.09698.53218.913.563
Special-token FP16 (no bₛ)0.0710.09698.53225.25 (+2.90%)3.999 (+12.24%)
Dynamic per-token INT8 (no bₛ)0.0710.09698.80240.31 (+9.78%)3.595 (+0.90%)
QuantVGGT0.0720.09698.53238.67 (+9.03%)3.595 (+0.90%)

Thank you for your further elaboration. I am willing to raise my score to 3.

Reviewer_HenyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper identifies a shortcoming of post-training quantization techniques for feed-forward reconstruction transformers. These models often rely on special tokens, e.g., scale or camera tokens, concatenated to the image patch sequence and uniform quantization might lead to suboptimal results. To this end, they develop the QMapAnything framework that contains 4 structural properties, measured pre-quantization, that inform quantization decisions. They show that existing smoothing and rotation transformations as well as sequence uniform affine corrections cannot account for special tokens if these follow a different weight/activation distribution. The construction informs a statistic, , that informs the decision of quantizing single, special tokens using an additional row-specific affine correction. The paper tests this on the MapAnything architecture where the Gamma statistic identifies the scale token as requiring row-specific treatment for quantization. Applying an affine correction to the scale token improves the performance of the quantized model consistently when compared to baselines that employ rotation and smoothing.

After rebuttal, I have changed my score to weak accept.

Critical strengths

S1. The paper’s result that existing post-training quantization methods based on rotation and smoothing can be studied from a unified perspective through a general form (Eq. 5) and are constrained for special tokens (Lemma 1) are interesting and potentially useful for future work on quantization. Particularly since sparse transformer architectures start adopting special tokens for landmarks and thus exhibit a similar structure [1].

[1] Leng, Jiaqi, et al. "Understanding and improving length generalization in hierarchical sparse attention models." International Conference on Learning Representations. Vol. 2026. 2026.

Critical weaknesses

W1. The proposed structural properties and empirical results are heavily tied to the MapAnything architecture. As a result the claim that the paper introduces a framework is only partially supported. In particular (S2) Multiplicative leverage in which the output is linear in the prediction based on a single token and (S4) sparse routing where the single token only is part of a subset of the transformer blocks are very specific and almost unique to the MapAnything architecture. Moreover, the statistic for (S1) is only evaluated against one positive case, MapAnything, which also initially provides the empirical motivation for the paper, which further raises concerns regarding how generalizable this framework is.

W2. While the paper compares against general purpose quantization techniques as baselines, prior works [2] have already identified issues and developed solutions for models with special tokens. For example, QuantVGGT [2] uses double smoothing with a global transformation and channel-wise local smoothing to improve quantization error of outlier tokens. This should apply to the scale token of MapAnything as well.

[2] Feng, Weilun, et al. "Quantized visual geometry grounded transformer." International Conference on Learning Representations. Vol. 2026. 2026.

Questions for authors

Q1. L290 states that is defined for “distinguished tokens that pass the S2 audit.” (S2) refers to the structural requirement of a single token with multiplicative leverage. In that sense the criteria is limited to tokens that affect the output linearly?

Q2. As only one positive sample it would be helpful to test and validate the criterion on another architecture. One example that comes to mind is MoGe-2 [3]. While MoGe-2 is a monocular model it has a similar structure with multiplicative scaling of the output predicted from a single token which should satisfy the criteria.

Q3. How does dual-smoothed quantization from QuantVGGT [2] compare both in the theoretical sense, w.r.t. Eq. 5, and empirically? Intuitively, local smoothing can implement token-specific quantization if outlier dimensions group post global rotation and thus may not require additional structural information. Thus I think it is a baseline that requires additional discussion.

[3] Wang, Ruicheng, et al. "Moge-2: Accurate monocular geometry with metric scale and sharp details." Advances in Neural Information Processing Systems 38 (2026): 35928-35959.

4 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for the constructive suggestions on generality, the structural conditions, and the comparison with QuantVGGT.

R2.1 Question: Are the structural conditions specific to MapAnything, and can MoGe-2 provide evidence of transfer?

Response: No, the applicability is not limited to MapAnything. MoGe-2 provides a second positive case.

(1) Task performance. Across three calibration seeds, adding bsb_s reduces metric-depth AbsRel from 0.225 ± 0.006 to 0.198 ± 0.003 and absolute log-scale drift from 0.137 ± 0.001 to 0.076 ± 0.003 (Table 6, Section 4.6).

Table 6. MoGe-2 W4A4 QDQ on 462 images (quantized rows: three-seed mean ± SD).

MethodMetric depth AbsRel ↓Scale-aligned depth AbsRel ↓Absolute log-scale drift ↓
FP0.1930.072—
Baseline (R + β)0.225 ± 0.0060.100 ± 0.0030.137 ± 0.001
QMapAnything0.198 ± 0.0030.085 ± 0.0010.076 ± 0.003

(2) Diagnostic and correctability evidence. Across 16 scenes, with one calibration image per scene, and 96 linear-layer inputs, the median Γ\Gamma is 25.651 (IQR [7.581, 66.175]), with all 1,536 records valid. The task loss is unaligned metric-depth AbsRel over valid ground-truth pixels, with predicted binary masks held fixed. In separate validation using two disjoint scene folds, fitting bsb_s on one fold and evaluating it on the other reduces residual energy by 24.57% and 24.21% in the two directions, with improvements at 94 of 96 locations in each direction.

(3) Structural applicability. MoGe-2 predicts a global multiplicative scale through a nonlinear CLS head, satisfying S2. Its CLS token is updated in every encoder block, demonstrating effective correction with dense updates. S4 therefore guides correction placement along the token’s update path.

We have added these results and the diagnostic and validation details to the revised manuscript.

R2.2 Question: Does S2 require a linear mapping from the special token to the output?

Response: No. S2 requires multiplicative coupling through the final scale, while the mapping from the special token to that scale can be nonlinear:

Dmetric=m⋅D~,m=g(hs).D_{\mathrm{metric}}=m\cdot \widetilde{D},\quad m=g(h_s).

With D~\widetilde{D} held fixed, the output is linear in m, while g can be nonlinear. In MoGe-2, the CLS scale head and exponential mapping produce the global scale. With this nonlinear head, adding bsb_s reduces absolute log-scale drift from 0.137 ± 0.001 to 0.076 ± 0.003 across three calibration seeds (Table 6, Section 4.6).

R2.3 Question: How does the method differ from QuantVGGT in mechanism and measured performance?

Response: QMapAnything differs from the adapted QuantVGGT baseline in two respects.

(1) Mechanism. QuantVGGT calibration optimizes shared channel smoothing and weight and activation clipping through blockwise reconstruction. QMapAnything uses bsb_s to explicitly correct the systematic residual of the output-critical row after token-uniform bias correction, while preserving the shared activation grid and integer GEMM path.

(2) Measured performance. Under W4A4, QMapAnything-Ref and QuantVGGT achieve comparable measured accuracy, with AUC@30 values of 93.60% and 92.96%, respectively. Both have scale and depth errors of 0.15 and 0.13 at the reported precision (Table 8, Appendix F.1). Under W8A8, both achieve an AUC@30 of 98.53%, while QMapAnything reduces latency from 238.67 ms to 218.91 ms, an 8.28% reduction in the evaluated L4 / TensorRT 10.8 implementations (Table 9, Appendix F.2).

Table 8. W4A4 comparison with indoor calibration and outdoor evaluation.

MethodScale rel ↓AUC@30 (%) ↑Depth rel ↓
Original FP (BF16)0.0798.570.09
Baseline-CF0.3087.630.14
QMapAnything-CF0.2688.670.13
Baseline-Ref0.1791.140.14
QMapAnything-Ref0.1593.600.13
Special-token FP160.2588.940.13
QuantVGGT0.1592.960.13

Table 9. W8A8 deployment with integer computation on L4.

MethodScale AbsRel ↓Unaligned depth AbsRel ↓AUC@30 (%) ↑Latency (ms) ↓Peak memory (GB) ↓
Baseline (R + β)0.0710.09598.47218.723.563
QMapAnything: R + β + bₛ0.0710.09698.53218.913.563
Special-token FP16 (no bₛ)0.0710.09698.53225.25 (+2.90%)3.999 (+12.24%)
Dynamic per-token INT8 (no bₛ)0.0710.09698.80240.31 (+9.78%)3.595 (+0.90%)
QuantVGGT0.0720.09698.53238.67 (+9.03%)3.595 (+0.90%)

Thank you for answering my questions and the additional experiments with MoGe-2 and QuantVGGT.
The additional results with MoGe-2 resolve my concerns regarding the transferability and provide evidence for the introduced method being a framework.

On the other hand, I agree with Reviewer u4nP in that the choice of scale token is limited to only a few selected architectures (MoGe, MapAnything) which limits the applicability of the method.

More importantly, checking the comparison against QuantVGGT in Appendix F.1, the proposed method achieves similar accuracy and only marginal computational gains over QuantVGGT (238ms -> 218ms). This is in the W4A4 setting where QMapAnything had the largest gains over other baselines in Table 2. In particular, the scale relative error (and thus metric prediction error), which QMapAnything is specifically targeting through the proposed row-selective bsb_s correction, is the same for both methods. Overall, this means QuantVGGT is more general, as it applies to architectures without a scale token and transfers to MapAnything without changes (as shown by the author’s experiment), while preserving almost all of the gain of the proposed method.

One follow up question would be if the bsb_s correction can be combined with QuantVGGT for improved results, or if QuantVGGT fully subsumes the proposed method?

Overall, the theoretical development shows that general quantization baselines lack row-specific correction and leads to a simple fix. However, in the context of geometry prediction models, the finding of large activations tied to special tokens is not new (QuantVGGT) and the empirical results do not provide consistent improvements over existing work while being less general (limited to scale token only a few methods use). As a result, it is hard to recommend acceptance of the paper in its current form.

AuthorsOpenReview ↗

Official Comment by Authors

(1) The choice of scale token is limited to only a few selected architectures (MoGe, MapAnything) which limits the applicability of the method.

Your main concern is that our method has limited applicability because it relies on scale tokens used by only a few architectures. We respectfully disagree. QMapAnything is governed by structural risk and residual correctability, without depending on the specific implementations of MapAnything or MoGe-2. We study critical representations that share an activation quantization grid and exert multiplicative influence over the global metric scale:

Dmetric=g(hs)D~.D_{\mathrm{metric}}=g(h_s)\widetilde D.

This structure allows a nonlinear scale head gg and does not require the special token to be updated in only a subset of blocks. We use

Γ=λsλˉpatch(max⁡t∈P∣ht∣2∣hs∣2−1)+\Gamma=\frac{\lambda_s}{\bar{\lambda}_{\mathrm{patch}}}\left(\frac{\max_{t\in\mathcal P}\\|h_t\\|_2}{\\|h_s\\|_2}-1\right)_+

to jointly screen for quantization disadvantage and output sensitivity, and use the residual statistic η\eta to assess correctability. Here, P\mathcal P contains patch tokens only, and λt=∣∂L/∂ht∣2\lambda_t=\\|\partial\mathcal L/\partial h_t\\|_2. S4 guides correction placement without requiring sparse updates.

MapAnything updates its scale token only in global blocks, whereas MoGe-2 uses a nonlinear CLS scale head and updates CLS in every encoder block. Despite this difference in update structure, MoGe-2 also benefits from the correction:

MoGe-2 W4A4 QDQ results on 462 test images. Quantized results are reported as mean ± standard deviation over three calibration seeds.

MethodMetric depth AbsRel ↓Scale-aligned depth AbsRel ↓Absolute log-scale drift ↓
FP0.1930.072—
Base0.225 ± 0.0060.100 ± 0.0030.137 ± 0.001
Ours0.198 ± 0.0030.085 ± 0.0010.076 ± 0.003

Base = R+βR+\beta, Ours = R+β+bsR+\beta+b_s.

Adding bsb_s reduces the three metrics by 12.0%, 15.0%, and 44.5%, respectively. Furthermore, 1,536 scene-layer records on MoGe-2 yield a median Γ=25.651\Gamma=25.651. Fitting the correction on one of two disjoint scene folds and evaluating it on the other reduces residual energy by 24.57% and 24.21% in the two directions. These results jointly validate structural risk diagnosis, residual correctability across scenes, and downstream task gains.

The framework also identifies cases with little additional benefit from correction. Both VGGT and DA3 yield Γ=0\Gamma=0 in the structural audit, and adding bsb_s changes AUC@30 by only −0.02 and +0.04 percentage points, respectively. Our contribution therefore includes identifying when additional row-selective correction is warranted, without requiring the same correction to provide equal gains across all architectures.

AuthorsOpenReview ↗

Official Comment by Authors

(2) The proposed method achieves similar accuracy and only marginal computational gains over QuantVGGT.

QMapAnything offers lower measured deployment cost while maintaining comparable prediction accuracy. In W8A8 deployment with two and four input views, it reduces latency by 8.28% and 7.01%, and peak memory by 0.89% and 2.87%, respectively, relative to QuantVGGT. This deployment advantage is observed in both evaluated view configurations.

Measured W8A8 deployment results. The four-view comparison uses a unified build configuration.

ViewsMethodScale AbsRel ↓AUC@30 (%) ↑Latency (ms) ↓Peak memory (GB) ↓
2QuantVGGT0.07298.53238.673.595
2QMapAnything0.07198.53218.913.563
4QuantVGGT0.03916998.6667492.364.284
4QMapAnything0.03830898.6556457.854.161

With four views, QMapAnything also reduces scale AbsRel by 2.20% and unaligned depth AbsRel from 0.081163 to 0.080715, alongside the reductions in latency and memory. Pose AUC@30 remains nearly unchanged, differing by only 0.0111 percentage points.

The row-selective correction itself also has little incremental deployment overhead. In the original two-view control experiment, adding bsb_s to the shared base configuration increases latency from 218.72 to 218.91 ms, an increment of only 0.19 ms, while peak memory remains 3.563 GB.

Furthermore, the combination experiment in our response to Question (3) directly tests whether this correction provides an additional benefit. Adding bsb_s to QuantVGGT reduces scale relative error from 0.14945 to 0.13766, a further 7.89% reduction. This provides direct evidence of complementarity beyond the accuracy comparison between the complete methods.

(3) If the correction can be combined with QuantVGGT for improved results, or if QuantVGGT fully subsumes the proposed method?

Yes. Combining bsb_s with QuantVGGT provides additional gains in scale prediction. Following your suggestion, we evaluated QuantVGGT with and without bsb_s:

MethodScale rel ↓AUC@30 (%) ↑Depth rel ↓
QuantVGGT0.1494592.960.12926
QuantVGGT + bsb_s0.1376693.040.12775

Adding bsb_s reduces scale relative error by 7.89% and depth relative error by 1.17%, while increasing AUC@30 by 0.08 percentage points. The reduction in scale error directly addresses the metric-scale prediction problem targeted by our method.

This complementary benefit has a concrete explanation in terms of residual correction. At a correction insertion point, the combined representation takes the form

H^combined=H^QuantVGGT+usbs⊤,\widehat H_{\mathrm{combined}}=\widehat H_{\mathrm{QuantVGGT}}+u_s b_s^\top,

where usu_s selects the row to be corrected. For a fixed base quantization configuration, let rs=h^s−hsr_s=\widehat h_s-h_s denote the difference between the quantized and floating-point representations of that row. The optimal constant bias satisfies

bs⋆=−E[rs],b_s^\star=-\mathbb E[r_s],

min⁡bsE∣rs+bs∣22=E∣rs∣22−∣E[rs]∣22.\min_{b_s}\mathbb E\\|r_s+b_s\\|_2^2=\mathbb E\\|r_s\\|_2^2-\\|\mathbb E[r_s]\\|_2^2.

Thus, a nonzero mean residual after base quantization leaves room for an additional reduction in local reconstruction error. Our combination experiment also demonstrates an improvement in downstream scale prediction, providing direct evidence that bsb_s complements QuantVGGT and is not fully subsumed by it in the evaluated setting.

Reviewer_FFdsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the post-training quantization for feedforward geometry transformer. Different tokens share the same activation quantization, but the same quantization may have different impacts to the output. This paper use MapAnything as example. The scale tokens are few but significantly affact the final results. This paper proposes a orthogonal rotation, token-uniform boas correction, and scale-token specific correction to improve the overall quality. Experiments show near-full precision performance under W4A8.

Critical strengths
  1. This paper is well-motivated for the importance of the token importance
  2. The proposed method is simple but effective. The ablations show effective improvement. Although on the VGGT and DA3, the correction doesn't have noticeable improvement, which is still consistent to the paper's analysis.
Critical weaknesses
  1. A direct alternative is retaining the higher precision for the important tokens like the scale token. It would be better if this paper can use it as a comparison and demonstrate why it is not suitable for the issue.
  2. The mapanything configuration uses gradient-based refinement for the correction parameters, which introduced another factors, making the ablation not accurate. It would be better if reporting the results before and after the refinement. It would help isolate the benefit of the token-specific correction itself.
  3. The validated feedforward geometry transformer methods are limited. Currently the paper tested on MapAnything, vggt, and da3. it would be better if testing on more models with different token designs.
Questions for authors
  1. How sensitive are the correction vectors to the calibration data? Do corrections calibrated on indoor scenes transfer reliably to outdoor scenes?
2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for the constructive comments on higher-precision alternatives, the contribution of special-row correction, and transfer across architectures and calibration settings. We address these concerns through controlled comparisons, additional MoGe-2 experiments, and measurements of deployment costs.

R1.1 Question: Why not retain the special token at higher precision?

Response: Retaining the special token in FP16 achieves the same reported AUC@30 (98.53%) as QMapAnything but increases latency by 2.90% and peak memory usage by 12.24% in our W8A8 implementation on an L4 GPU (Table 9, Appendix F.2). The FP16 branch stores an additional FP16 copy of the weights and performs a separate single-row matrix multiplication. In comparison, adding bsb_s to the baseline (R+βR+\beta) increases latency by only 0.19 ms, while measured peak memory remains unchanged at 3.563 GB.

Table 9. W8A8 deployment with integer computation on L4.

MethodScale AbsRel ↓Unaligned depth AbsRel ↓AUC@30 (%) ↑Latency (ms) ↓Peak memory (GB) ↓
Baseline (R + β)0.0710.09598.47218.723.563
QMapAnything: R + β + bₛ0.0710.09698.53218.913.563
Special-token FP16 (no bₛ)0.0710.09698.53225.25 (+2.90%)3.999 (+12.24%)
Dynamic per-token INT8 (no bₛ)0.0710.09698.80240.31 (+9.78%)3.595 (+0.90%)
QuantVGGT0.0720.09698.53238.67 (+9.03%)3.595 (+0.90%)

R1.2 Question: How can the contribution of bsb_s be separated from gradient-based refinement?

Response: The contribution of bsb_s is observable even without gradient-based refinement. Under closed-form calibration alone, adding bsb_s reduces scale error from 0.30 to 0.26, increases AUC@30 from 87.63% to 88.67%, and reduces depth error from 0.14 to 0.13 (Table 8, Appendix F.1). With the same refinement procedure, adding bsb_s further improves AUC@30 from 91.14% to 93.60%. These comparisons hold the rotation, quantized weights, and quantization scales fixed, isolating the incremental contribution of the special-row correction.

Table 8. W4A4 comparison with indoor calibration and outdoor evaluation.

MethodScale rel ↓AUC@30 (%) ↑Depth rel ↓
Original FP (BF16)0.0798.570.09
Baseline-CF0.3087.630.14
QMapAnything-CF0.2688.670.13
Baseline-Ref0.1791.140.14
QMapAnything-Ref0.1593.600.13
Special-token FP160.2588.940.13
QuantVGGT0.1592.960.13

R1.3 Question: Does the method transfer to architectures with different token designs?

Response: Yes. The correction transfers to MoGe-2, whose CLS token is updated in every encoder block. Under W4A4 quantize-dequantize evaluation, adding bsb_s reduces mean metric-depth AbsRel, scale-aligned depth AbsRel, and absolute log-scale drift by 12.0%, 15.0%, and 44.5%, respectively, across three calibration seeds on the same 462 test images (Table 6, Section 4.6). We have added these results and the experimental details to the revised manuscript.

Table 6. MoGe-2 W4A4 QDQ on 462 images (quantized rows: three-seed mean ± SD).

MethodMetric depth AbsRel ↓Scale-aligned depth AbsRel ↓Absolute log-scale drift ↓
FP0.1930.072—
Baseline (R + β)0.225 ± 0.0060.100 ± 0.0030.137 ± 0.001
QMapAnything0.198 ± 0.0030.085 ± 0.0010.076 ± 0.003

R1.4 Question: How sensitive are the corrections to the calibration data, and does indoor calibration transfer to outdoor scenes?

Response: Adding bsb_s yields lower observed variability across the evaluated MoGe-2 calibration runs and improves outdoor performance after indoor calibration on MapAnything. Across three calibration seeds on MoGe-2, the observed standard deviation of metric-depth AbsRel is 0.003 for QMapAnything versus 0.006 for the baseline (Table 6, Section 4.6). With indoor closed-form calibration and outdoor evaluation on MapAnything, the baseline and QMapAnything yield scale error (0.30 vs. 0.26), depth error (0.14 vs. 0.13), and AUC@30 (87.63% vs. 88.67%) (Table 8, Appendix F.1).

Thank you for the response and the additional experiments. The higher-precision comparison and ablations of separating correction from refinement, and indoor-to-outdoor results addressed my questions. I will maintain my weak accept rating.

Comments on the paper

1 comment

Dear Area Chair and Reviewers, We sincerely thank the Area Chair and all reviewers for their time and constructive feedback. The discussion motivated additional controlled experiments and helped us clarify the independent contributions, structural applicability, and deployment costs of QMapAnything. We summarize the main evidence below. Changes in the revised manuscript are marked in red.

  1. Independent contribution of special-row correction (R1.2; R3.1). With the rotation, quantized weights, and quantization scales fixed, adding bₛ under closed-form calibration reduces scale error from 0.30 to 0.26, reduces depth error from 0.14 to 0.13, and improves AUC@30 from 87.63% to 88.67%. With the same refinement procedure, AUC@30 improves from 91.14% to 93.60%. These controls isolate gains of 1.04 and 2.46 percentage points from the correction, before and after refinement, respectively (App. F.1, Table 8).
  2. Transfer across architectures and structural applicability (R1.3; R2.1–R2.2; R3.3; R2 follow-up Q1). On MoGe-2, adding bₛ reduces metric-depth AbsRel, scale-aligned depth AbsRel, and absolute log-scale drift by 12.0%, 15.0%, and 44.5%, respectively, across three calibration seeds on 462 test images (Sec. 4.6, Table 6; App. F.3). MoGe-2 uses a nonlinear CLS scale head and updates CLS in every encoder block. These results demonstrate that correction does not require a linear scale head or sparse token updates. We clarified that S2 concerns multiplicative coupling through the final scale, while S4 guides correction placement.
  3. Diagnostic value and residual correctability (R2.1; R3.1–R3.2). We distinguish risk screening with Γ from residual correctability assessment with η. On MoGe-2, the median Γ is 25.651 across 1,536 valid scene-layer records. Fitting the correction on one scene fold and evaluating it on another reduces residual energy by 24.57% and 24.21% in the two directions, with improvements at 94/96 locations in each direction. Separately, a 34.2% reduction in held-out special-row MSE corresponds to only 0.010% in whole-block MSE, demonstrating why token-averaged reconstruction loss can obscure errors in an output-critical row.
  4. Complementarity with QuantVGGT (R2.3; R2 follow-up Q3). Following the reviewer’s suggestion, we evaluated QuantVGGT combined with bₛ. Scale relative error decreases from 0.14945 to 0.13766, a 7.89% reduction, and depth relative error decreases from 0.12926 to 0.12775, a 1.17% reduction. AUC@30 increases from 92.96% to 93.04%. This combination experiment directly demonstrates additional benefit beyond QuantVGGT in the evaluated setting, particularly on the scale prediction targeted by our correction.
  5. Measured deployment advantages across view counts (R2.3; R2 follow-up Q2). Under W8A8, QMapAnything reduces two-view latency from 238.67 to 218.91 ms, an 8.28% reduction, at the same reported AUC@30 of 98.53% (App. F.2, Table 9). In the additional four-view comparison with a unified build configuration, latency decreases from 492.36 to 457.85 ms, a 7.01% reduction, and peak memory decreases from 4.284 to 4.161 GB, a 2.87% reduction. Four-view scale AbsRel also decreases by 2.20%, while AUC@30 differs by only 0.0111 percentage points.
  6. Higher-precision alternatives and incremental overhead (R1.1; R3.4). In the evaluated W8A8 implementation, retaining special-token computation in FP16 achieves the same reported AUC@30 as QMapAnything but increases latency by 2.90% and peak memory by 12.24%. Dynamic per-token INT8 increases latency by 9.78% and memory by 0.90%. By comparison, adding bₛ to the shared baseline increases latency by only 0.19 ms, with peak memory unchanged at 3.563 GB (App. F.2, Table 9).
  7. Calibration and evaluation clarifications (R1.4; R2.3). The controlled MapAnything gains in item 1 use indoor calibration and outdoor evaluation. Across three MoGe-2 calibration seeds, the observed standard deviation of metric-depth AbsRel is 0.003, compared with 0.006 for the baseline. We also explicitly distinguish W4A4 fake-quantization accuracy from W8A8 measured deployment, so the timing results are not presented as W4A4 integer-inference measurements.

We guarantee the release of the code and configuration files. We welcome any further questions from the Area Chair and reviewers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.