acceptodds
Under review as a conference paper at ICLR 2027

Beyond Linear Scaling: Recurrent Calibration for Cross-Resolution Transfer in Pretrained Visual Mamba Models

Abstract

Mamba has emerged as a leading paradigm for long-sequence modeling and has been widely extended to vision. Despite its linear complexity and appeal for high-resolution representations, pretrained visual Mamba models often degrade severely at unseen resolutions. In this paper, we reveal that resolution scaling in visual Mamba is not merely a one-dimensional sequence extension, but entails a fundamental change in the two-dimensional spatial rate of pretrained selective dynamics. The resulting structural mismatch propagates through state-dependent residual updates and hierarchical transitions. To quantify this resolution-induced trajectory drift, we formulate Cross-Resolution Trajectory Discrepancy (CRTD). Pre-calibration analysis reveals a strong association between raw trajectory discrepancy and cross-resolution accuracy degradation. Based on these insights, we propose Architecture-Native Uniform Residual/Update Calibration (Uniform-RU), a lightweight and label-free framework that calibrates recurrent dynamics and residual trajectories using only a small fraction of the training images while keeping the pretrained backbone frozen. Across diverse visual Mamba architectures and resolutions, Uniform-RU substantially restores cross-resolution accuracy on image classification and generalizes well to detection and segmentation tasks. These results demonstrate that recurrent calibration can effectively extend pretrained visual Mamba models, offering a dynamics-centric perspective for resolution transfer beyond sequence scaling. Code is available at https://anonymous.4open.science/r/Uniform-RU/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
State Space Models, Visual Mamba, Resolution Generalization, Model Calibration, Cross-Resolution Transfer

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7ss1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies why pretrained visual Mamba models degrade at resolutions beyond pretraining. It attributes this to changes in spatial recurrent rate and consequent mismatches in recurrent states and residual updates. It defines Cross-Resolution Trajectory Discrepancy (CRTD) and proposes Uniform-RU, combining deterministic spatial-dynamics transport with learned carried-state and update corrections. The backbone stays frozen and calibration uses unlabeled images. Evaluation spans visual Mamba architectures on ImageNet classification, COCO detection, and ADE20K segmentation.

Critical strengths
  1. The evaluation is broad across model families, resolution shifts, and downstream tasks, including classification, detection, and segmentation. The inclusion of an output-only control and component-level comparisons also shows a serious effort to distinguish the proposed mechanism from simpler alternatives.
  2. The method has a practical adaptation profile: it requires no class labels and keeps the pretrained backbone frozen. This makes the approach relevant to settings where unlabeled target-resolution data are available but retraining the original model is undesirable.
Critical weaknesses
  1. Calibration cost is non-negligible: 8,192 unlabeled calibration images, 1,024 model-selection images, eight epochs, and a separate calibrator per target resolution. Added parameters range from 0.5M to 8.0M, with additional FLOPs. Wall-clock and hardware costs should be reported.
  2. CRTD reduction and accuracy recovery occur together, while calibration explicitly enforces internal and output consistency. The output-only control helps, but does not fully isolate each proposed architectural mechanism.
Questions for authors
  1. What are the wall-clock and hardware costs per target-resolution calibrator, including native-resolution reference passes and model selection?
  2. Can the authors quantify the additional value of trajectory supervision over the output-only control across more than ViM-T and VMamba-T?
Reviewer_SDwLOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Uniform-RU for resolution transfer in pretrained visual Mamba models. It combines deterministic Spatial Dynamics Transport (SDT) with learned Carried-State Correction (CSC) and Additive-Update Correction (UC). Calibration keeps the backbone frozen and uses unlabeled native-resolution trajectories and predictions as supervision. The experiments cover classification, segmentation, and detection. Each target resolution is normally calibrated separately, and standard inference uses only the target-resolution branch.

Critical strengths
  1. Table 1 (p.7) shows VMamba-T recovering from 38.8% to 78.4% Top-1 at 1280² and VSSD-Micro from 0.8% to 76.3%. These gains make the method promising for high-resolution deployment, particularly given the study's coverage of seven architecture families and 11 backbone configurations.

  2. Keeping pretrained weights frozen and using unlabeled images makes the approach useful for adapting existing models, especially since the reference branch is removed at inference. The appendix also examines calibration through component ablations, matched output-only supervision, downstream evaluations, and calibrator-transfer experiments.

Critical weaknesses
  1. At 1024² in Table 4, SDT alone lowers VMamba-T from 57.4% to 47.4%, while CSC+UC without SDT already reaches 79.2%, close to the full method's 79.4%. This makes it difficult to attribute much of this model's recovery to SDT. ViM-T does benefit, improving from 38.2% without SDT to 52.2% with it, so the interaction clearly matters for that architecture. The paper acknowledges that SDT alone is insufficient, but it is still unclear how the results support a common causal explanation. The local-rate analysis cancels a geometric factor without establishing that it principally causes task errors, and ViM's SDT changes Conv1d dilation alongside recurrent-step scaling.

  2. The existing output-only control helps address the need for internal supervision, but the reported VMamba-T comparison gives only CRTD, without corresponding Top-1. The missing accuracy result leaves open whether the lower internal discrepancy translates into better task performance. It is also unclear whether ordinary adapters, feature distillation, or learned scale-only calibration could achieve similar recovery with the same unlabeled supervision and matched budgets. Without those comparisons, the added value of the proposed correction structure is hard to judge. Despite the substantial recovery results, these gaps and the mechanism issue tip the borderline assessment toward rejection.

Questions for authors
  1. What are the separate effects of recurrent-step scaling and convolution-dilation changes for ViM-T, and which architecture-specific conclusions do the ablations support? Isolating these effects would help assess the proposed mechanism.

  2. Please provide VMamba-T's Top-1 for the existing output-only control, along with a comparison of the proposed method against a simpler adapter or scale-only calibrator, matched for data, supervision access, trainable parameters, and computation.

  3. The exact internal loss equations, reductions, and pre/post-correction tensor locations need clarification. These details are necessary to understand what the proposed calibration and its controls optimize.

Reviewer_mP2vOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates degradation when pretrained visual Mamba models are evaluated at higher image resolutions. It interprets resolution changes as changes in the spatial execution rate of selective recurrence and measures internal drift using Cross-Resolution Trajectory Discrepancy (CRTD). The proposed Uniform-RU combines deterministic Spatial Dynamics Transport (SDT) with learned Carried-State Correction (CSC) and Additive-Update Correction (UC). The pretrained weights remain frozen, while correction modules are calibrated using unlabeled native/target-resolution image pairs. Evaluation spans multiple visual Mamba families, ImageNet classification, ADE20K segmentation, and COCO detection and instance segmentation.

Critical strengths
  1. Thorough and convincing experimental validation. The evaluation covers multiple visual Mamba families, model scales, and target resolutions, and extends from ImageNet classification to ADE20K segmentation and COCO detection and instance segmentation. The recovery can be substantial: VMamba-T Top-1 at 1024 increases from 57.4% to 79.4% (Table 4), while VSSD-Micro at 1280 increases from 0.8% to 76.3% (Table 1). Component ablations, matched output-only controls, trajectory diagnostics, and the single-calibrator transfer study provide complementary evidence about effectiveness and its limits. This breadth and depth make the empirical contribution persuasive.
  2. A research perspective with value beyond the proposed calibration method. Interpreting resolution changes through the spatial execution rate of selective recurrence offers a useful way to study why linear computational complexity does not ensure resolution generalization. The distinction between geometry-induced transport and residual state/update correction, together with CRTD, provides concrete hypotheses and tools for investigating cross-resolution behavior. Although the relative importance of these factors varies across architectures, this perspective opens a worthwhile research direction for understanding and adapting pretrained visual state-space models.
Critical weaknesses
  1. The mechanism and the necessity of trajectory supervision are not equally supported across architectures. At 1024, VMamba-T falls from 57.4% to 47.4% with SDT alone, while CSC+UC without SDT reaches 79.2%, nearly the full model's 79.4% (Table 4). ViM-T shows a larger SDT contribution, so the interpretation should be architecture-dependent. Appendix A.6 verifies the local rate cancellation, but not that this factor dominates task-level degradation. Table 8 reports CRTD improvements for a matched output-only control on VMamba-T, yet the corresponding Top-1 comparison is absent from Table 7. Since internal alignment is supervised, lower CRTD alone does not establish a better task solution. This limits the strength of the mechanistic contribution.
  2. The deployment and recovery claims need clearer qualification. Classification normally fits a separate calibrator for each target resolution using eight epochs over 8,192 images; downstream tasks also fit separate calibrators. Parameters and FLOPs are reported, but calibration time and memory remain unclear. Recovery is not uniform: VMamba-B ADE20K at 896 changes from 51.0 raw to 50.5 with RU; VMamba-S COCO box AP at R1024 changes from 49.3 to 49.2. Furthermore, Appendix A.1 describes restriction/lifting as oracle-only while deployment equations also use these operations. These issues affect practical reproducibility and the scope of the claimed gains, although they do not negate the large classification improvements.
  3. The geometric scope is narrower than general resolution adaptation. As acknowledged in Appendix A.7, the analysis focuses on isotropic resolution changes; anisotropic resizing and aspect-ratio variation are discussed as extensions. These changes induce direction-dependent recurrent execution rates and therefore directly test the proposed geometric interpretation, rather than merely adding another dataset. The current evidence does not establish how the transport and correction should behave in those settings. This is a limitation of the demonstrated applicability, rather than a contradiction of the strong results under the evaluated scaling protocol.
Questions for authors
  1. Clarification of existing results: What are the VMamba-T Top-1 results for the matched output-only control already evaluated in Table 8? Together with Table 4, do they support a narrower, architecture-dependent account of SDT and trajectory supervision?
  2. Implementation clarification: What is the exact restriction/lifting operation used during deployed inference? Is the oracle-only statement in Appendix A.1 intended to refer to the correction target rather than the mappings themselves?
  3. Deployment clarification: What calibration time and peak memory are required for representative target resolutions? This would allow the practical cost of target-specific calibration to be assessed alongside the reported inference FLOPs.
1 comment

Thank you to the authors for their response. I find the ablation studies on the capabilities of different model components and the comparison of parameter counts presented in the rebuttal very meaningful, as they provide further evidence of the effectiveness of the proposed method. Therefore, I will maintain my original positive assessment of this work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.