acceptodds
Withdrawn from ICLR 2027

TeleScopeMix: Depth-Aware Spatial and Channel Adaptation of Vision Transformers for Medical Imaging

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Adapting pretrained Vision Transformers to medical imaging is challenging when target datasets are small, imbalanced, and drawn from modalities that differ substantially from natural images. We introduce TeleScopeMix, a parameter-efficient adapter that combines depthwise spatial filtering, efficient cross-channel mixing, and a depth-dependent capacity schedule. Our analysis of supervised and self-supervised backbones indicates that adapter utilization generally increases toward later Transformer blocks, motivating greater channel-mixing capacity at depth. Across various medical-image classification datasets and Vision Transformer backbones, TeleScopeMix with optimizing fewer than of the backbone parameters achieves comparable or even better performance compared to full fine-tuning method.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Parameter Efficient Finetuning, Vision Transformers, Medical Imaging, Depthwise Convolutional Adapters, low rank, grouped convolutions, transfer learning, self-supervised models, supervised models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RTV1OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

In this work, the authors focus on PEFT methods for medical image classification. They propose ScopeMix and TelescopeMix, which use convolution-based adapters. The utility of these methods is demonstrated primarily on the HAM10000 dataset, while eight additional medical datasets are used to compare TelescopeMix with full fine-tuning in an ablation study.

Critical strengths

The authors focus on an important and highly relevant problem with direct implications for medical AI practitioners. Overall, the paper is clear, well-structured, and easy to read.

Critical weaknesses

Below I list my main concerns:

  • Limited novelty: The proposed methods appear to offer only incremental improvements over existing approaches, which may not be sufficient to meet the novelty threshold for an ICLR publication.

  • Narrow scope of related work: The related-work section focuses primarily on convolution-based PEFT methods, which represent only a relatively small subset of the broader PEFT literature. The discussion should be expanded to include a wider range of relevant approaches.

  • Insufficient comparison with relevant baselines: Comparisons with relevant SOTA, such as [1], are missing. In particular, [2] appears to follow a similar underlying philosophy and should be included. More critically, these comparisons are conducted only on HAM10000, limiting the strength and generalizability of the conclusions.

  • Limited task scope: The evaluation focuses exclusively on classification tasks.

  • Limited evidence for the telescoping component: In Tables 3 and 4, the performance differences between the telescoping and non-telescoping variants are consistently small across all model variants. This raises questions about the practical contribution of the telescoping component.

  • Incomplete efficiency analysis: Parameter efficiency alone does not fully characterize computational efficiency. The authors should also report metrics such as memory footprint, training time, and inference wall-clock time to provide a more comprehensive assessment of the proposed methods.

References

[1] Chen, S., Ge, C., Tong, Z., Wang, J., Song, Y., Wang, J., & Luo, P. (2022). Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in neural information processing systems, 35, 16664-16678.

[2] Mishra, U., Shukla, V., Hambarde, P., & Shukla, A. (2026, March). Improvise, Adapt, Overcome—Telescopic Adapters for Efficient Fine-tuning of Vision Language Models in Medical Imaging. In 2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 7605-7615). IEEE.

Questions for authors

In Table 2, although ScopeMix outperforms full fine tuning on three metrics by approximately 1-1.5% at best, it also shows a substantial 4% decrease in macro-precision and a smaller decline of more than 1% in macro F1. What might explain this variability across the evaluation metrics?

Reviewer_dFcBOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that medical imaging datasets are often small and class-imbalanced, which makes full fine-tuning computationally expensive and prone to overfitting. It further notes that existing parameter-efficient fine-tuning (PEFT) methods, such as LoRA, lack an explicit 2D spatial locality inductive bias, which motivates a PEFT approach that incorporates spatial information.

To address this limitation, the authors propose ScopeMix, a convolutional adapter for frozen Vision Transformers, and further introduce TeleScopeMix, which adjusts the number of tunable parameters across network depth by varying the number of groups in the pointwise convolution (G = 32/16/8).

The proposed methods are evaluated on HAM10000 with four backbones and on eight additional medical imaging datasets spanning several modalities. The authors report performance comparable to full fine-tuning while updating fewer than 1% of the model parameters.

Critical strengths
  1. Practical effectiveness of the proposed approach.

On HAM10000, ScopeMix clearly outperforms the depthwise convolution (DWConv) adapter of Zhang et al. (2025) (e.g., balanced accuracy 0.8373 → 0.8685 and macro F1 0.8007 → 0.8706 with G = 8; Table 2). This suggests that ScopeMix could serve as a practical PEFT method for medical imaging applications.

  1. Extensive evaluation across multiple backbones and datasets.

The proposed methods are evaluated with several backbone architectures, covering both supervised and self-supervised pretraining, and on multiple medical imaging datasets. These experiments suggest that the effectiveness of the approach is not limited to a single benchmark or backbone, providing some evidence of its generalizability.

Critical weaknesses
  1. Insufficient evidence supporting depth-aware telescoping, a key contribution of the paper.

TeleScopeMix does not consistently preserve the performance of uniform ScopeMix: it underperforms on most metrics for DeiT, Swin, and iBOT (Table 4), contrary to the conclusion. The importance of the direction of depth-wise parameter allocation is not validated, as no controlled comparison such as the reversed schedule (G = 8/16/32) is reported. Limited novelty and insufficient validation of the fixed telescoping schedule: closely related allocation methods (AdaLoRA, SPT) are not cited, and the schedule is applied unchanged across datasets without a uniform ScopeMix baseline in Table 5.

  1. Insufficient comparisons with existing methods and inaccurate interpretations of experimental results.

Limited comparisons with relevant PEFT methods, such as Convpass (already cited in the paper), AdaLoRA, and SPT. Several claims in the main text are not fully supported by the reported results

Questions for authors

Questions

  1. Effect of the telescoping direction. If the reversed schedule (G = 8/16/32, with the same number of trainable parameters) is evaluated on all four backbones, does the proposed telescoping schedule still provide consistent benefits?
  2. Performance degradation and backbone-specific characteristics. Why does TeleScopeMix underperform uniform ScopeMix on most metrics for DeiT, Swin, and iBOT (Table 4), even though the paper concludes that the schedule preserves predictive performance? Figure 2 and Section 4.4 indicate that the CKA similarity declines at different depths for supervised and self-supervised backbones. Could applying the same fixed schedule (G = 32/16/8) to both groups explain this degradation? Since iBOT, a self-supervised backbone, also degrades, are there other contributing factors? What motivates using the same schedule for supervised and self-supervised backbones despite their different CKA patterns?
  3. Uniform ScopeMix baseline across datasets. Could the authors report results for uniform ScopeMix (G = 8/8/8) in Table 5? This comparison is necessary to assess whether depth-aware telescoping provides consistent improvements across datasets.
  4. Comparison with existing PEFT methods. Have the authors compared the proposed method with Convpass, AdaLoRA, or SPT? AdaLoRA (Zhang et al., ICLR 2023) and SPT (He et al., ICCV 2023) also study depth-dependent capacity allocation but are not cited. Such comparisons would help clarify the contribution of the proposed approach relative to existing methods.
  5. Evidence for the benefit of the spatial inductive bias. Section 4.6 attributes the higher macro AUROC over full fine-tuning to the local spatial inductive bias. Given that LoRA, which has no explicit spatial bias, also achieves higher macro AUROC than full fine-tuning on all backbones in Table 6, what evidence supports attributing the improvement specifically to the local spatial inductive bias?
  6. Component-wise ablation. ScopeMix modifies the DWConv adapter in three ways at once (GroupNorm, PWConv, and the learnable scale α), whereas Section 4.3 attributes the improvement mainly to PWConv. Have the authors conducted ablations that isolate each component (e.g., removing GroupNorm or PWConv, or fixing α instead of learning it)?
  7. Dataset size and the benefit of PEFT. The introduction motivates PEFT for medical imaging by limited dataset sizes and class imbalance, which may make full fine-tuning less desirable. Does the proposed method provide a larger advantage over full fine-tuning as the training set becomes smaller? Since the largest improvements appear on relatively large datasets (e.g., OCTMNIST, with about 109K images), how do the authors reconcile these observations with the small-data motivation?

Clarifications

  1. CKA analysis (Figure 2) and backbone selection. The setup used to compute the CKA similarities is insufficiently described (e.g., the dataset and number of samples, the token representation, and the RBF kernel bandwidth). The criteria for selecting the four backbones in Table 4 from the 15 shown in Figure 2 are also unclear. In addition, the summary of the supervised backbones (a sharp decline at roughly one-third and two-thirds of the depth) does not reflect the considerable variation among their curves.
  2. Macro AUROC of full fine-tuning. On HAM10000, full fine-tuning has the highest accuracy and precision but the lowest macro AUROC in Table 2, below all PEFT methods. The paper does not analyze this discrepancy.
  3. Notation in Equation (1). xregx_{\mathrm{reg}} is only briefly described as "one or more register tokens," and RR is not defined. Please clarify their definitions and how these tokens are handled by the adapter.
  4. Claims in Section 4.3. The paper states that Grouped ScopeMix "achieves the highest overall performance while keeping similar parameter efficiency". However, its balanced accuracy is on par with Low-rank ScopeMix (r = 16; 0.8685 vs. 0.8688), while it uses more than twice as many trainable parameters (0.93M vs. 0.39M). Please revise this statement or clarify what "overall performance" refers to.
  5. Statements in Section 4.7. Several statements do not match Table 5. TeleScopeMix is said to "match" full fine-tuning on BloodMNIST, although it is about 0.6 percentage points lower on four of the five metrics. It is said to achieve a "higher ... F1 score" on Kvasir, although the F1 scores are identical (0.9461). Its "strongest gains" are attributed to histopathology datasets, whereas the largest improvement is on OCTMNIST (+3.24 percentage points in accuracy and +4.10 in F1). Please revise these statements.
Reviewer_RcxcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

In this paper, the authors first proposed a method named ScopeMix, which is a lightweight convolutional adapter that works with each frozen ViT block. It uses a depthwise convolution for spatial filtering, a grouped or low-rank pointwise convolution for channel mixing, a GroupNorm and a learnable per-layer scale. The authors then observe that adapted representations drift further from the pretrained ones in deeper layers, and used this to motivate TeleScopeMix, which gives early layers less channel-mixing capacity and later layers more. The experiments are provided on HAM10000 and eight other medical datasets, which show that TeleScopeMix is roughly competitive with full fine-tuning while training less than 1% of the parameters.

Critical strengths

In this paper, the authors proposed a method that is simple and easy to reproduce. The evaluation covers nine datasets across many imaging modalities and four backbones with both supervised and self-supervised pretraining. The results are averaged over five seeds. One interesting finding of the paper is that the adapters consistently beat full fine-tuning on AUROC and clearly outperform it on the large histopathology and OCT datasets, which suggests a useful regularization effect.

Critical weaknesses

Following are some of the main weaknesses that requires further exploration.

  • Novelty aspect of the paper is very limited. This area of adapters is already well-explored and well-saturated. ScopeMix is essentially the depthwise-conv adapter that is combined with standard components. The related comparison adapters such as Convpass, Conv-Adapter, and Mona are not compared against. Further prior work on depth-aware budget allocation, such as AdaLoRA and SPT are also not discussed.

  • The main claim of the paper that depth-aware allocation helps is not well supported. On DINOv3 the reported gains fall within one standard deviation. On the other three backbones, TeleScopeMix is actually worse than uniform ScopeMix in balanced accuracy and accuracy. The paper therefore needs clear re-writing and re-designing.

  • Results comparison also did not consider the linear probing and standard PEFT methods such as VPT or maybe AdaptFormer. The cross-dataset table is compared only against full fine-tuning which makes the comaprison invalid.

  • Some important details are missing or inconsistent. Can the authors clarify whether HAM10000 is split by lesion? if it is not that way, then the images of the same lesion can appear in both train and test, which results in data leakage. Furthermore, it is also unclear how the low-resolution MedMNIST images were upsampled to size of 384?

Questions for authors

Please refer to the weaknesses section. The paper overall would be strengthened by considering more comparative experiments with relevant baselines and re-considering the technical novelty of the method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.