acceptodds
Under review as a conference paper at ICLR 2027

GradPath: Task-Sensitive Pathway Tuning for Medical Image analysis

Abstract

Parameter-efficient fine-tuning (PEFT) commonly adapts pretrained models either by inserting newly initialized modules at manually specified locations or by updating independently selected pretrained weights. The former introduces additional computation and placement sensitivity, while the latter overlooks the structural coupling within pretrained networks. We propose GradPath, a structured PEFT method that treats each hidden pathway as a paired adaptation unit consisting of an input-projection row and its corresponding output-projection column. GradPath estimates pathway importance using an activation–gradient sensitivity score and jointly updates the paired projections of the selected pathways. The selected pathways form a bottleneck-shaped subnetwork within the pretrained transformation, without introducing new projection matrices. We further develop GradPath-S, which enhances selected spatial pathways using gated multi-kernel depthwise convolutions. Extensive experiments on medical image classification, segmentation, and few-shot learning demonstrate strong performance across diverse modalities, architectures, and supervision levels. GradPath further reduces training memory, computation, and latency. These results establish structured pretrained pathways as an effective and efficient unit for parameter-efficient medical image adaptation.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Parameter-efficient fine-tuning, Adapter, Medical Image Analysis, GradPath

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_JcmTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes GradPath, a parameter-efficient fine-tuning method for medical image analysis. Each hidden unit of a feed-forward block is treated as a "pathway" made of one row of the input projection and one column of the output projection. A score, the absolute product of the unit's activation and the loss gradient averaged over the training set, ranks the units, and the paired row and column of the top K units in every block are updated in place, without new modules. A variant, GradPath-S, adds gated depthwise convolutions with three kernel sizes on the selected channels. With ViT-B on seven classification datasets, GradPath and GradPath-S reach 90.96% and 91.55% average accuracy, against 90.77% for GPS and 90.70% for Adapter. Further experiments cover Swin-T and ConvNeXt-T, five segmentation datasets with DINOv3, few-shot adaptation of BiomedCLIP on seven datasets, reduced training-data ratios and VTAB-1K. The paper also reports lower training memory, FLOPs and time per iteration than full fine-tuning, GPS and DKA, and argues that the selected pathways are task-specific.

Critical strengths
  1. The method is simple and clean. A hidden unit with its input row and output column is a natural adaptation unit, the pretrained weights are updated in place, and no module is added, so the inference graph is unchanged and the idea carries over to Swin and ConvNeXt.
  2. The evaluation is broad: classification on seven datasets and three backbones, segmentation, few-shot adaptation of a vision-language model, reduced-data settings and VTAB-1K, with analyses of block-wise importance and of transferring selected pathways between tasks.
  3. Efficiency is measured as peak memory, FLOPs and time per iteration with a forward/backward breakdown, and not only as a parameter count.
Critical weaknesses
  1. The evidence for the two central ideas, pairing and task-sensitive selection, is thin. In Table 5, the four alternative selection or update schemes differ from GradPath by at most 0.7 points on three datasets, and on BTMRI the independent variant gives the same accuracy. Randomly selected pathways are not tested in the fully supervised setting. In Fig. 4, a Jaccard overlap of 0.18 to 0.27 is called low, but two random sets of 64 out of 3072 units would overlap by about 0.01, so a third or more of the selected pathways are shared between unrelated tasks. On three of the four datasets, pathways selected on other tasks recover 60 to 76% of the gain of target-specific over random pathways. The score itself is the first-order Taylor importance known from pruning (Molchanov et al., 2017), and structured sparse fine-tuning that selects FFN channels and updates pretrained weights in place (S2FT, Yang et al., 2024) is not discussed. The theory adds little: Proposition 1 follows from the definition of the score, and Appendix D holds for any fixed subset of parameters.
  2. The paper does not mention that the released training script first trains the classifier head in a warm-up stage before the pathway scores are computed. It is not stated whether the baselines were trained in the same way. The released scripts for GradPath and GradPath-S also differ in more than the spatial module (64 versus 32 pathways, no bias update versus bias update, one versus five warm-up epochs), so the gain attributed to the spatial enhancement may be confounded. Trainable parameters are not reported for Tables 1 and 4, and the tokens used for scoring are not specified. The efficiency comparison in Fig. 5 covers only full fine-tuning, DKA and GPS, leaves out Adapter, LoRA and SSF, and is measured for GradPath, while the best accuracies come from GradPath-S.
Questions for authors
  1. Were the baselines trained with the same protocol as in the released code (classifier warm-up, weight decay, validation-driven schedule, early stopping), and were learning rates tuned per method? Why are LoRA and VPT below linear probing?
  2. Where do the SSF and GPS rows in Table 9 come from? The GPS paper reports, for example, 81.1 on CIFAR-100 and 84.0 on dSprites-Loc, against 71.8 and 71.8 here.
  3. Is the classifier trained before the pathway scores are computed, and how is this handled for the randomly initialised segmentation decoder? Which settings (K, bias update, warm-up) were used for GradPath-S in Table 1, and what does GradPath-S give with the settings of GradPath?
  4. What is the accuracy with randomly selected paired pathways in the setting of Table 5? Why do the target-specific accuracies in Fig. 4(a) (79.3, 82.7, 73.1, 79.6) differ from the 8-shot GradPath results in Table 4 (82.0, 81.6, 75.2, 80.1)?
  5. The dataset splits follow the 11-dataset benchmark of BiomedCoOp. Why are different subsets of seven used for classification and for few-shot learning, and why is BUSI not used?
2 comments

Thank you for the thorough and extensive rebuttal. You have addressed all of my concerns to my satisfaction. Accordingly, I will raise my recommendation from 2 to 3, as I now have considerably greater confidence in the paper's contribution and merits.

Thank you for your thoughtful reassessment and encouraging feedback. We are glad that our responses have addressed your concerns. Your constructive comments have helped us improve the clarity and completeness of the paper.

Best regards, The Authors

Reviewer_9RtTOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

GradPath is a structured parameter-efficient fine-tuning method for medical imaging. It treats the paired input-projection row and output-projection column within pretrained FFNs as a complete pathway, uses activation-gradient sensitivity scores to select Top-K pathways for joint tuning, and forms an in-place bottleneck subnetwork without introducing new projection matrices or altering the inference graph. Its GradPath-Spatial variant further applies gated multi-kernel depthwise convolutions to the selected channels. The paper validates its parameter efficiency and cross-backbone effectiveness across classification, segmentation, and few-shot tasks, supported by theoretical analysis and ablation studies.

Critical strengths
  1. Abstracting FFN neurons into hidden pathways that pair input rows with output columns is mathematically equivalent to an in-place bottleneck adapter, elegantly balancing structured fine-tuning with weight reuse while eliminating the sequential hardware latency inherent in external adapters.
  2. The theoretical derivation is solid and comprehensive, mathematically proving the local optimality of selecting pathways based on activation-gradient sensitivity via a first-order Taylor expansion, while rigorously bounding the low-rank perturbation and establishing subspace convergence.
Critical weaknesses
  1. The classification performance of baseline LoRA in Table 1 is abnormally low and exhibits severe collapse on certain datasets, falling substantially behind linear probing and suggesting severe underfitting or suboptimal hyperparameter tuning for the baseline.
  2. The pathway sensitivity scores rely on a one-shot backward pass over the entire training set prior to fine-tuning, ignoring potential dynamic drift in feature importance as weights evolve, and this initial calibration overhead is omitted from the end-to-end training cost.
  3. The spatial variant GradPath-S introduces multi-scale depthwise convolutions and gating parameters, essentially degenerating into an external micro-adapter branch and compromising the theoretical purity of zero additional architecture and native computation graph reuse.
  4. The medical image segmentation evaluation is overly narrow by relying solely on the overlap metric IoU, omitting boundary distance metrics such as the 95th percentile Hausdorff Distance (HD95) or Average Surface Distance (ASD) essential for clinical contour assessment.
  5. The core mechanism of paired pathway tuning is inherently domain-agnostic and broadly applicable to general vision tasks, making its branding as an exclusively medical method somewhat disconnected from its foundational formulation.
Questions for authors
  1. Please detail the exact module placement, rank, scaling factor, and hyperparameter tuning protocol for LoRA in Table 1 to clarify the cause of its severe performance drop on medical classification tasks.
  2. Please report the specific wall-clock time and memory overhead required by the initial full-dataset gradient pass for pathway discovery, and discuss whether the selected pathways experience significant importance drift during training.
  3. Could the authors provide boundary distance metrics such as HD95 or Average Surface Distance (ASD) for the segmentation benchmarks in Table 3?
  4. Please provide the empirical inference latency and GPU memory overhead introduced by GradPath-S compared to the base GradPath model.
Reviewer_i9G1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GradPath is a structured parameter-efficient fine-tuning method that selects FFN hidden units using activation–gradient sensitivity and jointly updates their input-projection rows and output-projection columns. It reuses pretrained pathways without adding projection matrices. GradPath-S additionally introduces gated multi-scale depthwise convolutions on selected spatial channels. Experiments cover medical classification, segmentation, few-shot adaptation, multiple backbones, and VTAB-1K, with reported improvements in accuracy and training efficiency.

Critical strengths

Treating the two projections of an FFN hidden unit as a paired adaptation unit is a meaningful and well-motivated design choice. It offers a structured way to reuse pretrained transformations rather than selecting unrelated weights or introducing external modules. This is a useful research direction even though the sensitivity heuristic itself is established.

The evaluation extends beyond full-data medical classification to segmentation, limited-data adaptation, different architectures, and non-medical transfer. The stronger reported few-shot and VTAB results are encouraging. Comparisons with single-projection, independent-selection, random-path, and transferred-path variants also provide evidence relevant to the proposed design.

Critical weaknesses

The evidence does not yet sufficiently establish a robust advantage attributable to paired pathway tuning. Plain GradPath improves the Table 1 average over GPS by only 0.19 percentage points; the larger GradPath-S gain also includes an additional spatial module. The more encouraging limited-data results do not resolve the missing uncertainty estimates for the main comparisons and pairing ablations. Moreover, LoRA performs substantially below linear probing in Table 1, making its configuration and method-specific tuning important to clarify. Matched-budget, repeated comparisons are needed to determine whether the core design reliably improves upon strong alternatives.

Reproducing the task-sensitive selection and efficiency claims requires clearer operational details. For classification and segmentation, selection gradients depend on a task head whose state before discovery is unclear; the sensitivity of the selected paths and downstream performance to head initialization and calibration sampling is not established. The paper attributes computational savings to avoiding dense weight gradients, but the parameterization/backward implementation and total cost accounting, including pathway discovery, are insufficiently described. These are central to the proposed accuracy–efficiency tradeoff, not merely peripheral implementation choices.

Questions for authors

What is the task head’s state before pathway discovery, and how stable are selection and accuracy across head initialization and calibration samples? For few-shot experiments, please confirm that discovery uses only each run’s permitted support set.

Can you report matched-seed uncertainty for plain GradPath versus GPS and the pairing ablations, under comparable trainable-parameter budgets, and specify LoRA’s target modules, rank, scaling, and tuning protocol?

How does the implementation avoid dense gradients for frozen weights while retaining the required input gradients? Please report the timing protocol and total adaptation cost, including discovery, rather than only per-iteration savings.

Comments on the paper

1 comment

Thank you for reconsidering your assessment and updating your recommendation. We appreciate your time and constructive feedback, which have helped us strengthen the paper.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.