acceptodds
Under review as a conference paper at ICLR 2027

RTAF: Response aware Tensor Adaptive Function Block for Efficient 3D Medical Image Analysis

Abstract

Volumetric medical features contain complex anatomical structures and simple repetitive responses, while dense 3D convolutions apply the same transformation to all feature responses. This uniform processing introduces unnecessary computation for simple responses and insufficient modeling capability for complex structures. Existing efficient operators mainly reduce redundancy or enhance feature interactions, but they lack a unified design that balances nonlinear representation and efficient local processing. We propose the Response Aware Tensor Adaptive Function (RTAF) block, a plug-and-play feature transformation module that dynamically combines nonlinear modeling and lightweight local processing according to feature characteristics. RTAF consists of three components: Router, Nonlinear Tensor Branch, and Local Convolution Branch. The Router generates an input-dependent modulation map that adjusts the inputs to the nonlinear and local processing branches.The Nonlinear Tensor Branch enables lightweight nonlinear feature modeling by combining Gaussian basis expansion with learnable centers and grouped CP-factorized tensor weights, while the Local Convolution Branch efficiently captures local spatial structures.Extensive experiments on six MedMNIST3D classification datasets and five Medical Segmentation Decathlon benchmarks demonstrate that RTAF achieves competitive performance across 3D ResNet-18/50 and nnU-Netv2, while introducing substantially lower parameter and computational costs than dense 3D convolution blocks.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
3D vision, Volumetric learning, Efficient neural networks, Convolutional networks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hxd3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RTAF replaces a dense volumetric feature transformation with a grouped Gaussian-basis mapping using CP-parameterized weights and a lightweight local convolutional branch. A learned channel prior and input-dependent spatial gate modulate the branch inputs. The module is evaluated in ResNet-18/50 classification and nnU-Netv2 segmentation. Its contribution is a compressed nonlinear representation block with empirical accuracy-cost benefits, rather than a new mechanism for dynamically skipping computation.

Critical strengths
  1. Useful cross-task evidence on existing backbones. ResNet-18 improves AUC and accuracy on all six reported MedMNIST3D datasets. The five-fold nnU-Netv2 task means all improve, with overall Dice increasing from 81.83 to 82.64 and larger gains on Heart and Colon. These insertion results support a reusable module contribution rather than only a new architecture ranking.

  2. Substantial block-level compression with informative internal controls. At the profiled shape, parameters fall from 1769.73K to 248.65K and FLOPs from 231.93G to 45.17G relative to dense Conv3D. Table 3 includes a grouped-linear replacement for the Gaussian mapping: at comparable parameter count it scores 96.71 versus 97.47 Dice. The dense-weight comparison also supports the parameter-saving role of CP factorization. Together these results justify recognizing a limited but useful accuracy-cost contribution.

Critical weaknesses

The main limitation is uncertainty about the trade-off against conventional lightweight replacements at the same insertion points. The existing grouped-linear ablation is informative, but a standard depthwise/grouped convolutional replacement would better establish whether the added nonlinear construction is preferable to a simpler option. This limits confidence in the strength of the contribution, but does not outweigh the cross-task positive evidence. Block-level FLOP savings should not be presented as measured end-to-end acceleration, and the reported ResNet-50 Synapse and prostate peripheral-zone regressions limit universal claims.

Questions for authors

Could the authors provide an existing or small representative comparison against a conventional lightweight replacement at the same network locations and under the same training protocol, with parameter/operation counts? The existing grouped-linear ablation is acknowledged; this question concerns the practical value relative to a common alternative. Clarifying which matched-backbone baselines were rerun under the same settings would also help interpret the improvements.

I lean toward acceptance based on the demonstrated module-level savings and broad positive results. I do not require exhaustive multi-task ablations or an anatomical interpretation of the gate.

3 comments
AuthorsOpenReview ↗

Response to Reviewer hxd3

Comparison with conventional lightweight replacements

Thank you for recognizing the gains from inserting RTAF into existing classification and segmentation backbones, as well as the block-level savings and the role of the grouped-linear ablation. We appreciate your suggestion to compare RTAF with standard lightweight replacements at the same network locations.

The new results on Task09 Spleen and NoduleMNIST3D show that RTAF retains an advantage over conventional lightweight replacements.

We evaluated depthwise-separable convolution (DW), a parameter-matched DW variant (DW-budget), SCConv-3D, and Dynamic Conv-3D on MSD Task09 Spleen and NoduleMNIST3D. Within each backbone, all replacements used the same insertion points and training protocol as RTAF. The parameter and FLOP counts in the tables are for the full networks.

Task09 Spleen

nnU-Netv2 variantDice (%) ↑Params (M) ↓GFLOPs ↓
Original97.3531.1131050.876
DW96.4623.685967.157
DW-budget96.7624.442976.028
SCConv-3D95.7024.536976.533
Dynamic Conv-3D95.6854.4111050.938
RTAF97.4724.449980.680

At nearly the same parameter count as DW-budget, RTAF reaches 97.47, a gain of 0.71 Dice points with 0.48% more FLOPs. Standard DW has the lowest parameter and operation counts, with Dice 1.01 points below RTAF. Compared with the original nnU-Netv2, RTAF reduces parameters by 21.42% and FLOPs by 6.68%, with a 0.12-point increase in Dice.

NoduleMNIST3D

VariantR18 AUC / ACC ↑R18 Params (M) / GFLOPs ↓R50 AUC / ACC ↑R50 Params (M) / GFLOPs ↓
Original0.882 / 0.86033.141 / 35.4980.852 / 0.85546.139 / 47.882
DW0.877 / 0.84724.643 / 33.4630.878 / 0.85324.055 / 40.320
DW-budget0.883 / 0.84225.493 / 33.6690.893 / 0.84526.275 / 41.087
SCConv-3D0.892 / 0.86825.627 / 33.6970.884 / 0.83926.604 / 41.186
Dynamic Conv-3D0.868 / 0.85559.769 / 35.5690.872 / 0.677115.373 / 48.066
RTAF0.889 / 0.85525.498 / 33.7950.899 / 0.86826.291 / 41.543

Compared with DW-budget, RTAF improves AUC by 0.006 on both backbones and accuracy by 1.3 and 2.3 percentage points on ResNet-18 and ResNet-50, respectively. The parameter counts differ by less than 0.07%, and RTAF uses 0.37% more FLOPs on ResNet-18 and 1.11% more on ResNet-50.

On ResNet-18, SCConv-3D gives the highest AUC and accuracy and uses slightly fewer FLOPs than RTAF, with 0.129M more parameters. Compared with the original ResNet-18, RTAF improves AUC from 0.882 to 0.889 and reduces parameters by 23.06%, while accuracy falls from 0.860 to 0.855. On ResNet-50, RTAF gives the highest AUC and accuracy among the tested variants.

Full-network parameters and FLOPs

We agree that full-network counts are useful for interpreting the savings in Fig. 4a. The 80.52% FLOP reduction in that figure applies to a single block at the profiled input shape. The table below shows the full-network counts after replacing selected blocks with RTAF. We did not measure end-to-end latency.

Backbone and taskOriginal Params (M)RTAF Params (M)Original GFLOPsRTAF GFLOPs
3D ResNet-18, NoduleMNIST3D33.14125.49835.49833.795
3D ResNet-50, NoduleMNIST3D46.13926.29147.88241.543
nnU-Netv2, Task02 Heart30.78624.1221119.3411075.137
nnU-Netv2, Task04 Hippocampus5.6033.70638.28336.796
nnU-Netv2, Task05 Prostate44.57935.532947.695829.358
nnU-Netv2, Task09 Spleen31.11324.4491050.876980.680
nnU-Netv2, Task10 Colon30.70324.0401104.8131030.544

RTAF reduces full-network parameters by 23.06% and 43.02% on ResNet-18 and ResNet-50, respectively, and FLOPs by 4.80% and 13.24%. Across the five MSD tasks, the corresponding reductions are 20.29–33.86% for parameters and 3.88–12.49% for FLOPs.

We will add the Spleen comparison to Sec. 4.4 immediately after the component ablations. We will place the full classification table in the appendix. Since nnU-Netv2 configures the network and input patch size separately for each dataset, we will report the exact counts for each task plan in the appendix.

Thank you for adding the comparisons at the same insertion points and for rerunning the classification backbones in a shared codebase. The parameter-matched depthwise baseline on Spleen directly addresses my main concern. RTAF improves Dice by 0.71 points at nearly the same parameter and FLOP counts. The NoduleMNIST3D results also help establish the scope of the benefit, with SCConv-3D leading on ResNet-18 and RTAF leading on ResNet-50. These results strengthen the case for RTAF as a reusable lightweight block. I remain at weak acceptance.

AuthorsOpenReview ↗

Response to Reviewer hxd3

Thank you for taking the time to review the additional experiments. Your suggestion helped us clarify the accuracy–cost trade-off of RTAF and how it varies across backbones. We have included the Spleen comparison in Table 4 and the NoduleMNIST3D results in Table 9 of Appendix F.

Reviewer_4KmmOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes RTAF, a plug-in replacement for 3D convolution blocks with three parts: (1) Router: an SE-style channel gate, (2) Nonlinear Tensor Branch: RBF mapping with some factorization by channel-by-basis weights, and (3) Local Convolution Branch: a depthwise 3×3×3 convolution followed by a pointwise convolution. RTAF is tested with 3D ResNet (six MedMNIST3D datasets) and nnU-Net v2 (five MSD tasks).

Critical strengths
  1. The RTAF block is simple, drop-in, and described in a good way, with implementation details in the appendix and the code in the suppl. material.

  2. It is evaluated in both classification and segmentation, and against a strong segmentation baseline (nnU-Net v2, 5-fold CV).

  3. The paper is honest about negative results: the drop on ResNet-50 SynapseMNIST3D and the trade-off as well.

Critical weaknesses
  1. The major issue is the novelty is very limited and incremental. RTAF combines very well-known pieces:

a) an SE/CBAM-style channel and spatial gate; b) a depthwise-separable convolution; c) CP-factorized weights, which date back to CP-decomposed convolutions.

  1. Over nnU-Net v2, the gains are +0.12 (Spleen), +0.32 (Hippocampus), and +0.81 on the five-task mean. This is very minor. Also, only 5 of the 10 MSD tasks are used. The not reported ones (Liver, Pancreas, Lung, HepaticVessel, BrainTumour) include the small-lesion settings that the motivation emphasizes. The selection should be justified. Also, HD95 is not reported in Table 2.

  2. The ablations are weak, and they are run only on Spleen task. The differences are 0.07 to 1.1 points from a single configuration with no variance.

  3. The captions of Figure 1 and Figure 2 are too short. They should be more descriptive.

Questions for authors
  1. Why these five MSD tasks? Can you report results on the pancreas, Liver, and Lung?

  2. What are the per-fold mean and std on MSD, and how many seeds were used for MedMNIST3D?

  3. The HD95 metric is missing; it's an important and common metric in medical segmentation.

8 comments
AuthorsOpenReview ↗

Response to Reviewer 4Kmm

Thank you for noting that RTAF is easy to integrate into existing backbones and for highlighting the classification and segmentation experiments.

W1. What RTAF adds to prior work

Nonlinear tensor module. Prior work on CNN compression uses CP decomposition to factorize spatial convolution kernels. In RTAF, the CP factors reconstruct Wg(i,c,k)W_g(i,c,k), a tensor that combines Gaussian basis responses across channels at each voxel. The learnable Gaussian centers let output channels respond to different ranges of input values. A dense mapping needs KK coefficients for each input–output channel pair. CP lets channel pairs share basis weightings, keeping the nonlinear mapping compact (Sec. 3.3).

Response-aware modulation. SE and CBAM use attention weights to recalibrate a feature tensor. In RTAF, one response map MM adjusts the inputs to two different transformations in opposite directions. The nonlinear tensor branch receives (1+M)⊙Z(1+M)\odot Z, and the local convolution branch receives (1−M)⊙Z(1-M)\odot Z. For an entry where M=0.5M=0.5, the Gaussian bases see 1.5Z1.5Z, and the local convolution sees 0.5Z0.5Z. The scaling changes the Gaussian responses before they are combined. Both branches run and their outputs are fused (Secs. 3.1–3.2).

**Comparison with lightweight replacements on Task09 Spleen.**To test whether the complete RTAF block offers an advantage over a conventional lightweight replacement at a similar parameter budget, we compared it with DW-budget and other lightweight operators on Task09 Spleen.We tested depthwise-separable convolution (DW), a parameter-matched variant (DW-budget), SCConv-3D, and Dynamic Conv-3D. All replacements used the same insertion points, nnU-Netv2 task plan, and training protocol. The parameter and FLOP counts are for the full network.

\text{nnU-Netv2 variant} & \text{Dice (\\%)} & \text{Params (M)} & \text{GFLOPs} \\\\ \hline \text{Original} & 97.35 & 31.113 & 1050.876 \\\\ \text{DW} & 96.46 & 23.685 & 967.157 \\\\ \text{DW-budget} & 96.76 & 24.442 & 976.028 \\\\ \text{SCConv-3D} & 95.70 & 24.536 & 976.533 \\\\ \text{Dynamic Conv-3D} & 95.68 & 54.411 & 1050.938 \\\\ \text{RTAF} & 97.47 & 24.449 & 980.680 \end{array}$$ At nearly the same parameter count as DW-budget, RTAF gains 0.71 Dice points with 0.48% more FLOPs. Standard DW uses the fewest parameters and FLOPs and scores 1.01 Dice points below RTAF. Compared with the original nnU-Netv2, RTAF uses 21.42% fewer parameters and 6.68% fewer FLOPs, with a 0.12-point increase in Dice. We will add this comparison after the component ablations in Sec. 4.4. **W3. Component ablations on Task09 Spleen** The comparison above evaluates the full block. Table 3 examines what happens when we change individual components. The mean Dice differences are small, so we report the corresponding standard deviations below. $$\begin{array}{lc} \text{Variant} & \text{Dice (\\%), mean}\pm\text{SD} \\\\ \hline \text{Grouped linear map} & 96.71\pm2.18 \\\\ \text{Dense basis weights} & 97.40\pm0.93 \\\\ \text{Router disabled }(M=0) & 97.00\pm1.85 \\\\ \text{Without local branch} & 97.15\pm1.49 \\\\ \text{Full RTAF} & 97.47\pm1.18 \end{array}$$ Replacing the Gaussian mapping gives the largest observed Dice drop (0.76 points) and increases HD95 from 25.11 to 37.18, with fewer FLOPs (30.13G vs. 45.17G). The dense and CP variants differ by only 0.07 Dice points, and the dense variant has a lower SD. CP reduces the block parameter count from 375.4K to 248.6K at the same reported FLOPs. We will add the SDs to Table 3. **W2 / Q1. MSD task coverage** We evaluated RTAF on the five remaining MSD tasks, including Liver, Pancreas, and Lung. Compute time before submission limited the original evaluation to five tasks. The nnU-Net reference scores are from Cheng et al. (WACV 2025, Tables 1–2). $$\begin{array}{lrr} \text{MSD task} & \text{nnU-Net Dice (\\%)} & \text{RTAF Dice (\\%)} \\\\ \hline \text{Liver} & 85.86 & 82.29 \\\\ \text{Pancreas} & 67.21 & 68.26 \\\\ \text{Lung} & 73.97 & 78.16 \\\\ \text{Hepatic Vessel} & 68.96 & 69.72 \\\\ \text{Brain Tumour} & 69.56 & 80.29 \end{array}$$ RTAF scores higher on four tasks, with the largest differences on Brain Tumour and Lung. On Liver, tumor Dice is lower (68.10 vs. 75.97). In the original five-task comparison, mean Dice increases from 81.83 to 82.64, with small gains on Spleen (+0.12) and Hippocampus (+0.32). We will include the class-wise results for the five additional tasks in the appendix.
AuthorsOpenReview ↗

Response to Reviewer 4Kmm

Q2. Cross-validation and training seeds

We trained each MedMNIST3D model with three random seeds using the official splits. Table 1 reports the median results across the three seeds. Below we list all three runs for each dataset. Each cell gives AUC / ACC. B1–B3 denote the baseline runs, and R1–R3 denote the runs with RTAF.

3D ResNet-18

\text{Dataset} & \text{B1} & \text{B2} & \text{B3} & \text{R1} & \text{R2} & \text{R3} \\\\ \hline \text{OrganMNIST3D} & 0.995/0.897 & 0.996/0.908 & 0.905/0.902 & 0.997/0.925 & 0.995/0.915 & 0.996/0.928 \\\\ \text{NoduleMNIST3D} & 0.875/0.835 & 0.880/0.852 & 0.884/0.858 & 0.889/0.855 & 0.890/0.861 & 0.892/0.848 \\\\ \text{FractureMNIST3D} & 0.715/0.550 & 0.695/0.533 & 0.688/0.500 & 0.729/0.579 & 0.750/0.513 & 0.739/0.537 \\\\ \text{AdrenalMNIST3D} & 0.821/0.795 & 0.784/0.664 & 0.868/0.651 & 0.886/0.812 & 0.876/0.826 & 0.872/0.822 \\\\ \text{VesselMNIST3D} & 0.906/0.859 & 0.834/0.861 & 0.801/0.898 & 0.964/0.948 & 0.972/0.945 & 0.961/0.942 \\\\ \text{SynapseMNIST3D} & 0.786/0.756 & 0.809/0.750 & 0.913/0.827 & 0.867/0.810 & 0.867/0.808 & 0.870/0.836 \end{array}$$ **3D ResNet-50** $$\begin{array}{lcccccc} \text{Dataset} & \text{B1} & \text{B2} & \text{B3} & \text{R1} & \text{R2} & \text{R3} \\\\ \hline \text{OrganMNIST3D} & 0.991/0.897 & 0.992/0.889 & 0.995/0.905 & 0.996/0.910 & 0.994/0.915 & 0.997/0.916 \\\\ \text{NoduleMNIST3D} & 0.882/0.845 & 0.883/0.848 & 0.860/0.855 & 0.899/0.868 & 0.897/0.858 & 0.901/0.868 \\\\ \text{FractureMNIST3D} & 0.706/0.542 & 0.716/0.567 & 0.730/0.463 & 0.744/0.542 & 0.721/0.558 & 0.756/0.479 \\\\ \text{AdrenalMNIST3D} & 0.843/0.775 & 0.861/0.789 & 0.877/0.752 & 0.876/0.826 & 0.873/0.819 & 0.890/0.838 \\\\ \text{VesselMNIST3D} & 0.900/0.901 & 0.888/0.895 & 0.887/0.921 & 0.951/0.929 & 0.945/0.932 & 0.956/0.919 \\\\ \text{SynapseMNIST3D} & 0.801/0.795 & 0.907/0.838 & 0.866/0.818 & 0.742/0.741 & 0.776/0.778 & 0.740/0.750 \end{array}$$ On VesselMNIST3D with ResNet-18, all three RTAF runs exceed all three baseline runs in both metrics. On SynapseMNIST3D with ResNet-50, all three RTAF runs score below all three baseline runs in both metrics. We will state the seed count and median reporting in Sec. 4.1 and include the individual runs in the appendix. **Q3. HD95** HD95 complements Dice by measuring boundary distances. We computed HD95 using the same procedure for nnU-Netv2 with and without RTAF on the four tasks below. $$\begin{array}{lrr} \text{MSD task} & \text{nnU-Netv2 HD95} & \text{+ RTAF HD95} \\\\ \hline \text{Heart} & 3.88 & 2.01 \\\\ \text{Hippocampus} & 1.16 & 1.08 \\\\ \text{Prostate} & 4.36 & 4.66 \\\\ \text{Spleen} & 24.88 & 25.11 \end{array}$$ HD95 decreases on Heart and Hippocampus and increases on Prostate and Spleen, while Dice improves on all four tasks. We will report HD95 alongside Dice in the appendix. **W4. Revised figure captions** **Fig. 1. RTAF block.** The router scales the inputs to the nonlinear tensor and local convolution branches in opposite directions, and the fused output is added to the original input. **Fig. 2. RTAF router.** The product of the learned channel prior $M_c$ and the input-dependent spatial map $M_s$ gives the modulation map $M$.
Reviewer_4KmmOpenReview ↗

Follow up with the authors

Thanks to the authors for providing the rebuttal; I really appreciate their efforts. The contributions of most of the components of RTAF are still weak. In the above table you shared in your reply to W3, the difference between Dense basis weights and full RTAF is just 0.07 and within the noise and SD range. There is no significant improvement with/without the local branch, and if the Router enabled/disasbled, all are within the same range.

Reviewer_Z8YXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RTAF, a drop-in replacement for 3D convolutional blocks. After instance normalization, a router produces a soft map and rescales the input of two parallel branches: (1+M)·Z goes to a "Nonlinear Tensor Branch" and (1−M)·Z to a "Local Convolution Branch" (Eqs. 1–2). The nonlinear branch is a grouped, pointwise layer with Gaussian radial basis functions with learnable centres. The local branch is a depthwise 3×3×3 convolution followed by a pointwise convolution. The two outputs are concatenated, fused by a 1×1×1 convolution and instance norm, and added back residually (App. C). RTAF replaces a few blocks in 3D ResNet-18/50 (the second conv of the last two stages) and nnU-Net v2 (the last three encoder stages). It reports gains on six MedMNIST3D datasets (Table 1) and five MSD tasks (five-task mean 81.83 to 82.64 Dice; Table 2), and 86% fewer parameters and 81% fewer FLOPs than a dense 256-channel 3×3×3 conv at one profiled size (Fig. 4a).

Critical strengths
  1. Simple, genuinely plug-and-play module. It is evaluated on both classification and segmentation backbones, with nnU-Net's planning and training pipeline left unchanged (App. B). Consistent gains for ResNet-18 on MedMNIST3D. AUC and accuracy improve on all six datasets over the 2) 3D ResNet-18 numbers, and RTAF ranks first in mean AUC (0.889) among the listed methods (Table 1; I verified the averages).
  2. Unusually candid reporting of negative results. The ResNet-50 SynapseMNIST3D drop (0.851 → 0.742 AUC) and the Prostate peripheral-zone drop (73.31 → 70.60) are discussed openly (Secs. 4.2–4.3, App. D).
  3. Component and router ablations are provided (Tables 3–4), and the implementation details in App. C are specific enough to reconstruct the block. I reproduced the reported 248.65K parameters at C = 256 from them.
Critical weaknesses
  1. Underspecified or questionable design choices
  • The Gaussian bases have a fixed unit width, and the centres are initialized in [−1,1] with spacing ≈0.29 (App. C). The eight bases therefore overlap heavily and are nearly collinear on the instance-normalized input, which (1+M) scales by up to 2. Please justify the width choice and show the learned centres and widths. A learnable width or a sensitivity study over K would help.
  • The nonlinear branch is purely pointwise (no spatial context), yet it is described as modelling "complex anatomical structures" (Sec. 1). All spatial modelling comes from the depthwise branch and the spatial gate.
  • There is no sensitivity study for G, K, R or the insertion positions. Why are only the last stages replaced, and what happens when RTAF replaces more or all blocks?
  1. No comparison with the efficient or adaptive operators the paper positions itself against
  • Related work frames RTAF against OctConv, SCConv, ACS-Conv, Dynamic Convolution, CondConv and KACNet (Secs. 1, 2.2–2.3). Yet none of them is plugged into the same backbones at the same positions for a controlled comparison. Table 1's ACS and KACNet numbers are copied from other papers. At minimum, the paper needs depthwise-separable, SCConv, and CondConv/DynConv in place of RTAF inside nnU-Net and ResNet-18, with matched cost reported.
  1. Statistical support is missing
  • The MedMNIST "matched" ResNet-18/50+3D numbers appear to come from the MedMNIST v2 paper rather than being retrained in the authors' code (Sec. 4.1: "use results from the original papers for comparison methods when available"). If so, the improvement mixes RTAF with implementation and seed differences. Please confirm, and retrain the backbones with and without RTAF in one codebase over several seeds.
  • nnU-Net ± RTAF numbers come from the authors' five-fold cross-validation on the MSD training set, while the other ten methods' numbers are "from the original papers," which used different splits and protocols. "Leads every task mean, exceeding the result from other models" (Sec. 4.3) is therefore not a controlled claim. The only valid comparison in Table 2 is nnU-Net vs nnU-Net + RTAF. The nnU-Net configuration (3d_fullres? 2d?) is also not stated.
Questions for authors
  1. Were the ResNet-18/50 (2.5D/3D/ACS) baselines in Table 1 retrained in your codebase, or copied from MedMNIST v2? Can you report mean ± SD over at least three seeds for backbone vs backbone + RTAF?
  2. How does RTAF compare, in the same backbones and positions, with a depthwise-separable block, SCConv, CondConv/DynConv and a KAN-conv block at matched cost?
  3. How about inputting RTAF idea into the largest convolution kernel in 3D medical imaging domain paper?
  • Lee, H. H., Liu, Q., Bao, S., Huo, Y., & Landman, B. A. (2025). Spatially-Adaptive Gradient Re-parameterization for 3D Large Kernel Optimization. ICML 2026.

Will raise my score if all the weakness and questions are well addressed.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Z8YX

Thank you for checking our classification averages and reproducing the block parameter count. We also appreciate your recognition that RTAF can be integrated into existing backbones while retaining the nnU-Netv2 pipeline.

Gaussian width and basis overlap. We use ϕk(u)=exp⁡(−((u−μk)/s)2)\phi_k(u)=\exp(-((u-\mu_k)/s)^2) with fixed s=1s=1 as a default for instance-normalized features. The actual routed inputs to the two Gaussian modules have standard deviations of 1.318 and 1.298, so the unit scale is on the same order as the observed input variation.

We have added the Gaussian basis analysis to the appendix. The figure shows the initial and trained centers, the routed input densities with the eight Gaussian curves, the response correlations, and the singular-value spectra. We sampled inputs to both modules from all 52 validation volumes at one Hippocampus checkpoint. In both modules, the trained centers round to (−0.656,−0.468,−0.281,−0.094,0.094,0.281,0.468,0.656)(-0.656,-0.468,-0.281,-0.094,0.094,0.281,0.468,0.656). The scale stayed fixed at 1.

\text{Module} & \text{Mean off-diagonal }|r| & \text{Energy effective rank} & \text{Top-two energy (\\%)} \\\\ \hline \text{Earlier} & 0.589 & 2.043 & 98.92 \\\\ \text{Later} & 0.584 & 2.085 & 98.87 \end{array}$$ In both modules, the centers contracted almost uniformly toward zero and remained nearly symmetric and evenly spaced. The adjacent spacing fell from 0.286 to 0.187, increasing overlap at the fixed width. About 99% of the centered response energy lies in two directions, each of which combines multiple Gaussian responses. RTAF uses learned tensor weights to combine these responses, so we tested the effect of changing the number of bases directly on segmentation. We have added the width, basis-count, and placement comparisons below to the appendix. $$\begin{array}{llrr} \text{Ablation} & \text{Setting} & \text{Dice (\\%)} & \text{HD95} \\\\ \hline \text{Width} & s=0.5 & 89.03 & 1.30 \\\\ & s=1 & 89.37 & 1.08 \\\\ & s=2 & 89.07 & 1.29 \\\\ \text{Basis count} & K=4 & 88.99 & 1.30 \\\\ & K=8 & 89.37 & 1.08 \\\\ & K=16 & 89.00 & 1.29 \\\\ \text{Placement} & \text{Stages 0, 1} & 88.83 & 1.31 \\\\ & \text{Stages 2, 3} & 89.37 & 1.08 \\\\ & \text{Stages 0, 1, 2, 3} & 89.04 & 1.30 \end{array}$$ Among the tested widths and basis counts, $s=1$ and $K=8$ give the highest Dice and lowest HD95. Using four bases reduces Dice by 0.38 points, and using sixteen reduces it by 0.37 points. These results support retaining the defaults on Hippocampus. Learnable widths and independent sweeps over group count $G$ and CP rank $R$ remain open. Table 3 separately shows that CP reduces block parameters from 375.4K to 248.6K with similar Dice (97.47 with CP and 97.40 with dense weights). **Nonlinear feature modeling and spatial context.** Thank you for pointing out that Sec. 1 could be more precise. The Gaussian mapping is evaluated at each voxel on an input that already depends on its neighborhood. The $3\times3\times3$ spatial gate uses neighboring features to compute the spatial component of $M$. Two locations with the same center feature vector $Z$ can therefore give different Gaussian responses if their neighborhoods produce different $M$ and hence different $(1+M)\odot Z$ (Eqs. 1 and 5). The CP-parameterized weights combine Gaussian responses across feature-value ranges and channels within each group. In parallel, the depthwise branch directly aggregates neighboring features. RTAF fuses the two outputs. Table 3 tests the contribution of the nonlinear mapping while retaining the router and local branch. Replacing the Gaussian mapping with a grouped linear map gives the largest Dice drop among the tested component ablations on Spleen (97.47 to 96.71), and HD95 increases from 25.11 to 37.18. We will revise Sec. 1 to explain how the full RTAF block combines neighborhood-conditioned nonlinear channel transformations with direct spatial aggregation to model anatomical features. **Encoder placement.** Late placement applies the nonlinear transformation to deeper features while preserving the early encoder. With one replacement per selected stage, stages 2 and 3 give the highest Dice and lowest HD95 on Hippocampus. Extending the replacement to one block in each of the four stages lowers Dice by 0.33 points and raises HD95 from 1.08 to 1.30.
AuthorsOpenReview ↗

Response to Reviewer Z8YX

Lightweight replacements. We used the same insertion points and training protocol within each backbone. DW denotes depthwise-separable convolution, and DW-budget nearly matches RTAF's parameter count. Costs are reported as full-network parameters (M) / GFLOPs.

\text{Task09 Spleen}&\text{Dice (\\%)}&\text{Params (M)}&\text{GFLOPs}\\\\ \hline \text{Original}&97.35&31.113&1050.876\\\\ \text{RTAF}&97.47&24.449&980.680\\\\ \text{DW}&96.46&23.685&967.157\\\\ \text{DW-budget}&96.76&24.442&976.028\\\\ \text{SCConv-3D}&95.70&24.536&976.533\\\\ \text{DynConv-3D}&95.68&54.411&1050.938 \end{array}$$ RTAF gains 0.71 Dice points over DW-budget with 0.48% more FLOPs. Compared with the original nnU-Netv2, RTAF gains 0.12 Dice points with 21.42% fewer parameters and 6.68% fewer FLOPs. $$\begin{array}{lcccc} \text{NoduleMNIST3D}&\text{R18 AUC/ACC}&\text{R18 cost}&\text{R50 AUC/ACC}&\text{R50 cost}\\\\ \hline \text{Original}&0.882/0.860&33.141/35.498&0.852/0.855&46.139/47.882\\\\ \text{RTAF}&0.889/0.855&25.498/33.795&0.899/0.868&26.291/41.543\\\\ \text{DW}&0.877/0.847&24.643/33.463&0.878/0.853&24.055/40.320\\\\ \text{DW-budget}&0.883/0.842&25.493/33.669&0.893/0.845&26.275/41.087\\\\ \text{SCConv-3D}&0.892/0.868&25.627/33.697&0.884/0.839&26.604/41.186\\\\ \text{DynConv-3D}&0.868/0.855&59.769/35.569&0.872/0.677&115.373/48.066 \end{array}$$ Compared with DW-budget, RTAF improves AUC by 0.006 on both backbones and accuracy by 1.3 and 2.3 percentage points on ResNet-18 and ResNet-50, respectively. SCConv-3D achieves the highest AUC and accuracy on ResNet-18, and RTAF achieves the highest on ResNet-50. We will add the Spleen comparison to Sec. 4.4 and include the full classification comparison in the appendix. **Classification baselines and three-seed results.** The 2.5D, 3D, and ACS baselines in Table 1 come from MedMNIST v2. The RTAF entries in Table 1 report median scores. We ran both 3D backbones with and without RTAF using the same codebase, data splits, and training settings. We used three random seeds per model. Each cell gives AUC / ACC for one run. Runs 1–3 follow the recorded order within each model. **3D ResNet-18** $$\begin{array}{lcccccc} &&\text{Original}&&&\text{+ RTAF}&\\\\ \text{Task}&1&2&3&1&2&3\\\\ \hline \text{Organ}&0.995/0.897&0.996/0.908&0.905/0.902&0.997/0.925&0.995/0.915&0.996/0.928\\\\ \text{Nodule}&0.875/0.835&0.880/0.852&0.884/0.858&0.889/0.855&0.890/0.861&0.892/0.848\\\\ \text{Fracture}&0.715/0.550&0.695/0.533&0.688/0.500&0.729/0.579&0.750/0.513&0.739/0.537\\\\ \text{Adrenal}&0.821/0.795&0.784/0.664&0.868/0.651&0.886/0.812&0.876/0.826&0.872/0.822\\\\ \text{Vessel}&0.906/0.859&0.834/0.861&0.801/0.898&0.964/0.948&0.972/0.945&0.961/0.942\\\\ \text{Synapse}&0.786/0.756&0.809/0.750&0.913/0.827&0.867/0.810&0.867/0.808&0.870/0.836 \end{array}$$ **3D ResNet-50** $$\begin{array}{lcccccc} &&\text{Original}&&&\text{+ RTAF}&\\\\ \text{Task}&1&2&3&1&2&3\\\\ \hline \text{Organ}&0.991/0.897&0.992/0.889&0.995/0.905&0.996/0.910&0.994/0.915&0.997/0.916\\\\ \text{Nodule}&0.882/0.845&0.883/0.848&0.860/0.855&0.899/0.868&0.897/0.858&0.901/0.868\\\\ \text{Fracture}&0.706/0.542&0.716/0.567&0.730/0.463&0.744/0.542&0.721/0.558&0.756/0.479\\\\ \text{Adrenal}&0.843/0.775&0.861/0.789&0.877/0.752&0.876/0.826&0.873/0.819&0.890/0.838\\\\ \text{Vessel}&0.900/0.901&0.888/0.895&0.887/0.921&0.951/0.929&0.945/0.932&0.956/0.919\\\\ \text{Synapse}&0.801/0.795&0.907/0.838&0.866/0.818&0.742/0.741&0.776/0.778&0.740/0.750 \end{array}$$ RTAF improves mean AUC and accuracy on all six datasets with ResNet-18 and on five with ResNet-50. Both metrics decrease on SynapseMNIST3D with ResNet-50. **MSD protocol.** For each task, both nnU-Netv2 variants used `3d_fullres` with the same plans, preprocessing, training settings, and five cross-validation folds. Mean Dice increases from 81.83 to 82.64. **Large-kernel experiment.** Thank you for suggesting Rep3D (Lee et al., ICML 2026). On MSD Task09 Spleen, we compared a plain depthwise convolution with the same convolution equipped with Rep3D's Low-Rank Receptive Bias Modeling (LRBM), at the same position in nnU-Netv2. Adding LRBM increases Dice from 92.49% to 95.65%, a gain of 3.16 percentage points. We will cite Rep3D in Sec. 2.1, summarize this result in Sec. 4.4, and provide the experimental setup in the appendix.

Thank you again for your time and thoughtful reviews. With the discussion deadline approaching, we would appreciate it if you could review our response at your convenience. Please feel free to let us know if you have any further questions.

As the discussion period closes in a few hours, we would like to send a brief reminder. We addressed each of your questions in our earlier replies, and the corresponding changes are included in the rebuttal revision PDF. If you feel that any of your concerns remain unresolved, we would be glad to discuss them before the deadline.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on the revised manuscript

Thank you for taking another look at our results. We have uploaded the revised manuscript to OpenReview, with the new material marked in blue. Section 4.4 now includes the Task09 Spleen comparison with lightweight replacements, including the parameter-matched DW baseline, and the three-seed classification ablations (Table 3). Appendix F reports the full lightweight comparison on NoduleMNIST3D for both 3D ResNet backbones. We also added the Gaussian basis and encoder placement analyses (Appendix E), Dice and the available HD95 results across all ten MSD tasks (Appendix G), full-network parameter and FLOP counts (Appendix H), and the large-kernel comparison with and without LRBM (Appendix I). If these additions address your concerns, we would be grateful if you would reconsider your overall rating. We are happy to clarify any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.