acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Cross-Layer Attention for Channel and Spatial Recalibration in ConvNets

Abstract

Cross-layer attention enables convolutional networks to retrieve information from earlier layers, while channel and spatial attention recalibrate features within individual blocks. We investigate how cross-layer retrieval can inform channel and spatial recalibration. We propose Cross-Layer Attention Recalibration (CLAR), a module that constructs gates for the current block by attending to channel and spatial representations accumulated within the same network stage. CLAR maintains separate memories of channel descriptors and spatial gating maps. Input-dependent queries attend to keys from preceding blocks and the current block, and the retrieved values generate channel and spatial gates that sequentially modulate the current residual features. This design preserves the local residual shortcut and avoids storing full historical feature tensors. We additionally apply stochastic history-source dropping during training to regularize cross-layer retrieval. We evaluate CLAR on CIFAR-100 and ImageNet using residual networks of different depths. In single-run ImageNet experiments trained for 100 epochs, CLAR achieves best top-1 validation accuracies of 77.49% and 78.26% with ResNet-50 and ResNet-101, respectively. On CIFAR-100, history-source dropping increases the mean final accuracy of CLAR-equipped ResNet-50 from 78.70% to 79.13% across five matched seeds. These results support cross-layer attention as a mechanism for channel and spatial feature recalibration.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
cross-layer attention, channel attention, spatial attention, feature recalibration, convolutional neural networks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rpVrOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Cross-Layer Attention Recalibration (CLAR), which constructs channel and spatial gates by attending to compact representations from preceding blocks and the current block within each residual stage. Historical source dropping regularizes this retrieval. The method is evaluated on CIFAR-100, ImageNet-1K, and COCO.

Critical strengths
  1. Storing channel vectors and single-channel spatial maps instead of full historical feature tensors is a sensible design. It makes individual historical sources accessible with a compact explicit memory.

  2. The evaluation covers classification and dense prediction. The reported ImageNet Top-1 gains over CBAM are 1.26, 0.71, and 0.56 percentage points for ResNet-50/101/152, with small additional parameter counts over CBAM. These gains deserve recognition, particularly at depth 50, although training differences complicate attribution.

Critical weaknesses
  1. Limited conceptual advance over closely related attention designs: CLAR retains CBAM’s channel-then-spatial gating order (Woo et al., 2018), and the spatial branch uses the familiar channel pooling and convolutional construction. Its principal extension is retrieval over compact historical attention representations, together with source dropping. This is a meaningful distinction from local CBAM, but historical information reuse is already central to MRLA-L (Fang et al., 2023 ICLR), DLA-L (Wang et al., 2024 IJCAI), and DIA (Huang et al., 2020 AAAI). The paper does not yet establish that this combination yields a sufficiently substantial new insight. In Table 3, gains over CBAM on CIFAR-100 are 0.21/0.40/0.67 points, and ImageNet gains over DLA-L are only 0.19/0.29/0.19 points. These numbers should be interpreted with the training-protocol caveat below.

  2. The additional benefit of spatial recalibration is weakly established: In Table 1, adding spatial recalibration to the channel-only variant improves Top-1 by just 0.07 points on ResNet-50 and 0.22 on ResNet-101; ResNet-50 Top-5 decreases from 94.67 to 94.46. Spatial-only Top-1 is also below the listed ResNet references by 0.72 and 0.07 points. The stronger concern is that the matched channel-only versus full-model comparison provides limited evidence for the branch’s incremental value. No corresponding branch ablation is reported for ImageNet or COCO.

  3. Limited evidence for an advantageous accuracy–cost trade-off: In Tables 4 and 5, CLAR improves the headline COCO metrics over MRLA-L by only 0.1–0.2 AP while using approximately 5.32%–7.24% more total detector parameters. The corresponding parameter increase for ImageNet classification is 9.21%–10.07%. Without matched training and uncertainty estimates, these small AP margins do not establish a robust advantage. Please report bounding-box and mask AP to two decimal places using the unrounded evaluation outputs to make the differences clearer.

  4. Missing close COCO baselines: CBAM and DLA-L are omitted from the COCO tables despite their relevance to CLAR’s design. CBAM has been evaluated with Faster R-CNN on COCO (Woo et al., 2018), and DLA-L reports both Faster R-CNN and Mask R-CNN results (Wang et al., 2024). Including these methods under matched settings would clarify whether historical channel–spatial recalibration improves on the closest alternatives.

  5. Relevant recent attention modules are missing from the evaluation: SE (Hu et al., 2018 CVPR), CBAM (Woo et al., 2018 ECCV), MRLA-L (Fang et al., 2023 ICLR), and DLA-L (Wang et al., 2024) remain relevant baselines; their age alone is not a weakness. However, recent work directly addresses the claimed contributions: SCSA (Si et al., 2025 Neurocomputing) studies spatial–channel interaction on ResNet backbones, including ImageNet and COCO, while the extended DIA study (Huang et al., 2025 Neucom) examines sharing and calibrating attention across layers. Positioning CLAR against these works, with a matched comparison to a suitable recent module such as SCSA, would clarify its present contribution. Achieving the latest leaderboard result is not itself a requirement.

  6. Source weighting and source dropping are not disentangled: In Table 2, static and uniform weighting use no source dropping, whereas the reported dynamic configuration uses p = 0.4. The text states that dynamic weighting without dropping underperforms both alternatives, but the corresponding row is missing. Thus, the comparison does not isolate the benefit of input-dependent source weighting from regularization. Please report the missing result and compare all three weighting schemes at the same dropping probability to establish whether dynamic historical selection is necessary.

Questions for authors
  1. Under one common training recipe, does CLAR improve on CBAM and DLA-L? Please prioritize the missing COCO comparisons and explain any differences in initialization, schedules, augmentation, or detector settings that affect the reported results.

  2. What evidence establishes the added value of the spatial branch beyond channel-only historical recalibration? A matched branch ablation on one existing ImageNet or COCO configuration, together with per-run results where available, would help determine whether the small CIFAR gains are consistent.

  3. Can you report the missing dynamic p = 0 row and compare static, uniform, and dynamic source weighting at the same dropping probability? Which gain remains attributable specifically to input-dependent historical selection?

  4. What benefit does CLAR offer over DLA-L and a relevant recent module under comparable training and resource budgets? Measured memory or latency on one representative setting would help substantiate the efficiency motivation.

4 comments
AuthorsOpenReview ↗

(1/3) Comment for Reviewer rpVr

We thank the reviewer for the valuable feedback. We have supplemented the experiments as extensively as our time and computational resources permit. We apologize for posting our response on the final day of the response period due to the substantial experimental workload. We are still refining the revised manuscript and welcome further feedback, which we will do our best to address in the revision.

Novelty comparison with related methods (W1 & W5)

We thank the reviewer for highlighting these related methods, particularly SCSA and the extended DIA study, which we overlooked in our literature review. Including these recent works will make our discussion more current and comprehensive. The core design of CLAR is to reduce the cost of historical representations while retaining individual historical sources, allowing the current layer to access each source separately and recompute its weight based on the current input. This distinguishes CLAR from related attention-based methods we discussed below.

Specifically, DIA uses an LSTM to accumulate historical information into hidden and cell states. The extended DIA study further investigates sharing attention modules across layers and using an LSTM to calibrate correlated attention maps. MRLA-L and DLA-L both propagate historical information by recursively updating the aggregated output. These methods effectively reuse historical information, but the historical sources have been summarized into recurrent states, and the current layer no longer computes a separate matching weight for each previous layer.

SCSA further explores spatial–channel synergy, using spatial attention to guide subsequent channel attention. These interactions operate within the current feature, and no historical sources are explicitly accessed.

On the other hand, MRLA-Base and DLA-Base retain explicit source interactions at higher resource costs. Within the ResNet family, the ImageNet and COCO results reported for MRLA-Base focus on ResNet-50, and its appendix reports an out-of-memory issue for ResNet-101 at the original batch size; DLA-Base only reports results on CIFAR. In our reproductions, both Base methods with ResNet-101 exhibited training divergence within the first ten epochs. These observations motivated us to explore a more scalable design for explicit historical selection.

Inspired by CBAM, CLAR stores compact channel and spatial representations for each historical source, which the current layer's queries individually matches and reweights. This reduces explicit history storage from O(LCHW)O(LCHW) to O(L(C+HW))O(L(C+HW)) while preserving source-specific retrieval. Our ResNet-152 experiments demonstrate the feasibility and effectiveness of this design in deep networks.

The table below compares these methods across key design dimensions, highlighting the novel aspects of CLAR relative to prior studies.

MethodHistorical representationHow history is accessedRecalibration or aggregation target
CBAMNo explicit historical memoryAttention computed within the current featureChannel and spatial gates
SCSANo explicit historical memorySpatial-guided channel interaction within the current featureSpatial and channel gates
DIA / extended DIARecurrent hidden and cell statesCurrent information fused with historical statesAttention recalibration
MRLA-LRecurrent aggregated outputHistorical aggregate combined with the current contributionFull features
DLA-LContext-updated recurrent aggregateHistorical aggregate updated using context and combined with the current contributionFull features
MRLA-BaseIndividually retained historical feature representationsExplicit attention over historical sourcesFull features
DLA-BaseIndividually retained historical feature representationsContext-based refreshing of historical features followed by explicit attention over sourcesFull features
CLARSeparate compact channel and spatial representations for each sourceCurrent queries separately match and reweight individual sourcesChannel and spatial gates
AuthorsOpenReview ↗

(2/3) Comment for Reviewer rpVr

Contribution of the spatial branch (W2 & Q2)

We thank the reviewer for suggesting branch ablations on a larger-scale dataset. In addition to our four-GPU workstation, we rented a cloud server with the same configuration, but the matched multi-seed ImageNet comparisons and COCO baseline reproductions consumed most of our computational resources. For example, a single ImageNet ResNet-101 run takes approximately three days on two GPUs. We therefore supplemented the evaluation with single-seed branch ablations on ImageNet ResNet-50:

ConfigurationRunsTop-1 (%)Top-5 (%)
Spatial-only177.09893.608
Channel-only177.48293.782
Channel + spatial5 (mean)77.559293.784

Consistent with the CIFAR-100 ablations, the full model achieves higher mean Top-1 accuracy than either single-branch variant, with a modest gain over channel-only recalibration and little difference in ImageNet Top-5 accuracy. These observations provide preliminary evidence for the spatial branch's incremental value across datasets. We will provide these results in the appendix, explicitly label the number of runs, and note that the branch ablations have not yet been validated across multiple seeds.

Matched COCO comparisons (W3, W4 & Q1)

We thank the reviewer for pointing out the gaps in the COCO baselines and comparison settings. We added CBAM and DLA-L and reproduced all baselines and CLAR under the same detector training settings. The updated results below report AP to two decimal places. For each method, we initialized the backbone from the checkpoint of its best-performing ImageNet seed. We will detail the training schedules and data augmentation for each method in the appendix.

BackboneMethodFaster R-CNN box APMask R-CNN box APMask R-CNN mask AP
R50ResNet36.7337.6734.35
R50SE38.5839.2735.65
R50CBAM38.6739.7035.82
R50MRLA-L40.1740.8136.88
R50DLA-L40.0841.0336.94
R50CLAR40.4240.8036.58
R101ResNet39.2839.9636.14
R101SE40.4841.5237.34
R101CBAM40.8241.5937.31
R101MRLA-L42.1142.8238.46
R101DLA-L41.7742.5638.28
R101CLAR42.4243.1838.39

Under matched settings, CLAR outperforms CBAM at both depths with both detectors. Compared with MRLA-L and DLA-L, CLAR achieves higher box AP at both depths with Faster R-CNN and with Mask R-CNN ResNet-101, but does not outperform them on Mask R-CNN ResNet-50 or all mask AP comparisons. The updated results support CLAR's detection performance while showing that the gains vary across detectors and depths. We will update the manuscript accordingly and avoid generalizing these results into a superior accuracy–cost trade-off across all configurations.

AuthorsOpenReview ↗

(3/3) Comment for Reviewer rpVr

Accuracy–cost comparison (W3, W5 & Q4)

We supplemented the evaluation with measured inference throughput, latency, and peak training memory. The following ImageNet ResNet-50 comparison combines five-seed Top-1 accuracy (mean ± sample standard deviation) with costs measured on an RTX 4090 in FP32. Throughput and training memory use batch size 16; latency uses batch size 1.

MethodTop-1 (%) ↑Throughput (images/s) ↑Latency (ms) ↓Peak training memory (GiB) ↓
CBAM77.07 ± 0.231643.449.392.32
MRLA-L77.04 ± 0.101744.379.162.72
DLA-L76.91 ± 0.10824.8518.843.09
SCSA77.30 ± 0.161011.5614.841.98
CLAR77.56 ± 0.16916.9516.722.33

CLAR achieves the highest mean Top-1 accuracy among these methods. Compared with DLA-L, it also provides higher throughput, lower latency, and lower peak training memory. Compared with SCSA, the accuracy gain comes with higher runtime and memory costs. We will provide results for all evaluated depths and methods, including the detection setting, together with the detailed measurement protocol in the appendix.

Disentangling source weighting from source dropping (W6 & Q3)

We thank the reviewer for pointing out the inconsistent source-dropping settings in the original weighting comparison. We added experiments with static and uniform weighting at p=0.4p=0.4 and included the dynamic p=0p=0 results previously shown in Figure 2 but omitted from Table 2. The table below reports CIFAR-100 Top-1 accuracy as mean ± sample standard deviation over five random seeds.

WeightingppResNet-50 (%)ResNet-101 (%)
Uniform079.12 ± 0.3579.27 ± 0.42
Static078.95 ± 0.3279.39 ± 0.47
Dynamic078.70 ± 0.5479.01 ± 0.10
Uniform0.479.16 ± 0.1779.45 ± 0.28
Static0.479.11 ± 0.2379.41 ± 0.28
Dynamic0.479.25 ± 0.1379.59 ± 0.27

All three weighting schemes follow the same dropping rule: historical sources are randomly masked, the current source is always retained, and weights are renormalized over the retained sources. Under the same source-dropping probability of p=0.4p=0.4, dynamic weighting achieves higher mean Top-1 accuracy than both static and uniform weighting at both depths, supporting an additional contribution from input-dependent weighting beyond source dropping alone. We will include these complete comparisons in the revised manuscript and explicitly distinguish the roles of dynamic weighting and source dropping.

Thank you again for your time and constructive feedback. We have submitted the revised manuscript incorporating the additional experiments, updated comparisons, and clarifications discussed in our response, together with the complete appendix.

We would appreciate your consideration of the revised manuscript and our responses, and would be grateful if you could reconsider your overall assessment and score in light of the additional evidence.

Reviewer_ktcQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Cross-Layer Attention Recalibration (CLAR), which uses compact keys and values from the current and preceding blocks within a stage to form channel and spatial gates. During training, historical source dropping randomly omits earlier sources from the current block's attention while retaining the stored history for subsequent blocks.

Critical strengths
  1. The design reduces the explicit storage required for cross-layer interaction. Each block stores channel key/value vectors and single-channel spatial key/value representations rather than full historical feature tensors (Sec. 3.5). The additional model cost over CBAM is small: about 35K–110K parameters and under 0.01G counted FLOPs across the three ImageNet depths.

  2. The reported ImageNet ranking is consistent across depths. CLAR has the highest Top-1 and Top-5 at all three depths in Table 3, with nearly the same parameter count and counted FLOPs as CBAM. This provides encouraging evidence across model depths, subject to the training differences noted below.

Critical weaknesses
  1. The contribution of input-dependent historical selection is not isolated. Dynamic weighting without dropping has lower mean Top-1 accuracy than static and uniform weighting at both depths. Neither control is tested with dropping, so the results cannot rule out a fixed mixture of gate logits with source dropping as sufficient to explain the gain. This leaves the benefit of input-dependent selection, central to Contribution 1 and the abstract, unresolved.

  2. Key comparisons needed to assess the design are missing or insufficiently controlled. The comparison with a feature-level adaptation of Attention Residuals is deferred to Appendix A (L53), which I could not find in the supplied materials, leaving the empirical motivation for switching to gate-level interaction unverifiable. In addition, the ImageNet runs differ in batch size and warmup. The reported ranking therefore does not establish CLAR's advantage under matched training settings.

Questions for authors
  1. Could you add the dynamic p = 0 row referenced in L350–352 to Table 2, and report static and uniform weighting trained with source dropping at p = 0.4? If these controls match CLAR, how would you qualify the contribution of input-dependent selection?

  2. Could you provide the Appendix A referenced at L53, including the controlled comparison with the feature-level adaptation of Attention Residuals?

  3. L403–404 says CLAR "matches the listed ResNet Top-1 accuracy and trails SE by 0.08 points" on CIFAR-100 ResNet-50, but Table 3 lists CLAR at 79.25 versus ResNet 78.83 and SE 78.93. Which results are correct?

  4. The ablation selects p = 0.4, but ImageNet and CIFAR ResNet-152 use p = 0.2 (L377). How was p = 0.2 chosen for these settings?

4 comments
AuthorsOpenReview ↗

(1/2) Comment for Reviewer ktcQ

We thank the reviewer for the valuable feedback. We have supplemented the experiments as extensively as our time and computational resources permit. We apologize for posting our response on the final day of the response period due to the substantial experimental workload. We are still refining the revised manuscript and welcome further feedback, which we will do our best to address in the revision.

Disentangling source weighting from source dropping (W1 & Q1)

We thank the reviewer for pointing out the inconsistent source-dropping settings in the original weighting comparison. We added experiments with static and uniform weighting at p=0.4p=0.4 and included the dynamic p=0p=0 results previously shown in Figure 2 but omitted from Table 2. The table below reports CIFAR-100 Top-1 accuracy as mean ± sample standard deviation over five random seeds.

WeightingppResNet-50 (%)ResNet-101 (%)
Uniform079.12 ± 0.3579.27 ± 0.42
Static078.95 ± 0.3279.39 ± 0.47
Dynamic078.70 ± 0.5479.01 ± 0.10
Uniform0.479.16 ± 0.1779.45 ± 0.28
Static0.479.11 ± 0.2379.41 ± 0.28
Dynamic0.479.25 ± 0.1379.59 ± 0.27

All three weighting schemes follow the same dropping rule: historical sources are randomly masked, the current source is always retained, and weights are renormalized over the retained sources. Under the same source-dropping probability of p=0.4p=0.4, dynamic weighting achieves higher mean Top-1 accuracy than both static and uniform weighting at both depths, supporting an additional contribution from input-dependent weighting beyond source dropping alone. We will include these complete comparisons in the revised manuscript and explicitly distinguish the roles of dynamic weighting and source dropping.

Missing appendix and feature-level comparison (W2 & Q2)

We apologize for omitting the appendix. We originally planned to upload a code archive and an appendix PDF separately, but discovered shortly before the submission deadline that the system allowed only one attachment and could not merge the appendix into the main document in time. We will provide the complete appendix in the revised manuscript.

The appendix describes AttnAgg, our feature-level adaptation of Attention Residuals to CNNs. AttnAgg retains full historical feature tensors and aggregates them through depth-wise attention, preserving local residual shortcuts and aligning historical features at stage transitions. The following table summarizes its results alongside ResNet and CLAR.

ModelParams (M)GFLOPsCIFAR-100 Top-1 (%)
ResNet-5023.702.6078.84 ± 0.17
+ AttnAgg23.744.1678.83 ± 0.18
+ CLAR26.262.6179.25 ± 0.13
ResNet-10142.695.0278.81 ± 0.13
+ AttnAgg42.767.7879.06 ± 0.07
+ CLAR47.525.0479.59 ± 0.27
ResNet-15258.347.4479.05 ± 0.11
+ AttnAgg58.4411.7179.19 ± 0.06
+ CLAR65.037.4779.80 ± 0.32

Under its matched protocol, AttnAgg provides limited accuracy gains over ResNet while increasing counted computation by approximately 55–60%. This motivated us to explore cross-layer retrieval over compact gate representations. We will provide more details in the complete appendix of the revised manuscript.

AuthorsOpenReview ↗

(2/2) Comment for Reviewer ktcQ

Matched training (W2)

To address the training-protocol differences, we completed ImageNet-1K comparisons of ResNet, SE, CBAM, MRLA-L, DLA-L, and CLAR with ResNet-50 and ResNet-101 under the same training settings, including batch size, learning rate, and warmup, using five common random seeds. The table reports mean ± sample standard deviation.

MethodR50 Top-1 (%)R50 Top-5 (%)R101 Top-1 (%)R101 Top-5 (%)
ResNet75.50 ± 0.1392.54 ± 0.0376.89 ± 0.2093.25 ± 0.10
SE77.36 ± 0.1093.67 ± 0.0478.37 ± 0.1194.13 ± 0.05
CBAM77.07 ± 0.2393.50 ± 0.2078.16 ± 0.1793.99 ± 0.07
MRLA-L77.04 ± 0.1093.42 ± 0.0777.96 ± 0.1793.96 ± 0.05
DLA-L76.91 ± 0.1093.28 ± 0.1077.79 ± 0.0493.80 ± 0.05
CLAR77.56 ± 0.1693.78 ± 0.0778.67 ± 0.0794.25 ± 0.03

Under matched training, CLAR achieves the highest mean Top-1 and Top-5 accuracy at both depths. We will update the main tables with these results and provide detailed training configurations in the appendix.

Correction to the CIFAR-100 results description (Q3)

We thank the reviewer for pointing out this inconsistency. The statement that CLAR "matches ResNet and trails SE by 0.08" is incorrect. In the original Table 3, CLAR achieves 79.25% Top-1 accuracy, exceeding ResNet (78.83%) and SE (78.93%) by 0.42 and 0.32 percentage points, respectively. We will correct this description and update both the text and tables consistently using the supplementary matched-training results.

Choice of dropping probability (Q4)

We used p=0.2p=0.2 in early exploratory experiments, initially running a single seed on CIFAR-100 ResNet-152 and on ImageNet to assess the method's performance. Subsequent CIFAR-100 ResNet-50/101 ablations showed higher mean Top-1 accuracy at p=0.4p=0.4, but we could not complete the additional training under alternative settings before the submission deadline. The manuscript therefore honestly reported the completed p=0.2p=0.2 results. This value was an exploratory setting rather than an optimum selected through a systematic search.

The additional controlled weighting comparisons address my concern about disentangling dynamic weighting from source dropping. The matched ImageNet results resolve the training-protocol concern, and the AttnAgg comparison clarifies the motivation for gate-level interaction.

Thank you again for the thorough response and additional experiments. I wish you all the best with your paper.

Thank you for your encouraging feedback. We are glad that the additional experiments and comparisons have addressed your concerns. We have now submitted the revised manuscript incorporating these results and clarifications.

In light of your reassessment, we would be grateful if you could also reconsider your overall score to reflect the updated evidence. Thank you again for your time and constructive suggestions.

Reviewer_QGCwOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CLAR, a CBAM-style attention module for ResNets with cross-layer selection. Inside each stage, every block stores a compact channel descriptor and a one-channel spatial map, and the current block uses query-key attention over its own and earlier descriptors (one scalar weight per source) to build its channel and spatial gates. Earlier sources are randomly dropped during training. Results on CIFAR-100, ImageNet-1K and COCO (Faster/Mask R-CNN) show small gains over SE, CBAM, MRLA-light and DLA-light, plus CIFAR ablations on the two branches, the weighting scheme and the drop rate.

Critical strengths
  1. The design is simple and cheap. Keeping only a channel vector and a single-channel spatial map per block, instead of full feature maps, avoids the memory blow-up of MRLA-base / DLA-base and lets the method train on ResNet-152. It is also easy to plug into existing residual CNNs.
  2. The authors are quite honest about the limits of their own setup. Ablations use five seeds with std, and the text openly says where comparisons are not controlled and that dynamic weighting alone is not shown to beat the simple alternatives. I appreciate this.
Critical weaknesses
  1. The core idea is thinner than the paper suggests. CLAR is basically CBAM plus a softmax mix over earlier blocks' gate descriptors plus source dropout, which is close to MRLA, DLA, DIANet and stochastic-depth-style regularization. More importantly, Table 2 shows that without dropping, dynamic weighting is actually worse than uniform and static weighting at both depths. So the gain seems to come mostly from the dropping trick, not from the "explicit selection" that the title and contributions are built around.
  2. The gains are tiny and the experiments are not controlled well enough to trust them. ImageNet gains over DLA-L are 0.19-0.29 Top-1 and COCO gains over MRLA-L are 0.1-0.2 AP, which is within normal seed noise for these setups. Baselines use different batch size, LR and warmup; the CIFAR ResNet-152 result is a single run; COCO baselines are copied from papers while CLAR is trained by the authors. Some baselines also look under-tuned: on CIFAR-100, MRLA-L and DLA-L are about 2 points below plain ResNet at every depth. And CIFAR has no validation split, so p seems to be chosen on the test set.
Questions for authors
  1. Can you re-run SE, CBAM, MRLA-L and DLA-L with exactly the same recipe and the same seeds as CLAR, at least on ImageNet ResNet-50, and report mean ± std?
  2. What do you get with plain CBAM + stochastic depth (or CBAM + uniform weighting + your source dropping)? This would tell us whether the cross-layer attention adds anything beyond the regularization.
  3. Your FLOP counter excludes elementwise ops, softmax and pooling, which is exactly what CLAR adds. Can you report measured latency/throughput and peak training memory against CBAM, especially at detection resolution?
  4. You use one scalar per source for all channels. Did you try per-channel or multi-head weights? If a single scalar is enough, why?
  5. Clarifications: the intro points to Appendix A, but there is no appendix in the PDF. The CIFAR text says CLAR "matches ResNet and trails SE by 0.08" at depth 50, but Table 3 shows 79.25 vs 78.83/78.93. Which is correct? How was p selected without a validation set?
4 comments
AuthorsOpenReview ↗

(1/3) Comment for Reviewer QGCw

We thank the reviewer for the valuable feedback. We have supplemented the experiments as extensively as our time and computational resources permit. We apologize for posting our response on the final day of the response period due to the substantial experimental workload. We are still refining the revised manuscript and welcome further feedback, which we will do our best to address in the revision.

Novelty comparison with related methods (W1)

The core design of CLAR is to reduce the cost of historical representations while retaining individual historical sources, allowing the current layer to access each source separately and recompute its weight based on the current input. This distinguishes CLAR from methods such as DIA, MRLA, and DLA.

Specifically, DIA uses an LSTM to accumulate information from previous layers into hidden and cell states, through which the current layer accesses historical information. MRLA-L propagates historical information by recursively updating the aggregated output Ot−1O^{t-1}, while DLA-L further uses context to update this aggregated representation. These methods effectively reuse historical information, but the historical sources have been summarized into recurrent states, and the current layer no longer computes a separate matching weight for each previous layer.

On the other hand, MRLA-Base and DLA-Base retain explicit source interactions at higher resource costs. Within the ResNet family, the ImageNet and COCO results reported for MRLA-Base focus on ResNet-50, and its appendix reports an out-of-memory issue for ResNet-101 at the original batch size; DLA-Base only reports results on CIFAR. In our reproductions, both Base methods with ResNet-101 exhibited training divergence within the first ten epochs. These observations motivated us to explore a more scalable design for explicit historical selection.

Inspired by CBAM, CLAR stores compact channel and spatial representations for each historical source, which the current layer's queries individually matches and reweights. This reduces explicit history storage from O(LCHW)O(LCHW) to O(L(C+HW))O(L(C+HW)) while preserving source-specific retrieval. Our ResNet-152 experiments demonstrate the feasibility and effectiveness of this design in deep networks.

The table below summarizes the key design differences between CLAR and related cross-layer attention methods.

MethodHistorical representationHow history is accessedRecalibration or aggregation target
DIARecurrent hidden and cell statesCurrent information fused with historical statesAttention recalibration
MRLA-LRecurrent aggregated outputHistorical aggregate combined with the current contributionFull features
DLA-LContext-updated recurrent aggregateHistorical aggregate updated using context and combined with the current contributionFull features
MRLA-BaseIndividually retained historical feature representationsExplicit attention over historical sourcesFull features
DLA-BaseIndividually retained historical feature representationsContext-based refreshing of historical features followed by explicit attention over sourcesFull features
CLARSeparate compact channel and spatial representations for each sourceCurrent queries separately match and reweight individual sourcesChannel and spatial gates

Difference between source dropping and stochastic depth (W1)

Historical source dropping and stochastic depth both regularize training through random masking, but act on different components. Stochastic depth randomly skips residual branches and passes information through shortcuts, changing the effective network depth during training. In CLAR, historical source dropping instead acts on the current layer's access to historical sources. All residual blocks are still computed, and their historical representations are fully retained. When constructing the current layer's attention gates, a subset of historical sources is randomly excluded, the weights of the remaining sources are renormalized, and the current layer's own source is always retained.

AuthorsOpenReview ↗

(2/3) Comment for Reviewer QGCw

Disentangling source weighting from source dropping (W1 & Q2)

Q2 asked us to provide results of 'CBAM + stochastic depth' or 'CBAM + uniform weighting + your source dropping'. As we just clarified, stochastic depth acts on entire residual branches, whereas source dropping acts on access to historical sources. CBAM + stochastic depth evaluates the benefit of general residual regularization; our focus is on the contribution of dynamic historical selection under source dropping. We therefore adopted the reviewer’s alternative control, uniform weighting + source dropping, and additionally evaluated static weighting at p=0.4p=0.4. The table below reports CIFAR-100 Top-1 accuracy as mean ± standard deviation over five random seeds.

WeightingppResNet-50 (%)ResNet-101 (%)
Uniform079.12 ± 0.3579.27 ± 0.42
Static078.95 ± 0.3279.39 ± 0.47
Dynamic078.70 ± 0.5479.01 ± 0.10
Uniform0.479.16 ± 0.1779.45 ± 0.28
Static0.479.11 ± 0.2379.41 ± 0.28
Dynamic0.479.25 ± 0.1379.59 ± 0.27

Under the same source-dropping probability of p=0.4p=0.4, dynamic weighting achieves higher mean Top-1 accuracy than both static and uniform weighting at both depths. These results suggest dynamic weighting contributes to final performance beyond source dropping alone. We will include these complete comparisons in the revised manuscript.

Matched training and repeated runs (W2 & Q1)

We have supplemented the evaluation with matched comparisons and clarify the selection protocol below. On ImageNet-1K, we completed comparisons of ResNet, SE, CBAM, MRLA-L, DLA-L, and CLAR with ResNet-50 and ResNet-101 under the same training settings and five common random seeds. The table reports mean ± sample standard deviation.

MethodR50 Top-1 (%)R50 Top-5 (%)R101 Top-1 (%)R101 Top-5 (%)
ResNet75.50 ± 0.1392.54 ± 0.0376.89 ± 0.2093.25 ± 0.10
SE77.36 ± 0.1093.67 ± 0.0478.37 ± 0.1194.13 ± 0.05
CBAM77.07 ± 0.2393.50 ± 0.2078.16 ± 0.1793.99 ± 0.07
MRLA-L77.04 ± 0.1093.42 ± 0.0777.96 ± 0.1793.96 ± 0.05
DLA-L76.91 ± 0.1093.28 ± 0.1077.79 ± 0.0493.80 ± 0.05
CLAR77.56 ± 0.1693.78 ± 0.0778.67 ± 0.0794.25 ± 0.03

Under matched training, CLAR achieves the highest mean Top-1 and Top-5 accuracy at both depths. We also completed matched CIFAR-100 comparisons, including five runs for CLAR with ResNet-152, and reproduced all COCO baselines under the same conditions as CLAR rather than relying on published reference results. We will update the main tables accordingly and provide detailed training configurations in the appendix.

ImageNet training with ResNet-152 is substantially more time-consuming: one MRLA-L run takes approximately 100 hours on four RTX 4090 GPUs with 48 GB memory each. Despite continuous training since submission and renting an additional cloud server with the same four-GPU configuration, the full comparison across all methods and seeds remains incomplete. We will explicitly mark table entries based on fewer than five seeds and specify the number of completed runs.

Under the matched training recipe, MRLA-L and DLA-L still underperform plain ResNet on CIFAR-100. We have not established the cause of this gap or conducted separate hyperparameter searches for each method. These results therefore characterize performance under a common recipe rather than each method’s individually optimized performance.

Validation split and selection of p (W2 & Q5)

Our CIFAR experiments held out 10% of the training set for validation and used validation accuracy to select checkpoints. The dropping probability pp, however, was selected based on the test-set ablation results.

AuthorsOpenReview ↗

(3/3) Comment for Reviewer QGCw

Measured runtime costs (Q3)

We supplemented the evaluation with an accuracy–cost comparison of Faster R-CNN ResNet-50 at detection resolution. Cost measurements use an RTX 4090, FP32, and inputs of 800×1344800\times1344. Throughput and peak training memory use batch size 2; latency uses batch size 1. Inference timing uses 1,000 fixed proposals per image and covers the backbone, FPN, RPN logits, and ROI head, excluding NMS.

MethodBox AP ↑Throughput (images/s) ↑Latency (ms) ↓Peak training memory (GiB) ↓
CBAM38.6740.2325.694.66
MRLA-L40.1744.7320.645.25
DLA-L40.0836.3225.895.77
CLAR40.4239.2026.704.67

Compared with CBAM, CLAR improves box AP by 1.75 points with similar peak training memory, approximately 2.6% lower throughput, and 3.9% higher latency. We will provide results for all evaluated methods and depths, together with the detailed measurement protocol, in the appendix.

Source-weight granularity (Q4)

We use standard single-head cross-attention to compute a scalar weight for each historical source and aggregate the corresponding channel value vectors. The resulting gates therefore remain channel-specific. We also considered a multi-head design, but have not completed the experiments due to time and computational resource constraints. We thank the reviewer for suggesting per-channel and multi-head weighting, which we consider valuable extensions and plan to evaluate in future work.

Missing appendix (Q5)

We apologize for omitting the appendix. We originally planned to upload a code archive and an appendix PDF separately, but discovered shortly before the submission deadline that the system allowed only one attachment and could not merge the appendix into the main document in time. We will provide the complete appendix in the revised manuscript.

Correction to the CIFAR-100 results description (Q5)

We thank the reviewer for pointing out this inconsistency. The statement that CLAR "matches ResNet and trails SE by 0.08" is incorrect. In the original Table 3, CLAR achieves 79.25% Top-1 accuracy, exceeding ResNet (78.83%) and SE (78.93%) by 0.42 and 0.32 percentage points, respectively. We will correct this description and update both the text and tables consistently using the supplementary matched-training results.

Thank you again for your time and constructive feedback. We have submitted the revised manuscript incorporating the additional experiments, updated comparisons, and clarifications discussed in our response, together with the complete appendix.

We would appreciate your consideration of the revised manuscript and our responses, and would be grateful if you could reconsider your overall assessment and score in light of the additional evidence.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for the Updated Manuscript

We thank all reviewers for their constructive feedback. We have submitted a revised manuscript with the following main updates:

  • Matched comparisons: Updated CIFAR-100 and ImageNet-1K results under common training settings, including five-seed ImageNet comparisons at depths 50 and 101 and explicit labels for single-run entries.

  • Controlled ablations: Added complete dynamic, static, and uniform weighting comparisons across source-dropping probabilities, as well as supplementary single-run branch observations on ImageNet.

  • COCO evaluation: Added CBAM and DLA-L baselines under matched detector training settings and updated the discussion to reflect differences across detectors, depths, and metrics.

  • Accuracy and cost: Added measured throughput, latency, and training memory comparisons, with complete results and measurement protocols in the appendix.

  • Clarifications and appendix: Expanded the related-work discussion, clarified the motivation for gate-level interaction, corrected inconsistent descriptions, and provided the complete appendix covering training details, per-seed statistics, the feature-level adaptation, and the method comparison matrix.

We hope these revisions address the concerns raised and make the evidence and presentation clearer. We would appreciate the reviewers’ consideration of the updated manuscript when reassessing the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.