acceptodds
Under review as a conference paper at ICLR 2027

Width Expansion via Parallel Block Interaction for Vision Model Training

Abstract

Increasing the width of a vision backbone improves its representation capacity. However, directly widening increases the cost of dense channel processing. This paper argues that block-level lateral connectivity, a dimension largely abandoned since the advent of deep learning, deserves to be reopened. We propose Parallel Block Interaction (PBI), a general block-level width expansion framework that distributes the feature channels across multiple parallel blocks while enabling lightweight information exchange among them. Each parallel block retains the original backbone-specific block design while processing a subset of the channels. The shared public channels provide an internal information path across parallel blocks without changing the feature width passed to downstream modules. Public-to-public, private-to-public, and private-to-private channel interactions enable sufficient and controllable information exchange across blocks. On ConvNeXt, ViT, Swin Transformer, and VMamba backbones, PBI improves image classification and dense prediction performance under comparable parameter and computation budgets. Comparisons with direct channel expansion and Mixture-of-Experts, and full ablation studies show favorable accuracy–efficiency trade-offs and clear contribution of each component. Class-level and spatial-region analysis further validates that PBI strengthens the representation diversity of the parallel blocks. PBI reveals the advantage of width expansion via block-level intra-layer connectivity, a potential model scaling dimension orthogonal to “going deep”.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer Vision

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hjSUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed Parallel Block Interaction (PBI), a universal module to increase the feature width of a vision model using limited extra cost. Instead of processing all CC channels with one wide block, it divides them among GG narrower and independently parameterized blocks. Each block processes its own private channels with a small set of public channels. Through pub-to-pub, priv-to-pub, and priv-to-priv communication, PBI tries to recover useful interaction loss due to grouping, so that the saved computation can then support a wider representation.

The main results are consistent across different architectures, backbones, datasets, and tasks. Although these results remain modest relative to the original backbones, they can support the main motivation of PBI and demonstrate that PBI can work for different model types (CNN, transformer, Mamba). Therefore, PBI can be a plausible and potentially useful architectural contribution.

Critical strengths
  1. PBI is a genuinely architecture-agnostic mechanism. PBI operates on the channel interface. Concretely, PBI partitions channels into groups, runs the unmodified block per group, and exchanges a small number of shared states. In this way, PBI can be applied to convolutional, attention, and space-space blocks without touching their internal architectures. The cross-family results (including ConvNext, DeiT, Swin, VMamba) support this claim, and the design is easy to reproduce.

  2. The motivation is very clean and well supported by the cost analysis/calculation. The 1/G1/G saving is easy to verify by inference time and memory measurement (Figure 2)

  3. The empirical coverage is board. Three architectures are examined and tested across different tasks, i.e., ImageNet-1K, COCO/ADE20K, and ImageNet-21K pretraining is substantially more than a single-backbone/task demonstration.

  4. The component ablation is informative. Pub-to-pub is correctly identified as the dominant contributor. The interactions are complementary, and the paper compares against channel shuffle and pointwise mixing. Tables A8/A9 transparently explain why configurations with public channels can have fewer parameters/FLOPs.

Critical weaknesses
  1. I think my personal biggest concern is there is no matched-budget dense-widening control on a real backbone. The central claim ("better than widening") is insufficiently tested at equal compute. Table 1 compares PBI to the original narrow model (Swin-T: 29M/4.5G →\rightarrow 33M/5.2G, i.e., +14% parameters and +16% FLOPs for +1.2 points), and Table 4 compares at equal representation width, not equal compute. Eq. A32 gives the conversion, and the 3-layer ViT at 9.3G corresponds to a dense width of 1130, so the control is cheap. Without it, the gain (+1.17 over the dense backbone) cannot be separated from simply spending more capacity.

  2. The conceptual contribution is relatively thin. Each ingredient has a close precedent: grouped channel processing for cheaper width (ResNeXt: Xie et al., 2017; ShuffleNet: Zhang et al., 2018), a shared latent that modules read from and write to (Shared Global Workspace: Goyal et al., 2021), multi-branch blocks (Inception: Szegedy et al., 2016; vision MoE: Riquelme et al., 2021; Xue et al., 2022), and compact cross-branch channel mixing (SCHEME: Sridhar et al., 2023). These work are mostly cited in related work, but a justification on this paper’s contribution is expected. Also, it is natural to compare the performance with Shared Global Workspace/SCHEME at matched budget.

  3. Some results are not convincing. For example, Dense + 3072 in Table 4 shows NaN seems to be a problem of training setup rather than an limitation of dense widening. Meanwhile, Appendix E.3 states that the pointwise-mixing baseline uses "no additional LayerNorm, activation, or residual connection", which explains why it collapses to 72.86%. Therefore, Table 6 does not fairly represent that public sharing is the better interaction mechanism.

  4. The elaboration on why PBI is working is suboptimal. My concern is, CKA, Grad-CAM, and classifier-weight magnitudes are correlational, and weight magnitude ignores activations. More specifically, the private-to-public interaction is motivated as an anti-oversmoothing measure, yet it supplies the same updated public state to every branch. It is unclear why identical shared input should reduce inter-branch similarity. Results of inter-branch CKA with/without Pr →\rightarrow Pu are maybe helpful.

Questions for authors

Questions are basically corresponding to the weaknesses above.

  1. Why is there no matched-budget dense-widening control? Or it is already represented in existing experiments? A single run with the same recipe would directly test the central claim. Without it, how should we attribute the performance gain to PBI rather than to extra capacity?

  2. What is the empirical results of SCHEME-style grouped channel mixing and a Shared-Global-Workspace-style shared latent at matched parameters/FLOPs?

  3. What training recipe produced NaN for Dense + 3072, and was it tuned (LR, initialization, normalization, drop path)? Would a tuned dense model train, and at what accuracy? And was the pointwise-mixing baseline re-run with LayerNorm and a residual connection?

  4. How does the priv-to-pub update reduce oversmoothing when the updated public state is identical across branches? Please report inter-branch similarity (e.g., CKA with/without Pr →\rightarrow Pu)?

I am open to reconsidering the final assessment if these questions can be addressed, especially for 1 and 4.

4 comments
AuthorsOpenReview ↗

Response to Reviewer hjSU (part1)

Thank you for your careful review and constructive suggestions. We address the four concerns below, including the new matched-budget comparisons and analyses of the interaction mechanisms.

W1/Q1. Matched-budget dense-widening controls and whether gains come from added capacity

A1. PBI outperforms dense widening on both the 12-layer and 3-layer ViTs at comparable computational budgets. These new comparisons directly examine whether the gains can be explained by additional computational capacity alone.

12-layer ViT. Starting from the ViT-S design, we compared dense widening, MoE widening, dense deepening, and PBI widening using the same ImageNet-1K training recipe for 100 epochs. The results are reported in Table 6, with configurations and training details in Appendix E.

Expansion strategyConfigurationParams (M)GMACTop-1 (%)
Dense widening12-layer, width 5764910.077.9
MoE widening12-layer, width 576, 2 FFN experts8110.078.7
Dense deepening27 layers, width 3844910.376.4
PBI widening12-layer, 2 blocks, 288 private channels each, P=96P=96499.979.3

At approximately 49M parameters and 10 GMAC, PBI exceeds dense widening by 1.4 percentage points and dense deepening by 2.9 points. MoE has similar computation but more parameters (81M), and remains 0.6 points below PBI.

3-layer ViT. We also added Dense and MoE controls at the 9.3G and 11.8G budgets of the PBI models (Table 5). At 9.3G, the dense width is 1120, close to your estimate of 1130.

BudgetModelParams (M)GMACTop-1 (%)
9.3GDense, width 1120479.367.0
9.3GMoE, width 1120, 2 experts789.365.6
9.3GPBI499.374.9
11.8GDense, width 12646011.870.8
11.8GMoE, width 1264, 4 experts17511.870.9
11.8GPBI6311.876.5

Dense and MoE use per-head QK normalization at both budgets. PBI exceeds dense widening by 7.9 and 5.7 percentage points, respectively. Unlike the equal-representation-width comparison in Table 4, these controls guarantee that all the methods are compared under the same computational budgets and provide more direct evidence for the effectiveness of PBI's expansion strategy.

W2/Q2. Differences from SCHEME and Shared Global Workspace and matched-budget comparisons

A2. Unlike SCHEME's grouped feature mixing and SGW's separate shared workspace, PBI enables information exchange among complete backbone blocks through shared public channels while preserving their private representations. Our budget-control comparisons further show that PBI outperforms both SCHEME-style and SGW-style adaptations.

SCHEME uses channel covariance attention to mix grouped FFN features, while Shared Global Workspace (SGW) introduces a separate shared state accessed through read and write operations. PBI instead maintains separate private feature streams across successive backbone layers and exchanges information between blocks through shared public channels. This design preserves the original internal convolution, attention, or state-space operators.

We compared three interaction mechanisms in a two-block, 12-layer ViT, including SCHEME-style CCA, SGW-style shared workspace, and PBI-S (ViT with PBI interactions). All models were trained on ImageNet-1K for 100 epochs (Table 9).

Interaction mechanismParams (M)GMACTop-1 (%)
SCHEME-style CCA225.270.4
SGW-style workspace224.470.6
PBI-S214.677.1

At comparable parameter and computational budgets, PBI-S exceeds the SCHEME-style and SGW-style adaptations by 6.7 and 6.5 percentage points, respectively.

AuthorsOpenReview ↗

Response to Reviewer hjSU (part2)

W3/Q3. Training instability in dense widening and fairness of the pointwise-mixing baseline

A3. QK normalization stabilizes equal-budget dense widening, while Pre-LN and residual mixing improve the pointwise-mixing baseline. PBI remains more accurate than both.

Dense widening. The original width-3072 dense run (Table 4) encountered NaN gradients that corrupted its weights. Filtering did not stabilize training, with accuracy falling from about 30% to 5%. We observed large QK dot products and concentrated attention distributions. Per-head QK LayerNorm stabilized Dense and MoE at both 9.3G and 11.8G (Table 5). Dense then reaches 67.0% and 70.8%, compared with 74.9% and 76.5% for PBI.

Pointwise mixing. We agree that the original baseline lacked normalization and a residual connection. We added Pre-LN and residual mixing and tested GELU under the same four-block Swin-T, 100-epoch setup (Table 8, Appendix E.4).

Interaction method (four-block Swin-T)Params (M)FLOPs (G)Top-1 (%)
No additional within-stage interaction33.875.20377.51
Channel shuffling33.875.20378.37
Original pointwise linear mixing42.516.59172.86
Pointwise mixing + Pre-LN + residual42.526.60579.47
Pointwise mixing + Pre-LN + residual + GELU42.536.60579.05
Public-channel sharing31.774.98581.60

The improved mixer rises from 72.86% to 79.47%, supporting your concern about the original optimization setup. Public-channel sharing still achieves 2.13 percentage points higher accuracy than this improved baseline, with fewer parameters and lower reported FLOPs. We have updated the manuscript with the new experiment results.

W4/Q4. The role of Priv-to-Pub with identical public inputs and its effect on inter-block CKA

A4. Priv-to-Pub introduces input-dependent shared information before the first layer of each stage. It is not intended to directly minimize inter-block similarity. Although all blocks receive the same public state, they retain different private features and independent block parameters.

We agree that our original explanation did not adequately clarify the role of Priv-to-Pub when all blocks receive the same public update. With Pub-to-Pub alone, information from private features enters the shared public state after the first layer's block computations and aggregation. Priv-to-Pub instead introduces information from private features at stage entry, making input-dependent shared context available from the first layer. Each block processes this shared information together with its own private features using independent parameters. We have revised the manuscript to clarify this mechanism and its role in cross-block information exchange, and removed the previous claim on over-smoothing properties to avoid misinterpretation.

Public and private CKA. We measured inter-block CKA on the same 10,000 ImageNet-1K validation images using four-block Swin-T models, with Pub-to-Pub and Priv-to-Priv enabled and Priv-to-Pub toggled off or on. CKA was computed before public aggregation and averaged over the six block pairs (Table 10).

Feature typeMean CKA, 12 layers (off → on)Final-layer CKA (off → on)
Private channels0.5522 → 0.48760.6359 → 0.6365
Public channels0.6040 → 0.76010.8257 → 0.9996
Full features0.5802 → 0.69760.6371 → 0.9907

Private CKA decreases in 8 of 12 layers, with a lower mean but almost no change at the final layer. Public and full-feature CKA increase. Thus, higher similarity in shared public features does not necessarily indicate high similarity among private representations.

Input-dependent shared information. Because CKA and visualizations are correlational, we also performed inference interventions with fixed model weights. Across all 50,000 ImageNet-1K validation images, replacing each image's Priv-to-Pub output with that of another image reduces Top-1 accuracy from 82.44% to 72.49% or 72.47%, while the mean correct-class logit gap falls from 3.38 to 2.30 or 2.31 (Table 11). On a 10,000-image subset, mismatching only the final stage reduces accuracy from 82.54% to 76.30% (Appendix E.6). These results show that predictions depend on shared information matched to the input.

Training ablation. With Pub-to-Pub and Priv-to-Priv enabled, adding Priv-to-Pub improves Top-1 accuracy from 82.24% to 82.47% (Table 7), providing complementary evidence of its contribution.

The rebuttal substantially addresses my primary concern through matched-budget dense-widening comparisons, including a standard 12-layer ViT and a 300-epoch evaluation. The corrected pointwise-mixing baseline also strengthens the empirical evidence. I am revising my assessment upward toward weak acceptance.

Thank you for your encouraging feedback. We sincerely appreciate your constructive suggestion, particularly on the computational budget comparisons and pointwise-mixing baseline, which strengthen our experimental evaluation. We will continue refining the manuscript to improve its clarity and presentation. Thank you again for your time and careful consideration.

Reviewer_HSpnOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a width-expansion framework with Parallel Block Interaction (PBI) module for vision models to improve the representation capacity with wider backbones. It introduces public-to-public, private-to-public, and private-to-private channel interactions to exchange information across different combinations of shared and private channels. PBI is evaluated across several vision architectures such as ConvNext, Swin, DeiT, Mamba etc and across different downstream tasks such as classification, detection and segmentation.

Critical strengths
  • The idea of public and private channels is interesting and simple. It keeps each backbone’s block design intact, so it applies across convolutional, attention, and state-space backbones without redesigning operators.
  • The evaluation covers four backbone families, three model sizes for Swin and VMamba, dense prediction, and ImageNet-21K pretraining.
  • Ablation studies are fairly comprehensive
Critical weaknesses
  • Novelty of the method is incremental as width expansion has been studied extensively in several prior works Inception, ResNeXt and grouped convolutions, ShuffleNet, CondenseNet, and MoE etc.

  • The related work misses two closely related research works. 1. Parallel ViT blocks (Touvron et al., 2022, “Three things everyone should know about Vision Transformers”), and 2. Packed-Ensembles for Efficient Uncertainty Estimation (Laurent et al., ICLR 2023). Discussing how PBI differs from these would help clarify its contribution.

  • The usefulness of the proposed method beyond accuracy is not clearly shown since the performance improvement is also not significant. The class-preference analysis in Fig. A10 compares separate PBI blocks against channel groups taken from a dense model. Those dense groups are arbitrary, since nothing in a dense layer organizes its channels into groups, so it’s expected that their class profiles look uniform. Grad-CAM results are only three qualitative examples.

  • The performance improvements are marginal and is decreasing with increasing model size. Gains at the larger sizes are +0.3 to +0.5 (Swin-B, VMamba-S/B, Swin-B/L on ImageNet-21K), and the small-scale gains come at increased parameters or FLOPs.

  • Does the method work with smaller models like mobile or edge optimized models (fewer than 10 or 5 M parameters)? It is unclear whether PBI helps mobile or edge models under 10M parameters, where channel widths are already narrow, and splitting them further may hurt.

  • Computational cost is only reported in comparison to other blocks not with other models in Table 1 and Table 2. The real word latency (or throughput) introduced by the method can be significant despite the lower params and FLOPs due to the narrow parallel blocks plus averaging and broadcasting.

  • “Swin-B + PBI” model has different costs in Table 1 and Table 3. This suggests a different PBI configuration was used for the ImageNet-21K experiments, and the paper doesn’t explain why. It shows that PBI may be sensitive to its hyperparameters and needs adjustment for every setting.

  • In Table 6, pointwise 1×1 mixing (72.86%) does worse than no interaction at all (77.51%), despite having more parameters. This might be an optimization or training problem rather than the limits of dense channel mixing.

Questions for authors
  • See weaknesses above.
  • What are the training settings (epochs, recipe) for each 3-layer ViT experiment (Table 4, Fig. 3a, 4b, A7)? The 3-layer ViT results also vary widely across experiments: about 62-63.5% in Fig. A7, 63-66% in Fig. 4b, and 70-76% in Table 4.
  • For ConvNeXt-T, the classification backbone grows from 29M to 32M, yet the Mask R-CNN model stays at 48M/262G. Is it the same model?
7 comments
AuthorsOpenReview ↗

Response to Reviewer HSpn (part1)

Thank you for your detailed review and suggestions. We address each concern below.

W1: Contribution of PBI relative to grouped, multi-branch, and MoE methods.

A1. PBI introduces a block-level approach to GENERAL width expansion, enabling lightweight information exchange among complete backbone blocks without redesigning their internal operators.

Inception combines parallel transformations, ResNeXt aggregates grouped transformations, and CondenseNet learns structured convolutional connectivity. These approaches mainly organize computation within particular modules or operators. PBI instead maintains separate private feature streams across successive backbone layers and exchanges information through shared public channels.

ShuffleNet exchanges information by reassigning channels across groups, whereas PBI preserves each block's private features and shares information through public channels. In the four-block Swin-T comparison (Table 8), public-channel sharing achieves 81.60% Top-1 versus 78.37% for channel shuffling. Vision MoE typically increases parameter capacity through routed FFN experts without necessarily widening the representation retained for each input. PBI processes every input through all parallel blocks to form a wider representation.

The distinction is therefore the level at which width expansion and information exchange are organized. PBI preserves the original convolution, attention, or state-space operators while introducing cross-block connectivity. Its results on ConvNeXt, DeiT, Swin, and VMamba, as well as COCO and ADE20K, support this architecture-level applicability. We have clarified this contribution in the revised manuscript.

W2. Differences from Parallel ViT blocks and Packed-Ensembles

A2. PBI forms a single model from interacting parallel blocks, unlike the independent predictive members of Packed-Ensembles or the parallel operations within a Parallel ViT block. We provide the comparisons in Appendix E.5 (Table A16).

Packed-Ensembles. Packed-Ensembles executes multiple predictive members using grouped operations and combines their predictions at the output. PBI exchanges features between blocks across layers and uses one classification head over their concatenated private features. Our four-member Packed-Ensembles adaptation of ConvNeXt-T achieves the following ImageNet-1K results.

ConvNeXt-T modelParams (M)Reported computeTop-1 (%)
Packed-Ensembles adaptation325.0 GMAC80.1
PBI324.6G82.6

At the same reported parameter count, PBI achieves 2.5 percentage points higher accuracy.

Parallel ViT. Parallel ViT allows multiple attention modules to process the same representation and sums their outputs, with a similar arrangement for FFNs. PBI retains separate private feature streams and the original internal structure of the backbone blocks. The published Parallel ViT result and our 300-epoch PBI experiment give the following comparison.

ViT modelEpochsParams (M)GMACTop-1 (%)
Parallel ViT-S12×2 (published)300459.581.6
12-layer ViT-S + PBI (2 blocks)300499.982.4

PBI achieves 0.8 percentage points higher accuracy than the published Parallel ViT result, with slightly higher parameters and computation. We use the same PBI configuration as in our 12-layer width-expansion experiments, which was originally designed for comparison with dense widening at a similar budget. This accounts for the differences in parameters and computation between PBI and Parallel ViT.

AuthorsOpenReview ↗

Response to Reviewer HSpn (part2)

W3. Class-preference analysis, Grad-CAM, and value beyond accuracy

A3. Fig. A10 compares all output channels of the dense model with the actual PBI branches, rather than arbitrarily partitioning the dense model into groups.

Fig. A10(a) reports the mean absolute classification-head weights over all dense-model output channels. Figs. A10(b)–(d) report the same quantity separately for each PBI branch. The dense model shows relatively uniform class-wise magnitudes, while PBI branches exhibit different class associations. For example, Block 1 has larger magnitudes for higher-indexed classes, whereas Block 2 is more strongly associated with lower-indexed classes. We have revised the main text and the caption of Fig. A10 to clarify how the channels are compared.

Value beyond accuracy. PBI provides a general width-expansion design through lateral connections among blocks, without modifying their internal operators. It is applicable to convolutional, Transformer, and visual state-space backbones and can be transferred to COCO detection and ADE20K segmentation. Its contribution is therefore not limited to a classification gain on one architecture. More broadly, separating private and public features provides a compact interface for cross-block information exchange, potentially allowing external knowledge to be introduced through public channels to guide model predictions in a more controllable manner.

Additional evidence. Grad-CAM illustrates differences in spatial responses, while the input-mismatch intervention tests whether predictions use the shared information. With Swin-T + PBI weights fixed, replacing each image's Priv-to-Pub output with another image's output reduces Top-1 accuracy from 82.44% to 72.49% and 72.47% across two permutations (Table 11). The mean correct-class logit gap also falls from 3.38 to 2.30 and 2.31. This provides evidence of input dependence beyond the three qualitative Grad-CAM examples.

W4. Whether gains diminish with model scale or require more computation

A4. PBI's gains do not decline monotonically with model scale, and several improvements come with fewer parameters and less computation.

All eight paired ImageNet-1K backbone comparisons in Table 1 improve, with an average gain of 0.54 percentage points. Swin-B improves by 0.5 points, compared with 0.4 for Swin-S. After ImageNet-21K pretraining, Swin-B and the larger Swin-L improve by 0.3 and 0.4 points, respectively (Table 3). These results do not show a monotonic decrease with scale.

The gains also do not always require greater cost. Swin-B improves from 83.5% to 84.0% while dropping from 88M/15.4G to 87M/15.3G. VMamba-B improves from 83.9% to 84.2% while dropping from 89M/15.4G to 87M/15.1G. DeiT-S improves from 79.8% to 80.0% at the same budget 22M/4.6G.

Improvements at this magnitude also occur across backbones. For example, ConvNeXt-B and FocalNet-B exceed Swin-B by 0.3 and 0.4 points, respectively, in the reported ImageNet comparisons.

A further 300-epoch, near-matched-budget 12-layer ViT comparison in Appendix E.5 achieves 82.4% with PBI at 48.5M/9.9 GMAC, versus 82.0% for dense widening at 48.9M/10.0 GMAC.

W5. Applicability below 10M parameters

A5. Two-block PBI also works in our tested 5.7M-parameter ViT-Tiny setting without increasing the reported parameters or computation. The comparison is included in Appendix E.5.

ModelParams (M)GMACTop-1 (%)
ViT-Tiny5.71.366.7
PBI-Tiny (2 blocks)5.71.366.9

PBI-Tiny improves Top-1 by 0.2 percentage points, showing that channel partitioning does not necessarily hurt accuracy below 10M parameters. Mobile and edge-optimized architectures involve additional design considerations, such as hardware efficiency and operator selection, which fall outside the scope of this work, and we will investigate this issue in the future.

AuthorsOpenReview ↗

Response to Reviewer HSpn (part3)

W6. Full-model cost and inference efficiency

A6. Tables 1 and 2 report complete-model parameters and FLOPs, and Fig. 2 provides measured inference time and memory. Table 1 includes the full classifier. Table 2 includes the detection backbone, feature pyramid, and detection head.

For the three-layer ViT expansion comparison in Fig. 2, the measured results are as follows.

ModelInference time (ms)Inference memory (GB)
Dense widening2.562.88
MoE widening2.162.76
PBI without public channels or interactions1.511.82
Full PBI1.662.05

W7. Different Swin-B costs in Tables 1 and 3

A7. Both Swin-B + PBI variants use identical core hyperparameters GG, dd, and PP. Their different costs arise from the Priv-to-Pub encoder implementation, not a change in branch count or channel allocation. The configurations are specified in Appendix E.2 (Table A13).

Both variants use three branches and identical stage-wise private and public widths. Table 1 uses MLP encoders at all stages and reports 87M/15.3G. Table 3 uses lightweight Swin encoders in the first two stages and MLPs in the last two, reporting 89M/15.6G. Swin encoders also model local spatial interactions, whereas MLP encoders provide lighter feature transformations.

Both implementations improve over their corresponding baselines by 0.5 points on ImageNet-1K and 0.3 points after ImageNet-21K pretraining, respectively, demonstrating that PBI is effective with either encoder implementation.

W8. Fairness of the pointwise-mixing baseline

A8. The original pointwise-mixing baseline was affected by its optimization setup. Pre-LN and residual mixing substantially improve it, but public-channel sharing still achieves higher accuracy with lower cost. The revised baselines appear in Table 8, with the original mixer and implementation details described in Appendix E.4.

Interaction method (four-block Swin-T)Params (M)FLOPs (G)Top-1 (%)
No additional within-stage interaction33.875.20377.51
Channel shuffling33.875.20378.37
Original pointwise linear mixing42.516.59172.86
Pointwise mixing + Pre-LN + residual42.526.60579.47
Pointwise mixing + Pre-LN + residual + GELU42.536.60579.05
Public-channel sharing31.774.98581.60

The improved pointwise mixer rises from 72.86% to 79.47%, confirming that the original setup limited its performance. Public-channel sharing still exceeds the improved mixer by 2.13 percentage points, with fewer parameters and lower reported FLOPs. This comparison supports its advantage under the evaluated setup.

Reviewer_DugkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Parallel Block Interaction (PBI), a framework for width expansion in vision backbones. PBI exchanges information across blocks through three distinct interaction pathways, organizing the processing of wider representations into narrower parallel blocks and explicit information exchange pathways while preserving the original backbone-specific block design. By partitioning operations whose computational cost grows quadratically with channel width, the proposed framework mitigates the cost of direct width expansion. The paper evaluates multiple backbones on benchmarks including ImageNet-1K, ImageNet-21K, COCO, and ADE20K.

Critical strengths

(S1) The paper demonstrates improvements across multiple backbones, including ConvNeXt, DeiT, Swin, and VMamba, on image classification, object detection, and segmentation tasks. (S2) The ablation studies and analyses, particularly Table 5, demonstrate the practical benefits of information exchange among parallel blocks.

Critical weaknesses

Please identify one or two critical weaknesses of the paper that support rejection. For each, explain briefly why it is consequential to your overall assessment. The goal is not to enumerate every negative point, but to identify and substantiate the considerations that most strongly determine your recommendation. Enter “None” if no material weakness can be identified.

(W1) Although improvements are demonstrated across multiple backbones, the performance gains are modest, and the central comparison with alternative expansion methods is limited to a three-layer ViT. Comparisons against baselines that increase depth or jointly adjust depth and width under comparable parameter and computation budgets would help establish whether PBI is a competitive model expansion strategy.

(W2) Although analyses of G, P are provided, the model-specific settings and selection procedures for these architectural hyperparameters, as well as whether the interaction coefficients are learned, are not sufficiently specified.

(W3) (L.208) The authors introduce private-to-public channel interaction as a countermeasure against the risk of over-smoothing. While this design is plausible, Figure 4(a) does not show a consistent reduction in similarity when comparing the orange curve (public-to-public) with the pink curve (public-to-public + private-to-public). At some later layers, the orange curve exhibits lower inter-block similarity. Thus, the interpretation that private-to-public interaction mitigates over-smoothing may not be supported by this analysis.

Questions for authors

Ask only questions whose answers could materially affect your assessment. Distinguish clarification requests from suggestions for future work.

See major weaknesses (W1, W2, W3).

3 comments
AuthorsOpenReview ↗

Response to Reviewer Dugk (part1)

Thank you for your careful review and constructive suggestions. We address the three concerns below.

W1. PBI versus conventional widening and deepening at comparable budgets

A1. Our new 12-layer ViT comparisons show that PBI achieves higher accuracy than dense widening, MoE widening, and dense deepening at comparable computational budgets. The results are included in Table 6 of the revised manuscript, with training details in Appendix E.

Expansion strategyConfigurationParams (M)GMACTop-1 (%)
Dense widening12 layers, width 5764910.077.9
MoE widening12 layers, width 576, 2 FFN experts8110.078.7
Dense deepening27 layers, width 3844910.376.4
PBI widening12 layers, 2 blocks, 288 private channels each, P=96P=96499.979.3

At approximately 49M parameters and 10 GMAC, PBI exceeds dense widening by 1.4 percentage points and dense deepening by 2.9 points. MoE uses more parameters (81M) at similar computation but underperforms PBI by 0.6 points.

Cross-backbone results. All eight paired backbone comparisons on ImageNet-1K in Table 1 show improvements, with an average gain of 0.54 percentage points. Swin-S improves from 83.0% to 83.4% while the number of parameters decreases from 50M to 48M and computation from 8.7G to 8.5G. VMamba-B improves from 83.9% to 84.2%, also with fewer parameters and less computation. These results show that PBI's gains are consistent across the evaluated backbones and do not necessarily require a larger budget.

W2. Selection of GG and PP across backbones and whether interaction coefficients are learned

A2. We select the block count and public width jointly based on backbone structure, accuracy, and computational budget. Two blocks are a simple and practical choice. The interaction coefficients are fixed.

  • Hyperparameter selection. We choose configurations that balance accuracy and computational cost, keeping PBI's parameters and FLOPs close to those of the original backbone. The model-specific settings, including GG, stage-wise private and public widths, and Priv-to-Pub encoders, are provided in Appendix E.2.
  • Practical choice of block count. Two blocks provide a simple and effective starting point across different backbones. We use two-block configurations across several PBI backbones, including DeiT-S, Swin-S, and VMamba-T/S/B. In a near-equal-parameter Swin-T comparison, two and four blocks achieve 82.30% and 82.56% Top-1, respectively, showing that two blocks perform well, while more blocks can offer additional gains.
  • Interaction coefficients. In Eqs. (5) and (6), α\alpha and β\beta are set to 1 when their respective interactions are enabled and 0 otherwise. Neither coefficient is learned or tuned.
AuthorsOpenReview ↗

Response to Reviewer Dugk (part2)

W3. Whether Fig. 4(a) supports an anti-over-smoothing interpretation of Priv-to-Pub

A3. Priv-to-Pub makes input-dependent information available in the shared public channels from the first layer of each stage. It is not intended to directly minimize inter-block similarity. We appreciate your observation that the orange curve in Fig. 4(a) is below the pink curve at some later layers. In fact, Priv-to-Pub is designed to introduce input-dependent shared information rather than consistently reduce inter-block similarity. To avoid misinterpretation, we have removed the description of Priv-to-Pub as a countermeasure against over-smoothing in the revised manuscript.

With Pub-to-Pub alone, information from private features enters the shared public state after the first layer's block computations and aggregation. Priv-to-Pub instead introduces this information before the first layer. Although the blocks receive the same public state, they retain separate private features and independent parameters. Accordingly, overall CKA alone does not show whether their private representations become more similar.

Public and private CKA. We measured inter-block CKA on 10,000 ImageNet-1K validation images, separately for public, private, and full features. Arrows indicate Priv-to-Pub disabled → enabled. The first setting keeps Pub-to-Pub active and Priv-to-Priv disabled (Appendix E.6, Table A17). The second keeps both interactions active (Table 10).

Other active interactionsMean private CKA (12 layers)Final private CKAFinal public CKAFinal full-feature CKA
Pub-to-Pub0.5229 → 0.44420.6738 → 0.37900.7390 → 0.99940.6753 → 0.9825
Pub-to-Pub and Priv-to-Priv0.5522 → 0.48760.6359 → 0.63650.8257 → 0.99960.6371 → 0.9907

Mean private-feature CKA decreases in both settings, although not at every layer. With all interactions considered, it decreases in 8 of 12 layers and is nearly unchanged at the final layer, while public and full-feature CKA increase. Higher public-feature similarity therefore does not necessarily indicate the high similarity of private representations.

  • Input-dependent shared information. With model weights fixed, replacing each image's Priv-to-Pub output with another image's output in the same batch reduces Top-1 accuracy from 82.44% to approximately 72.5% on ImageNet-1K (Table 11). This shows that predictions depend on public information matched to the current input.
  • Training ablation. With Pub-to-Pub and Priv-to-Priv enabled, adding Priv-to-Pub improves Top-1 accuracy from 82.24% to 82.47% (Table 7), confirming its positive contribution.

Dear Reviewer Dugk,

As the discussion period is nearing its end, we wanted to briefly follow up on our responses to your comments. If you have had a chance to review them, we would appreciate knowing whether our clarifications have addressed your concerns or whether any points remain that we could clarify.

We would be happy to provide any further clarification before the discussion closes.

Thank you for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.