- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates spatial continuation for training CNNs (primarily ResNet-20 and VGG-11) on image classification tasks. Specifically, the method temporarily reduces the internal spatial resolution of feature maps via adaptive max-pooling inserted after an early residual stage, and progressively restores the native resolution before training concludes. Because the model returns to its standard resolution and architecture, inference cost and parameter count are completely unchanged.
The authors evaluate this intervention across various unaugmented experimental settings covering multiple datasets (CIFAR-10, SVHN, STL-10), architectures, optimizers (SGD, Adam, AdamW, RAdam), activations, normalizations, and budgets. In these settings without data augmentation, the method yields consistent mean test accuracy improvements ranging from 0.38 to 6.28 percentage points. The paper also introduces comparison points with Gaussian feature-map smoothing (G) and combined resolution/blur interventions (RG), and compares against prior methods including Curriculum by Smoothing (CBS) and SDPoint. Extensive controls analyze the necessity of coarse-to-fine ordering, single-resolution reductions, hyperparameter selection via held-out validation splits, training-loop wall-clock speedups, and loss landscape geometry.
Critical strengths
-
The authors clearly demonstrate and emphasize where the benefits diminish or vanish—specifically under standard strong data augmentations (random crops and horizontal flips) and in longer training regimes.
-
The authors investigate whether the coarse-to-fine schedule is essential by reversing or mixing resolution stages and by testing a single reduced resolution. These experiments provide evidence that temporary resolution reduction, rather than the particular progression, accounts for much of the improvement. Evaluation of normalization handling, showing how BatchNorm calibration conventions (saved statistics, clean recalibration, loader-matched momentum) alter baseline rankings, particularly when benchmarking against SDPoint and CBS.
-
The paper evaluates Hessian trace and leading eigenvalues via Lanczos iterations and filter-normalized random-direction sensitivities. The authors do not claim that lower loss curvature is a universal explanation for the gains, noting that reduced curvature in the reference setting does not hold as a universal mechanistic explanation across all optimizers (e.g., AdamW) or dataset splits.
-
The method is straightforward to implement and does not add trainable parameters or permanent inference-time computation. This makes it potentially useful in resource-constrained training settings.
Critical weaknesses
Major Weaknesses
-
Limited methodological novelty: The central operation is temporary adaptive max-pooling within a CNN, followed by restoration of native resolution. Although the authors distinguish their approach from SDPoint, CBS, and progressive resizing, the conceptual novelty over existing internal downsampling and curriculum-based methods is not sufficiently strong. The paper would benefit from a clearer account of what fundamental insight the proposed method contributes beyond an empirically effective variant of temporary downsampling.
-
The primary limitation is that modern deep learning pipelines for computer vision almost universally rely on data augmentation (e.g., random crops, flips etc) and extended training horizons. When basic crop-and-flip augmentation is added, the paired gain of the method over Plain drops drastically (e.g., from +3.25 down to +0.19 on CIFAR-10 at 60 epochs, and showing negligible or negative differences at 160 epochs depending on the BN calibration policy). While the paper notes that unaugmented regimes matter when domain-appropriate transformations are unavailable, the practical scope of the method remains largely restricted to data-scarce or unaugmented domains.
-
The empirical evaluation is restricted to small-scale convolutional models (ResNet-20 and VGG-11-BN) trained on relatively small image datasets (CIFAR-10, SVHN, STL-10). The paper contains no experiments on ImageNet-1k, modern ConvNets (e.g., ConvNeXt), or Vision Transformers (where patch/sequence reduction are present). This makes it difficult to ascertain whether internal spatial continuation scales to realistic computer vision pipelines.
-
While the paper does an good job showing that several hypotheses fail to explain the effect (the coarse-to-fine schedule order does not matter; Gaussian smoothing alone does not explain it; loss curvature reduction fails to track performance across optimizers), it does not provide much theoretical explaination of why temporary spatial downsampling acts as a regularizer.
Minor Weaknesses
-
While the authors transparently acknowledge using CIFAR-10 test set feedback to guide initial choices for reduction sites, schedules, and learning rates (Section 6, Appendix B), this iterative reuse introduces potential selection bias. Although the validation split experiment in Appendix G.3 partially mitigates this concern, its hyperparameter search remains limited, as both Plain and R selected the boundary learning rate of 0.04 without further expansion of the grid.
-
The sensitivity of relative performance rankings to the BatchNorm recalibration convention (P1 vs. P3) under augmented settings is significant (e.g., shifting differences between R and SDPoint by over 1 percentage point). While this is analyzed thoroughly, it indicates that empirical conclusions in these regimes are fragile.
-
The paper could more clearly specify when users should use temporary resolution reduction, when they should prefer standard training, and how to choose the reduction ratio and schedule without consulting test-set performance.
Questions for authors
-
Could you provide preliminary empirical evidence or analysis showing whether internal spatial continuation transfers to modern vision architectures—such as ConvNeXt or Vision Transformers (e.g., through token merging or downsampling)—beyond the smaller models (ResNet-20 and VGG-11-BN) evaluated in the paper?
-
Do the consistent improvements reported across CIFAR-10, SVHN, and STL-10 scale to more complex, large-scale benchmarks such as ImageNet-1k, where networks feature significantly deeper hierarchical representations?
-
Given the limited benefits observed under data augmentation, what specific applications or data domains are expected to benefit most from this intervention?
3 comments
Response to Reviewer xzV9 — 2/3: Preliminary ConvNeXt experiment
2. Preliminary ConvNeXt experiment.
We have additionally tested a ConvNeXt variant without BatchNorm, which helps probe whether the intervention can be applied beyond the original residual/VGG configurations and without relying on BatchNorm's running-statistics behavior.
Protocol. ConvNeXt-Femto (depths 2/2/6/2, widths 48/96/192/384, depthwise 7 x 7 convolutions, LayerNorm, 2 x 2 patchify stem) on CIFAR-10 with the same 45,000 / 5,000 / 10,000 split as above, no augmentation; 30 epochs, effective batch 128, AdamW with weight decay 0.05, 60 warmup updates then cosine to zero. R applies adaptive max pooling at the first stage, to 8 then 12 then the native 16, at epochs 0 / 6 / 12. The learning-rate grid 2.5e-4-0.032 is shared by all methods, extended twice by the frozen edge rule so that every selected rate is interior; selection on validation seeds 10-12, confirmation on test seeds 13-15, paired by seed. This network has no BatchNorm, so the three evaluation policies coincide.
| Method | Validation-selected LR | Test accuracy (mean ± SD) |
|---|---|---|
| Plain ConvNeXt | 0.016 (epoch 29) | 68.65 ± 3.49 |
| ConvNeXt + R | 0.008 (epoch 23) | 76.62 ± 0.51 |
| ConvNeXt + SDPoint | 0.008 (epoch 29) | 78.78 ± 0.35 |
The paired mean difference is +7.97 ± 3.33 percentage points, positive on 3 of 3 seeds. However, the Plain runs exhibit substantial variability (SD 3.49 against 0.51 for R; per-seed differences +9.80, +4.12, +9.98), so this numerical difference is not yet a stable estimate of the benefit. The selection rule sends Plain to 0.016, which is its best validation rate (72.55% against 67.93% at 0.008) but also the rate at which it is most seed-dependent, while R and SDPoint both select 0.008 and are stable there; SDPoint shows the same profile against Plain (+10.13 ± 3.83 points, 3 of 3). We include it as an exploratory compatibility result rather than a robust performance claim. It motivates the next scaling tests, but the stronger, presently interpretable transfer evidence is the ResNet-18/Imagenette experiment above. We have not tested Vision Transformers, for which changing the token sequence would introduce different design choices.
Response to Reviewer xzV9 — 3/3: Application domains and scientific contribution
3. Application domains and concrete guidance.
The submitted paper already shows that the benefit is strongest in no-augmentation settings, with substantial gains across twenty targeted combinations of datasets, architectures, optimizers, normalizations, and budgets. It also explicitly reports how conventional augmentation can reduce the extra gain. We view this as a useful characterization of the method's operating regime rather than a tension with the main conclusion.
An additional test on OrganAMNIST makes the application case more concrete. Its axial CT images include left- and right-specific class labels, so horizontal flipping is not automatically label-preserving. Under the validation-selected comparison described in our response to Reviewer pGXo, R improves on Plain by +0.61 ± 0.30 percentage points, positive on 3 of 3 seeds and under every evaluation policy, and has a higher mean accuracy than SDPoint (+0.68 ± 0.18 points, 3 of 3 under saved statistics; +0.15 ± 0.25 after recalibration). This does not exclude other medical-image augmentations; it illustrates why label-aware augmentation can require domain-specific decisions. A training-only internal intervention offers a complementary option without changing the input-label pairing.
For practitioners, we propose the following evidence-based guidance.
- First consider R when training without augmentation or when appropriate augmentations are limited. This is where the original evidence is strongest, and the new OrganAMNIST and unaugmented Imagenette experiments broaden that case.
- Expect smaller incremental gains when a strong augmented recipe is already effective. The original paper makes this dependence explicit, and the new augmented Imagenette result is in the same direction. R is a candidate to validate, not a guaranteed improvement.
- A complex multi-stage schedule is not essential to test the underlying idea. Appendix G.2 shows that a single, moderately reduced side (24 rather than 32 in the reference ResNet-20 setting) retains most of R's gain, while stronger reduction need not help. The original schedule/site were selected in the CIFAR-10 exploration and carried into transfer settings with scale/budget adaptations; new applications should select adaptations on validation data, not test accuracy.
- Inference remains exactly the original network. R introduces no extra inference-time operation or trainable parameters. Training time is recipe-dependent; the original timing results find it comparable in the short reference experiment and lower in the controlled long-SGD timing study, without claiming universal speedups.
4. The broader scientific contribution, and how SDPoint fits into it.
We agree with the reviewer's underlying request to identify an insight that transcends the particular implementation. Importantly, the paper already makes this distinction. The original motivation is spatial continuation, but our matched-exposure and single-resolution experiments demonstrate that temporary internal resolution reduction matters more than the monotone coarse-to-fine progression. Our result is therefore not simply that one hand-selected schedule works.
SDPoint is valuable in this interpretation. It reduces internal resolution through a different procedure—random, per-update average pooling at varying positions—whereas R uses deterministic, fixed-site max pooling. Yet both can improve the final model when evaluated at full resolution. Our additional stopped-SDPoint control (Appendix E.4) also shows that ending its random reductions when R returns to native resolution changes mean test accuracy by at most 0.16 points under the primary P1 BatchNorm evaluation policy in the three tested recipes. This is not a proof that all implementations are equivalent, but it supports studying training-time exposure to reduced internal spatial resolution as a shared empirical factor rather than attributing the effect to a single ordering rule.
Our contribution is precisely to analyze that factor. We separate reduction from progression, compare it with smoothing, document its dependence on augmentation and optimizer, and investigate geometric hypotheses without treating them as established explanations. The evidence supports an empirical generalization phenomenon, while leaving a complete mechanistic theory open. The new results broaden the architectural and domain scope of that account, rather than changing its original thesis.
We will include these additional experiments and make the already-present intervention-level conclusion more prominent in the revision, reporting every selection grid in full. We appreciate the reviewer identifying the most valuable extensions to test.
We would be happy to clarify any remaining questions during the discussion period.
Response to Reviewer xzV9 — 1/3: Imagenette / ResNet-18 transfer
We thank the reviewer for the thoughtful evaluation, including the recognition of our schedule ablations, BatchNorm controls, and careful separation of geometric observations from mechanistic explanations. We have run additional experiments aimed at the requested extensions to higher-resolution images, deeper convolutional models, and a modern ConvNet, and we can now be more specific about potential application domains.
1. Transfer to 128 × 128 images with ResNet-18.
To test whether the effect is restricted to the original 32 × 32 benchmarks and small ResNet-20 models, we evaluated R on Imagenette, a ten-class subset of ImageNet, with 128 × 128 inputs and ResNet-18. We considered both an unaugmented recipe and a recipe with input augmentation.
Protocol. Official imagenette2 320-px release, centre square crop and bilinear resize to 128, splits 8,469 / 1,000 (drawn from train) / 3,925 (official validation used as test); ResNet-18-BN; 30 epochs, effective batch 128, SGD (momentum 0.9, weight decay 5e-4), 60 warmup updates then cosine to zero. R pools at the first stage, to 16 then 24 then the native 32, at epochs 0 / 6 / 12, with the native network from epoch 13 on; the pooling is absent at inference. The augmented recipe adds zero padding 16, a uniform 128 crop and a horizontal flip with probability 1/2. The learning-rate grid 0.02-1.28 is shared by all methods and every selected rate is interior to it; selection on validation seeds 10-12, confirmation on test seeds 13-15, paired by seed, under saved BatchNorm statistics at the selected epoch. SDPoint is included under the same grid and rule. The protocol was frozen before any run was launched, and test accuracy was never consulted during selection.
Imagenette test accuracy (%)
The improvement of +3.79 ± 0.28 percentage points without augmentation, positive on 3 of 3 seeds, is encouraging evidence that the effect persists with a deeper network and substantially larger images. With augmentation, the mean advantage is smaller (+0.48 ± 0.78 points, positive on 2 of 3 seeds and therefore not resolved at this seed count; +1.03 ± 0.89, 3 of 3, after clean BatchNorm recalibration), consistent with the trend already documented in the submitted CIFAR-10 experiments. The takeaway is not that the same numerical gain holds universally; rather, the original relationship between augmentation strength and the additional value of resolution reduction also appears in this more demanding setting.
The augmented recipe also shows the clearest qualitative difference between the two interventions. SDPoint, selected under the same grid, reduces accuracy relative to Plain there (SDPoint − Plain = −1.41 ± 0.77 points, negative on 3 of 3 seeds), whereas R does not (R − SDPoint = +1.89 ± 1.07, positive on 3 of 3). Without augmentation the ordering is reversed (R − SDPoint = −1.49 ± 0.65, 0 of 3). We report both directions rather than either alone.
Imagenette is not ImageNet-1k, and we do not present it as such. It nonetheless directly addresses an intermediate, concrete scaling question. The phenomenon is not confined to ResNet-20 or 32 × 32 inputs.