acceptodds
Under review as a conference paper at ICLR 2027

Layer-Wise Input Recoverability in Deep Neural Networks

Abstract

The inner workings of deep neural networks are still not well understood. The information bottleneck framework, which characterizes deep networks by the mu- tual information between their hidden representations and the network’s input, has been used to identify compression of input information with respect to an input image and output labels. We instead study how much of a layer’s own input is recoverable from the layer’s output in convolutional neural networks (CNNs). A convolutional layer applies its filters to one input patch at a time, so we eval- uate recoverability on sampled input patches with a reconstruction-based probe and not on the entire feature map. The probe’s size thus depends on channels and kernel size and not on feature map size, which lets our method scale to deep CNNs with wide feature maps. For each layer, we compare pretrained, random, and reconstruction-optimized encoders, using linear and linear-plus-residual-MLP decoders. We measure recoverability as R2 between input patches and their re- constructions on a held-out set. Across nine ImageNet CNNs, recoverability gen- erally decreases with depth but is more consistently predicted by the compression of the layer. Learning-attributed recoverability, the difference between pretrained and random encoders, is not uniform across architectures. We introduce single- property random controls, which transfer one pretrained property onto a random encoder, and find that each control changes recoverability with a consistent sign across models: copying pretrained singular vectors tends to raise recoverability, while copying the pretrained singular-value spectrum or activation sparsity lowers it.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
patch recoverability, convolutional neural networks, representation analysis, layer-wise probing, reconstruction, information bottleneck, forward-forward learning, biologically plausible learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_QoC3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates how much of a convolutional layer’s own input can be reconstructed from its output. The proposed approach samples local input patches, encodes them using the layer’s transformation, and trains a decoder to reconstruct the patches. Recoverability is measured using held-out R2R^2. The authors compare pretrained, randomly initialized, and reconstruction-optimized encoders, and introduce random controls that match pretrained singular vectors, singular-value spectra, activation sparsity, or BatchNorm parameters. Experiments cover nine ImageNet-pretrained CNN architectures and additional VGG16 and ResNet18 variants trained on CIFAR-10 and CIFAR-100. The reported findings are that recoverability generally decreases with depth, effective compression is a more consistent correlate of recoverability, and the pretrained-minus-random gap varies across architectures. Property-matching controls exhibit more consistent directions of change. The authors explicitly distinguish their decoder-relative reconstruction metric from Shannon information and investigate this distinction in the appendix.

Critical strengths
  1. The local reconstruction target is well motivated and useful. Reconstructing a layer’s own input, rather than the original image from a deep representation, reduces the confounding effects of preceding transformations. Comparing encoders on identical input patches also makes the comparisons more interpretable. The probe size is independent of the feature map’s spatial dimensions, enabling evaluation across a reasonably broad set of CNN architectures.

  2. The paper includes substantial controls and explicitly investigates its metric’s limitations. Beyond pretrained-versus-random comparisons, property-matching controls offer a useful way to investigate the sources of recoverability differences. The appendices examine patch sampling, decoder capacity, and alternative random initialization. They also demonstrate that recoverability and mutual information can rank encoders differently. This explicit acknowledgment makes the scope of the contribution more precise.

Critical weaknesses
  1. Reported effective-compression values conflict with the stated definition and architectural dimensionality. Section 4.2 defines effective compression as k98/Coutk_{98}/C_{\mathrm{out}}, where k98k_{98} is the number of principal components explaining 98% of input-patch variance. For a 1×11\times1 convolution, this necessarily implies $ k_{98}\leq C_{\mathrm{in}}, \frac{k_{98}}{C_{\mathrm{out}}} \leq \frac{C_{\mathrm{in}}}{C_{\mathrm{out}}}. $ However, page 7, lines 326-328, reports effective-compression values of 0.88–0.94 for 1×11\times1 expansion layers in architectures including ResNet50. In standard torchvision ResNet50, the bottleneck expansion convolution has Cout=4CinC_{\mathrm{out}}=4C_{\mathrm{in}}, so the stated definition gives an upper bound of 0.25. This could be a textual error, or it could reflect a discrepancy in the PCA input, denominator, or layer identification. The manuscript does not resolve these possibilities. Since effective compression is a central explanatory variable, its calculation and the associated results need verification before I can accept this conclusion.

  2. The substantially positive held-out R2R^2 under shuffled targets remains unexplained. Appendix A.2 reports mean held-out shuffled-target R2R^2 values of 0.107 and 0.124 for the 4×4\times and 16×16\times decoders, respectively, with some values above 0.2 in Figure 8. If shuffling removes the input–target relationship, training and validation are independent, and R2R^2 uses a coordinate-wise mean predictor as its zero baseline, ordinary training-set memorization does not explain this validation performance. Section 3.3 specifies Var(X)\mathrm{Var}(X) without stating its reduction axes. If variance is computed around a single global scalar mean, differences between coordinate means can allow a constant vector predictor that ignores the representation to achieve positive scores. Furthermore, the residual decoder is initialized from a fitted linear solution, and it is unclear whether this solution is recomputed using shuffled targets for the control. Consequently, the observed behavior cannot yet be interpreted simply as “memorization.” The authors should distinguish normalization, shuffling, and initialization effects. This matters for absolute recoverability and cross-layer comparisons, beyond the appendix diagnostic itself.

Questions for authors
  1. Could you provide the layer name, CinC_{\mathrm{in}}, CoutC_{\mathrm{out}}, k98k_{98}, and PCA input-matrix shape for a representative ResNet50 expansion layer discussed on page 7? How are the reported values of 0.88-0.94 compatible with the dimensionality bound above? If this reflects a calculation or layer-matching issue, do Figure 3 and the main correlation results hold after correction?

  2. What are the exact reduction axes for MSE and variance? What is shuffled, when is it shuffled, and which targets are used for validation? Is the linear initialization refitted on shuffled training targets? For representative layers with high shuffled-target scores, could you report a constant mean-vector baseline and results using $ R^2 = 1-\frac{\sum_i|x_i-\hat{x}_i|^2} {\sum_i|x_i-\bar{x}|^2}, $ and establish whether the principal comparisons remain unchanged?

  3. The introduction does not establish the significance of the chosen measurement sharply enough. The introduction moves from biological information bottlenecks to mutual information, representation inversion, and probing, but provides a less developed explanation of why recovering a layer’s own input patch is scientifically useful. The paper should identify the precise question this local measurement can answer and explain what new understanding it provides beyond dimensionality reduction and reconstruction error. The biological motivation could be shortened to make room for this argument.

  4. The method description is insufficiently explicit about how the probe corresponds to different architectures. Section 3.2 describes a fully connected encoder optionally followed by BatchNorm and an activation, and asserts equivalence to the convolutional layer’s output. However, the evaluation includes depthwise convolutions, residual architectures, and networks with different normalization and activation placement. A compact implementation table should specify the input and output endpoints, handling of grouped/depthwise connectivity, BatchNorm mode and statistics, and whether residual addition is included. These details are necessary to assess the claimed equivalence.

  5. The depth plots omit the coordinates needed to identify the reported effects. Figures 2, 5, 13, 14, and 15 show progression over depth without numerical layer ticks or sufficiently identifiable stage labels. Readers can see trends but cannot reliably associate spikes, drops, or transitions with particular layers. The figures should include layer indices or stage boundaries and identify important exceptions discussed in the text. For architectures with parallel branches, the authors should also explain how the displayed ordering is constructed: connecting every point with a line can otherwise suggest a sequential processing path that the architecture does not have.

Reviewer_qFobOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript explores "layer-wise input recoverability" for a broad range of CNNs and specializes the idea to be per layer on its input patches. Numerous experiments are conducted to assess the behavior of input recoverability and some exploration is done to try and understand what properties of pre-trained weights matter.

Critical strengths

The manuscript contains a fairly comprehensive set of empirical experiments and the "per layer" and for "input patches" ideas are sensible to enable analysis of larger models.

Critical weaknesses

The manuscripts major weakness is that despite the relatively robust slate of experiments, there is no cohesive narrative or clear insights built out of them. Bluntly, the manuscript reads like a lab notebook rather than a paper that builds off the data in the notebook. There is no clear hypothesis and, consequently, no clear narrative around the experiments. (I should carefully note here that a narrative need not mean that all results are "aligned" around a singular message, but rather that interesting things can be learned from the results. It need not be an unequivocally supported message, but there should be a message.)

There are, to continue the lab notebook analogy, some methodological developments. However, I do not feel that they are strong enough on their own to carry the manuscript. As written, the come across more as what can be done for large models rather than having particular effectiveness at addressing interesting questions beyond the broad connections to information bottlenecks and probing.

Questions for authors
  • I don't think W is properly defined before it appears on line 240. I presume this is because the encoder structure is not well specified. More generally, this makes it somewhat tricky to precisely understand some of the single property controls.
Reviewer_EagvOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

Proposes layer-wise input recoverability: linear, or linear plus residual MLP reconstructing a conv layer’s input patches from its output. Patch-level probing scales to deep CNNs. Compares pretrained, random and end-to-end encoders across nine ImageNet CNNs plus CIFAR variants. Introduces learning-attributed recoverability (pretrained minus random) and single-property random controls.

Critical strengths

Well written, clearly scoped, honest about limitations. Careful execution: patch-budget ablations, decoder-capacity and memorization checks, multiple seeds. Patch-level probing is a neat, cheap way to analyze individual layers. Explicit about recoverability being decoder-relative, not Shannon information. Synthetic checks against exact information. Single-property controls a clever idea, worth reusing.

Critical weaknesses

Main results largely predictable from the definition. Recoverability under a linear decoder similar to effective rank vs width. Control effects follow from first principles. Predictions not stated up front. Learning-attributed gaps small, inconsistent across models, often within decoder sensitivity. Task dependence comes from earlier layers, not the probed one. Undercuts the local framing. Unclear what we learn about trained networks beyond effective compression. Related work on invertibility and injectivity of ReLU/conv layers missing. Information-bottleneck motivation loosely tied to what’s measured.

Questions for authors

How do results compare with an analytic PCA or rank bound? What would be surprising under this measure, and did anything turn out that way? Can the objective be varied with data fixed (coarse vs fine labels, random labels, self-supervised)? How does recoverability evolve during training, and does it track neural collapse?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.