acceptodds
Under review as a conference paper at ICLR 2027

Sensitivity Before Connectivity: Minimal Pairs Reveal How CNNs Learn a Global Visual Rule

Abstract

A network can see every pixel a visual rule depends on and still not use them. We study this gap with seeded connectivity on near-critical grids, where a full-access convolutional network faces zero Bayes risk. Path-conditioned AUC suggests that competence spreads from short to long paths during learning, but this picture is misleading. We introduce minimal pairs for spatial rules: two images that differ in a single pixel outside the query's local window, where closing that pixel disconnects the seed from the query. Size-matched placebo edits, which remove at least as much of the query's component without disconnecting it, separate connectivity from cluster size. Minimal pairs reveal three stages. With k training examples, the model's discrimination does not exceed what the seed position alone provides, and it ignores cuts. With k, it responds to distant cuts at every path length, but mostly in proportion to how many pixels an edit removes. With M, it computes connectivity itself, including on paths that need more certified propagation steps than it has layers. Within individual training runs, the response to cuts appears at all path lengths at once. At first, it depends about equally on how many pixels an edit removes and on whether it disconnects. The weight on disconnection then grows until removal size no longer matters. Path length orders the strength of the response rather than its presence, and response strength is what path-conditioned AUC detects.

Primary area:
learning theory
Keywords:
visual reasoning, connectivity, convolutional neural networks, minimal pairs, counterfactual evaluation, algorithmic reasoning, long-range dependencies, mechanistic analysis, sample complexity

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2RKROpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how convolutional neural networks learn global visual connectivity rules, with particular attention to the distinction between having access to relevant information and actually utilizing that information.

The authors introduce a minimal-pair evaluation framework in which single-pixel interventions outside a restricted observation window alter the connectivity between a seed and a query point. Size-matched placebo interventions are further employed to distinguish genuine connectivity-related responses from sensitivity to changes in connected-component size.

The experiments identify a three-stage progression in the studied LocalCNN: initial reliance on seed-position information, followed by sensitivity to distant structural changes that is largely associated with component removal size, and eventually connectivity-specific discrimination.

The authors also argue that path-conditioned AUC can produce a misleading impression of competence expanding from shorter to longer paths, whereas minimal-pair analyses reveal that nonlocal sensitivity can emerge across path lengths before becoming sufficiently strong to affect aggregate discrimination metrics.

The study includes controlled interventions, multiple random initializations, within-run training trajectories, regression analyses, and supplementary comparisons involving alternative neural architectures and input distributions.

Critical strengths

1. Clear research motivation and a carefully constructed evaluation methodology

The paper addresses a meaningful distinction between information accessibility and actual information utilization. The minimal-pair construction, together with restricted-access controls and size-matched placebo interventions, provides a relatively rigorous experimental framework for separating nonlocal sensitivity from a particular class of shortcut-based responses.

2. Coherent experimental analysis of learning dynamics

The experimental workflow is reasonably comprehensive within the primary setting. The authors combine multiple random initializations, checkpoint-level analyses, regression-based decomposition, and propagation-depth comparisons to support their observations.

The discrepancy between path-conditioned AUC and minimal-pair accuracy is particularly informative: the results provide useful evidence that weak but systematic responses can remain obscured by aggregate performance measurements.

Critical weaknesses

1. Limited methodological novelty and an insufficiently established general contribution

The proposed evaluation framework is carefully implemented, but its conceptual contribution reads as relatively narrow given the surrounding literature. Minimal-pair evaluation, controlled counterfactual interventions, and contrastive examples for separating target behavior from shortcut-based predictions have each been explored in prior work the paper itself acknowledges — including contrast sets, counterfactual data augmentation, and earlier studies of neural connectivity and algorithmic reasoning. The contribution is therefore best characterized as the particular combination of single-pixel connectivity interventions, restricted observation windows, and component-size-matched placebo controls on a specific spatial reasoning task. A useful experimental design, but one that does not clearly rise to a general new methodology or theoretical account.

A second interpretive gap concerns the inference from sensitivity to competence. Detecting a consistent directional response to an intervention is not necessarily equivalent to demonstrating reliable competence on the underlying task, and the discrepancy between minimal-pair accuracy and path-conditioned AUC is informative precisely because the two metrics measure different properties rather than providing competing estimates of the same capability.

Consequently, the methodological implications seem more limited than the broader framing suggests.


2. Insufficient experimental breadth and limited generality of the central learning-dynamics findings

The three-stage progression is established primarily on LocalCNN under a fixed 64 × 64 near-critical Bernoulli connectivity setup. Although U-Net, dilated CNN, recurrent CNN, Transformer, and MPNN variants are included, the principal minimal-pair analyses and detailed checkpoint-level trajectories are not systematically reapplied across these architectures. This is a material limitation: U-Net reaches low error with substantially fewer training examples, while recurrent models exhibit different dependencies on computational steps, so it remains unclear whether the reported progression is a general phenomenon or a LocalCNN-specific training-dynamics artifact.

Three concrete empirical caveats further narrow the generality claim:

  • Specificity gap at 256k: pair accuracy of 0.759 coexists with branch specificity of only 0.560, indicating that much of the early response is not connectivity-specific; the later improvement at 1M is encouraging but does not on its own rule out alternative structural proxies.
  • Selection effects in the branch-placebo analysis: the procedure operates on a qualifying subset of constructed pairs, and the resulting subset composition is not fully quantified.
  • Trajectory and budget confounds: the within-run progression is demonstrated with only two training trajectories, and cross-pool-size comparisons involve different realized optimization budgets rather than controlled ablations.

I consider the empirical observations credible within the investigated configuration, but the experimental evidence does not yet establish that the identified progression is a broadly applicable characteristic of neural networks learning global visual rules.

Questions for authors

1. Methodological contribution and positioning

Could the authors clarify what they consider the primary methodological advance beyond existing minimal-pair, contrast-set, and counterfactual evaluation approaches?

In particular, is the intended contribution a reusable general evaluation framework, or a carefully controlled empirical diagnosis of one specific global connectivity task? The paper currently sits between these two readings, and making the positioning explicit would directly determine how reviewers should weigh the significance and generality claims.

Clarifying this distinction up front would also help subsequent work decide whether to adopt the protocol as-is or treat the current results as a case study.


2. Generality across architectures

Could the authors provide evidence that the sensitivity-before-connectivity progression replicates in at least one additional successful architecture — e.g. U-Net, dilated CNN, or recurrent CNN?

These variants are already listed in the experiments, but the decisive metrics (pair accuracy and branch specificity tracked across training checkpoints) appear to be reported only for the main LocalCNN configuration. Extending the same intervention-based trajectory analysis to one architecturally distinct baseline would substantially narrow the alternative explanation that the three-stage pattern is a LocalCNN-specific optimization artifact.

A minimal viable extension would be: same checkpoint grid + same qualifying-pair selection pipeline, even on a single held-out architecture.


3. Robustness of connectivity-specific discrimination

How stable are the branch-specificity conclusions under:

  • alternative placebo constructions (beyond matching connected-component removal size), and
  • additional controls for confounding topological proxies such as bottleneck geometry and path redundancy?

The 256k checkpoint already signals a non-trivial gap (pair accuracy 0.759 vs. branch specificity 0.560), and the branch-placebo subset is necessarily filtered. It would be useful to know whether (i) varying the candidate-selection procedure changes the magnitude of that gap, and (ii) adding a bottleneck- or redundancy-matched placebo condition shifts the later-stage specificity numbers.

This is the key test for whether the model is tracking connectivity itself versus a correlated structural feature of the generated graphs.

Reviewer_3njXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper examines whether CNNs learn graph connectivity when all relevant visual information is available. It studies four types of datasets, mainly focusing on synthetic binary grids. The paper introduces minimal pairs, where changing a single pixel breaks connectivity between the seed and query. It also uses placebo edits that remove at least as many pixels without breaking connectivity. This helps distinguish whether the model learns actual connectivity or simply relies on component size. Three learning stages are identified: dependence on seed position, sensitivity to component size, and learning connectivity itself. These stages are mainly demonstrated using a LocalCNN without spatial pooling or downsampling. Other architectures are also examined, with U-Nets generally learning the task with fewer training examples.

Critical strengths

The paper introduces two clean methodological controls.

  1. The minimal-pair evaluation separates genuine connectivity sensitivity from simpler cues such as component size.
  2. The experiments try to mitigate shortcomings of AUC measurements.
Critical weaknesses
  1. The motivation needs further clarification. Having access to all relevant pixels does not guarantee that a CNN will learn the intended connectivity rule. Learning also depends on the architecture, training distribution, and optimization. There is a gap between information that is available for learning and the ultimate learning dynamics.

  2. The three learning stages are mainly demonstrated using a LocalCNN without spatial pooling or downsampling. However, U-Nets achieve substantially better performance with fewer training examples.

  3. The main findings are established on synthetic 64×64 binary grids. Although other distributions are examined, the experiments do not establish whether the same learning behavior occurs in realistic visual tasks. This shows a gap in the study performed and real world datasets.

Questions for authors
  1. Could the authors clarify this distinction mentioned in weakness 1 and explain how it affects the motivation and conclusions of the paper?

  2. Could the authors apply the same minimal-pair and size-matched placebo evaluations to successful U-Net runs at different training checkpoints? Specifically, does the U-Net exhibit the same three-stage progression, including the intermediate dependence on component size, or does it learn connectivity differently? This will be an important experiment in my opinion.

  3. The broader applicability remains unclear. Could the authors evaluate the proposed methodology on a more realistic visual connectivity task, such as road-network extraction? Given a satellite image, a model identifies roads and determines how they connect to form a road network. A small gap in a predicted road could incorrectly disconnect two neighborhoods.

Reviewer_FsDWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper tries to explain how convolutional neural networks acquire the ability to recognise global connectivity in binary images, distinguishing between having access to relevant information and actually using it. The authors study a seeded-connectivity task on near-critical 64×6464 \times 64 grids, where the objective is to determine whether a border seed and a central query pixel belong to the same four-connected component. The main contribution being a minimal-pair evaluation framework designed to distinguish nonlocal sensitivity from genuine connectivity-specific behaviour. The proposed evaluation uses single-pixel cuts that disconnect the seed from the query, together with size-matched branch-placebo edits that preserve connectivity.

Experiments report three stages: at 64k training examples the model uses seed position only and ignores cuts; at 256k it responds to distant cuts at all path lengths but mostly in proportion to the number of pixels removed (PA≈0.76\mathrm{PA}\approx0.76, B≈0.56B\approx0.56); at 1M it prefers the connected placebo in about 99.6% of size-matched pairs, including on examples needing more certified propagation steps than the network has layers. Two checkpointed 1M runs show a gradual shift from removal-size sensitivity to disconnection sensitivity, and the authors argue that path-conditioned AUC reflects response strength rather than coverage.

Critical strengths

1. Minimal-pair evaluation framework

The proposed counterfactual evaluation framework goes beyond conventional classification accuracy and path-conditioned AUC. By constructing image pairs that differ by a single pixel outside the local observation window, the authors can directly test whether a model responds to nonlocal information. The size-matched branch-placebo edits is also valuable as it helps distinguish sensitivity to component removal size from sensitivity to actual disconnection.

2. Transparent reporting and a useful methodological lesson about difficulty-conditioned metrics

The paper reports per-initialization ranges and bootstrap intervals, includes a seed-only reference scorer, and states openly that pool size and optimization budget are entangled. It shows concretely that AUC can miss a real response: at 256k the longest path bin has ΔAUC=−0.011\Delta\mathrm{AUC}=-0.011 over the local model, yet pair accuracy there is 0.830.83. It also shows that, at fixed removal size, short paths beat long paths, so the high long-path pair accuracy at 256k is a removal-size effect. The certified-propagation analysis is a careful way to say which update rules are excluded, and the authors state that it does not identify the mechanism.

Critical weaknesses

1. The claim that the network "computes connectivity" goes beyond what the pair tests establish (Sections 4.3, 4.6).

(a) Cut and placebo edits are selected by different procedures. The cut is the shortest-path pixel nearest the query outside the window, so it is an articulation pixel on the shortest path and lies systematically close to the window boundary. The placebo is an off-path pixel of the query's component, drawn from at most 16 random candidates and chosen to leave the smallest remaining component, and it exists for only 5,8635{,}863 of 9,5559{,}555 pairs.

(b) Edit distance from the query, local neck or bottleneck geometry, path membership, and seed-side component size are not matched. The authors acknowledge that the placebo rules out a component-size proxy "but not every conceivable shortcut", and their matched-pair control explicitly does not test the percolation-proxy explanation. Branch specificity near 11 therefore shows that component size is not the whole story, not that the model implements seed–query connectivity.

(c) Pair accuracy is a relative comparison, not correct classification. The paper reports PA≈0.99\mathrm{PA}\approx0.99 for dprop∈[64,80)d_{\mathrm{prop}}\in[64,80), but Appendix E and Figure 9 report positive error of 0.210.21–0.310.31 for the strong 1M runs at dprop>64d_{\mathrm{prop}}>64, and one of five 1M runs has overall error 0.1120.112. A model can rank the intact image above the cut while still placing both on the same side of the decision threshold.

(d) The evidence for the "beyond a flood-fill budget" claim is thin where it matters most. For dprop≥96d_{\mathrm{prop}}\ge96 branch specificity rests on 3636 pairs (initialization range 0.7780.778–1.0001.000), and the authors concede that channels could encode connectivity summaries. The contribution list's word "verified" is stronger than this supports.

2. The three-stage learning account is confounded by training budget, thinly replicated, and partly at odds with its own continuity finding (Sections 4.1, 4.4, 4.5, Table 11).

(a) Pool size and update count are entangled. Selected checkpoints come from about 1.31.3k, 8.88.8–1010k and 4343–4848k updates at 64k, 256k and 1M. The paper's own numbers suggest a time explanation: the 256k stage (PA≈0.76\mathrm{PA}\approx0.76, B≈0.56B\approx0.56) resembles the 1M trajectory at roughly 6–7k updates (PA≈0.73\mathrm{PA}\approx0.73–0.800.80, B≈0.55B\approx0.55–0.570.57, Table 6). The cross-pool comparison therefore cannot separate sample-complexity effects from optimization time.

(b) The within-run evidence is two trajectories of one architecture, trained on the same data stream, on one distribution and one grid size. The authors' regression shows a gradual shift in the weight on disconnection, which is better described as a continuum than as discrete stages. The reported share βconnβconn+βsizelog⁡153\frac{\beta_{\mathrm{conn}}}{\beta_{\mathrm{conn}}+\beta_{\mathrm{size}}\log 153} depends on assuming the logit drop is linear in log⁡(1+pixels removed)\log(1+\text{pixels removed}) and on evaluating it at the median cut, and it is undefined once βsize≤0\beta_{\mathrm{size}}\le0.

(c) The headline that sensitivity "appears at all path lengths together" is measured with pair accuracy, which at 256k is dominated by removal size. The paper declines to claim a path-length order for branch specificity (the final bin has 1919 pairs). It is therefore not established that the connectivity-specific response appears at all path lengths together. Given this, the argument that path-conditioned AUC "misleads" is weaker than presented: at 256k the long-path pair accuracy is a removal-size effect, so AUC's "no advantage" there is consistent with no connectivity-specific computation.

(d) The account is established only for the LocalCNN. The title and abstract speak of how "CNNs" learn the rule, but the U-Net, which reaches low error at 256k in 7 of 10 seeds, and the recurrent models were not scored with minimal pairs, and the dilated CNN was scored only for pair accuracy.

Questions for authors

Q1. Can the authors add placebos matched to the cut on distance from the query, path membership and bridge structure, for example an on-path pixel whose closure does not disconnect (a detour exists), and report a proxy based on the size of the seed's component?

Q2. Can the authors train at different pool sizes (for example 64k, 256k, 1M) for the same fixed number of updates, with early stopping disabled, and report pair accuracy and branch specificity versus updates for each pool size?

Q3. Does the within-run progression (onset at all path lengths, gradual rise in the disconnection share) hold across more than two trajectories, different data streams, other occupancy probabilities or grid sizes, and other architectures such as the dilated CNN or U-Net?

Q4. How sensitive is the reported disconnection share to the functional form of the size term and to the choice of the median cut? Can it be reported within bins of removal size?

Q5. Can the authors generate enough additional pairs to estimate branch specificity per path bin during early training (5–10k updates), so that the claim "appears at all path lengths together" is tested for the connectivity-specific component and not only for pair accuracy?

Comments on the paper

1 comment

Dear reviewers,

The discussion phase is nearing its end. If you haven't already, could you please read the author's rebuttal and respond?

Thank you & kind regards,

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.