acceptodds
Under review as a conference paper at ICLR 2027

Representational Frustration: Local Incompatibility in Neural Networks

Abstract

Examples that are nearby in a ReLU network's hidden-state space can nevertheless activate different sets of units. We call this local mismatch representational frustration and ask how it is organized across geometric scales and shaped by learning. We measure activation-support dissimilarity among pairs selected at increasing distance thresholds in hidden-state space. Since frustration has no intrinsic quality ordering, we investigate it through a temporary learning-rate increase, motivated by evidence that large steps can reorganize learning dynamics. We then restore the baseline and compare the resulting trajectory with a matched control. In CIFAR-10 MLPs, support incompatibility among control-nearby pairs remains displaced through the observed training horizon, after the learning rate returns to baseline and the large accuracy disruption subsides. Comparing local changes with the all-pair response reveals a reorganization across geometric scales. Under cross-entropy, the closest pairs initially change less than the global pair population but retain excess incompatibility afterward; under mean-squared error, positive local excess is already present at the measured pulse checkpoints. The retained local excess also depends on training conditions: it is negative in the full-batch runs, and different hidden layers can retain opposite signs. These observations show a persistent, scale-dependent organization of representational frustration that predictive accuracy alone does not describe. What these persistent differences retain from learning, and whether they play a role in what the network remembers, remain open questions.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
representational frustration, representation geometry, activation support, training dynamics, optimization path dependence

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vKUwOpenReview ↗
Rating
1
Confidence
2(−1)
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces “representational frustration,” measured as Jaccard dissimilarity between ReLU activation supports among pairs selected by hidden-state distance. It studies how this statistic responds to a temporary learning-rate increase, using control-selected pairs and subtracting the all-pair response to characterize local excess. Experiments primarily use shallow, width-512 MLPs on 10,000 CIFAR-10 training examples, with mostly two seeds per condition. The reported differences persist through epoch 200 and vary across objectives, optimizer settings, pulse timings, and layers. These results establish an exploratory descriptive phenomenon.

Critical strengths
  1. Selecting pairs using the control geometry avoids conflating a change in pair membership with a change in support dissimilarity in the same comparison. This is a useful methodological choice for interpreting the reported differences.

  2. Opposite retained local-excess signs in the two hidden layers recur in both reported seeds. This supports studying the phenomenon beyond a single aggregate statistic.

Critical weaknesses
  1. Strict positivity gives arbitrarily small activations the same status as large, consequential activations. Indeed, the example in Section 2 demonstrates that substantial “frustration” can arise from vanishingly small activation differences. This exposes a central ambiguity: the statistic may largely reflect threshold crossings with negligible computational consequences. The paper provides no threshold-sensitivity analysis, magnitude-aware comparison, occupancy-matched null model, or functional intervention that resolves this ambiguity. Subtracting the all-pair response does not control for support occupancy, as Appendix B acknowledges. The claimed added perspective also needs stronger justification. At epoch 200, test cross-entropy already distinguishes the reference runs substantially: 2.716 versus 2.312. Thus, recovery of accuracy does not establish that the proposed statistic reveals a consequential difference missed by standard behavioral measurements. The memory connection remains explicitly hypothetical, and Appendix E uses a different cycle-based statistic without validating a bridge to the main observable.

  2. Appendix D states that pulse and control runs are trained separately rather than branched from a common pre-pulse checkpoint, and that the source of their pre-pulse differences remains unresolved. For reference seed 43, the maximum pre-pulse excess across checkpoints and scales is 0.001593, while the reported epoch-200 excess at the 1% scale is 0.00394. These are different summaries, but their magnitudes make clear that baseline discrepancies are not negligible relative to some headline effects. There is no control–control distribution establishing how much persistent difference arises without an intervention. Interpretation is further limited because selected pairs can cease to be nearby in the perturbed representation, and their identities change over time. Consequently, persistent aggregate displacement does not establish persistent incompatibility among pairs that remain locally comparable. Mostly two seeds, one principal dataset and model family, unmatched update counts across batch sizes, and unequal recovery durations across pulse timings further restrict the conclusions. The authors disclose these limitations clearly, but disclosure does not supply the missing validation. The resulting contribution remains a small descriptive case study.

Questions for authors
  1. What causes the pre-pulse divergence reported in Appendix D? Beyond initialization and training examples, were minibatch orders, random-number-generator states, and numerical execution settings matched?

  2. Do the retained effects survive branching both arms from exactly the same pre-pulse model and optimizer checkpoint, using matched subsequent minibatch sequences? How do their magnitudes compare with independently repeated control–control differences?

  3. Do the main signs and persistence results survive reasonable positive activation thresholds, magnitude-aware support comparisons, and occupancy-matched controls? For control-selected nearby pairs, how much of the effect remains when requiring proximity in both representations?

2 comments

We appreciate the reviewer's detailed technical assessment and thoughtful suggestions. Several of these raise questions we find genuinely interesting beyond the present evaluation.

On experimental controls: We appreciate the suggested checkpoint-branching and control–control comparisons. Beyond strengthening attribution, they raise an interesting possibility: persistent, scale-dependent differences may reflect not only learning-rate perturbations but a more general dependence of learned representations on training history. We find this distinction worth investigating.

On measurement: Binary activation support is a deliberate choice: we measure which ReLU units are active, rather than their magnitudes or functional contributions. We agree that near-zero crossings and support occupancy may contribute to the response. Threshold-sensitive, magnitude-aware, and occupancy-matched comparisons could help distinguish these effects. If the response is concentrated near activation boundaries, that opens a deeper question about why such differences emerge and persist, and whether they carry functional significance. We find this a particularly interesting direction to explore.

On locality and persistence: Our measurement compares identical control-selected pairs across runs at each checkpoint, while allowing neighborhoods to evolve over training.The reported persistence therefore concerns an aggregate, scale-dependent response, not the continued incompatibility of the same pairs across training. Requiring proximity in both representations and tracking fixed pairs over time could offer complementary perspectives on how this organization evolves.

On significance: Test cross-entropy does distinguish the runs, as reported in the paper. Our interest is not in claiming that frustration reveals an otherwise undetectable difference, but in characterizing how activation-support disagreement is organized across geometric scales and layers. Understanding how these structural and behavioral differences relate remains an open question.

The earlier cycle-based observations under label corruption in Appendix E motivated the present investigation; they are not intended as validation of the pairwise observable. Similarly, the connection to memory remains a hypothesis. Understanding whether and how these structural differences carry task-relevant information is an important direction for further investigation.

Given the paper's current standing, we do not intend to undertake a rushed experimental campaign solely for this rebuttal. We nevertheless consider the suggested controls and comparisons important for understanding the phenomenon more rigorously.

We genuinely appreciate the reviewer's careful engagement and the specificity of these suggestions. They help sharpen the questions we are interested in pursuing, and we are curious to see what emerges from investigating them more carefully.

Reviewer_vKUwOpenReview ↗

Official Comment by Reviewer vKUw

Thank you for the response. It clarifies that the reported persistence is an aggregate pattern over control-selected pairs that can change across checkpoints, and that the link to memory remains a hypothesis. My main concerns remain unresolved. The source of the pre-pulse differences is still unclear. I will maintain the rating but reduce the confidence.

Reviewer_hSeAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper explores differences in the sets of active neurons for training examples that have geometrically similar activations in a ReLU MLP. The authors call this phenomenon representational frustration and define, for a given run rr, the frustration profile F≤qr(t)F_{\le q}^r(t) at training checkpoint tt as the mean Jaccard dissimilarity between the sets of active neurons for pairs of input examples that are within the top qqth percentile of closest pairs, as measured by Euclidean distance between their activations. They observe how this frustration profile is affected by a temporary perturbation in learning rate during training and find that the effects of this perturbation are measurable in terms of frustration (and loss) even long after train and test accuracy have returned to match control levels. By varying qq they study the effect of this perturbation at differing geometric scales (ranging from the effect on the top 0.1% of closest input pairs to the full set of pairs). Finally they explore frustration under various training conditions, such as the training objective (cross-entropy vs MSE), optimizer settings, timing of the perturbation within the training run and depth of the layer being analysed in the MLP.

Critical strengths

The paper gives a clear explanation of the definition of frustration and in particular the example of (1,ϵ,0)(1, \epsilon, 0) vs (1,0,ϵ)(1, 0, \epsilon) was useful for building intuition around the main ideas of the paper.

All analysis is performed on a real task rather than a synthetic dataset. The question asked by the paper is an interesting one and generally the presentation is strong, which makes it pleasant to read.

Critical weaknesses

It is unclear to what extent some of the results shown in the paper can be explained by reseeding - most of the experiments are performed only on small number of seeds (just 2). Importantly, for the core results in figures 1 and 2, it is unclear how much of the observed response can simply be explained by the learning rate perturbation being large enough such that the training run is then "optimising from a different starting point" (i.e. a reseed).

The paper would benefit from giving a stronger motivation for why frustration is an important or useful thing to measure. The authors themselves state that frustration "has no intrinsic quality ordering". Additionally, appendix B.3 states "for nearby states, the differing active coordinates can be small even when many support bits differ", i.e. a large component of ΔF~≤q(t)\Delta \widetilde{F}_{\le q}(t) could just be explained by units whose activations are very close to zero, flipping the unit between active and inactive without any consequence on task relevance or performance. Broadly speaking, it would be useful to see some kind of demonstration of why local support incompatibility matters.

Questions for authors
  1. What do Figure 1 and Figure 2 look like for a pure reseed? For instance, rather than perturbing the learning rate, how much difference is there in frustration profile between two runs, control c\text{c} and reshuffle r\text{r} that are initialised and trained in the exact same way, evaluated on control-selected pairs Pc\mathcal{P}^{\text{c}} and differ only in the ordering of data?
  2. How does the scale of perturbation affect the results? Currently all experiments were performed with a triangular learning-rate pulse from 10−310^{-3} to 10−210^{-2} and back - what do results look like for pulses with peak lr of 2×10−32 \times 10^{-3}, 3×10−33 \times 10^{-3}, etc? This interpolation, in combination with the results from my first question, should make for an interesting comparison of "perturbation scale vs reseeding effect".
  3. As noted in Appendix B.3, there is no magnitude weighting in the current computation of frustration - it would be informative to repeat experiments with some magnitude weighting so that we could distinguish between the case of "many units with activations near zero flipping between active and inactive" versus a meaningfully different support.
3 comments

We appreciate the reviewer's careful reading and constructive suggestions. We find the proposed experiments particularly insightful.

On the learning-rate perturbation: Our motivation comes from earlier work showing that large and small learning rates can produce qualitatively different training dynamics (Li et al., 2019; Leclerc & Madry, 2020), influencing what a network learns rather than merely how quickly it converges. Our pulse probes whether a temporary increase leaves a persistent trace in local activation-support organization after returning to baseline. We do not establish that our chosen pulse crosses a distinct regime boundary. We agree that varying its magnitude would help characterize whether the response is gradual, abrupt, or specific to stronger perturbations.

On reseeding: We agree that a control–control comparison differing only in minibatch ordering is an important baseline. Together with additional seeds and varying pulse magnitudes, this would help distinguish the pulse response from ordinary training trajectory divergence. Whether comparable structural differences arise from ordinary training stochasticity is itself an interesting question about the dependence of learned representations on training history.

On significance: We do not propose frustration as a performance metric, but as an observable of how hidden-state proximity relates to activation-support sharing. The scale- and layer-dependent responses reveal aspects of activation-support organization that predictive accuracy or the all-pair average alone cannot characterize. Whether that organization carries task-relevant information remains an open question, and one we consider worth investigating. We view the present work as a step toward characterizing this relationship, rather than establishing its functional role.

On magnitude weighting: Our statistic deliberately measures binary activation support rather than magnitude. We agree that near-zero crossings may explain part of the response, but that raises a further question: why do these differences emerge and persist, and do they reflect incidental threshold crossings or meaningful local differentiation? Magnitude-aware comparisons, alongside functional interventions, could help distinguish these possibilities. We find this a particularly interesting direction to investigate.

Given the paper’s current standing, we do not intend to undertake a rushed experimental campaign solely for this rebuttal. We are, however, equally curious about the larger story that emerges from these comparisons.

We genuinely appreciate the specificity and insight of these suggestions. They point toward questions we are keen to investigate more carefully. We welcome any further thoughts or experimental comparisons the reviewer considers especially revealing.


Li, Y., Wei, C., & Ma, T. (2019). Towards explaining the regularization effect of initial large learning rate in training neural networks. NeurIPS. https://arxiv.org/abs/1907.04595

Leclerc, G., & Madry, A. (2020). The two regimes of deep network training. https://arxiv.org/abs/2002.10376

Thank you for your response. I agree with you that these are all interesting directions to take this work in. As it stands, since the rebuttal does not contain any new results or analysis, I will maintain my current rating (2: Weak Rejection). I still feel that the submission needs to be strengthened in the areas I mentioned in order for it to be given a higher rating.

In particular, the points I think the authors should prioritise are:

  1. Significance - currently, the paper makes an interesting observation, but one that is unconnected to any broader conclusion that would have a consequence for the study of representational geometry, learning dynamics, etc.
  2. Magnitude weighting - as per the authors' response, this would be a natural and interesting direction in which to take this work. I think we would see a cleaner picture of what is going on with experiments that take magnitude into account.

For my points on the learning rate perturbation sweep and reseeding, I see these as part of ordinary results for completeness and due diligence that any strong paper should include. I would suggest that these are also considered in any future versions of this work.

Agreed. We have spent more effort trying to find the right question than doing the due diligence needed to support it. There is uncertainty at both ends: what these observations mean more broadly, and how robustly we can establish them.

An earlier version explored a broader range of training conditions beyond learning-rate perturbations, which we significantly cut back to improve focus, but without the experimental rigor needed. Magnitude-aware comparisons, as you suggest, could give us a clearer picture of what we are observing. Perhaps this could also help us extend beyond ReLU, where thresholding may be useful when gating is not explicit.

Thank you for taking the work seriously and for holding us accountable.

Reviewer_FQjUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3(+2)
Read review
Paper summary

This paper studies the overlap in active neurons between two inputs that are a certain distance apart in ReLU networks. It does not clearly link this metric to any debates in the field, or improved performance.

It also has major clarity issues. To start with the abstract, this includes phrases like "frustration has no intrinsic quality ordering" which I could not understand. Or "support incompatibility among control-nearby pairs remains displaced through the observed training horizon", without defining support incompatibility, control-nearby pairs, or displacement.

The significance of these results for any purpose remains unclear (as noted in the final sentence of the abstract). Neither the intro nor the discussion links the results to ongoing concerns in the field.

Critical strengths

The results appear to be sound.

Critical weaknesses

The paper is unclear to the point that it could not be published in present form.

The significance of the results, if any, is not discussed.

Questions for authors

What open questions or performance metrics do the results in the paper shed light on?

3 comments

We find several aspects of this review difficult to reconcile.

First, the Paper Summary describes the metric but omits the paper's central empirical investigation: the persistent, scale-dependent response of local activation-support dissimilarity to a temporary learning-rate perturbation. Instead, it moves directly to questioning the metric's utility. The reviewer may find our results unimportant, but an assessment of their significance should first engage with what was actually investigated.

Second, the review claims that the paper does not engage with ongoing concerns in the field. Sections 1 and 6 explicitly discuss representational drift, activation codes, optimization dynamics, and memorization. These connections may be considered insufficient, but they are not absent.

Third, the reviewer states that "the results appear to be sound," yet assigns the lowest soundness score without identifying a specific mathematical, methodological, or experimental flaw.

We welcome criticism of the paper's significance, presentation, and technical substance. We do not expect agreement on the contribution's merits. We do, however, expect the reviewer to hold their review to the same standard of clarity and substantiation they expect of the paper.

Thank you for the response. On your first and second points, my comment on significance is in reference to the empirical investigation, not the metric alone. My concern remains that the results are not connected to any consequences for the field. What conceptual picture should I revise in light of the results? What do the results say about the behavior of neural networks? What lessons can be drawn?

The paper currently reads as a definition followed by a number of experiments, with no discussion of the implications of these experiments. It would be improved by discussing the implications of these experiments. Why are these experiments important?

I will restate my question: What open questions or performance metrics do the results in the paper shed light on?

Section 1 says: "Prior work compares representations across networks (Kornblith et al., 2019), studies the encoding properties of activation patterns (He et al., 2021), and examines how paths in Euclidean input space map into activation space (Lewandowski et al., 2025). Studies of representational drift also show that internal states can evolve while task performance remains stable (Pashakhanloo & Koulakov, 2023)."

To relate the present metric and results to these prior works, it would be useful to explain how a reader's conceptual understanding of these prior works should be revised (or not) in light of the results. As it stands, the list of prior work has not been connected to the proposed metric or results for this reader at least to be able to understand the implications. Having read the paper, I cannot explain the importance of the results to this prior work.

Similarly Section 6, which I have just re-read, lists a number of prior works and restates empirical results but does not explain how the empirical results should change the conceptual ideas arising in these prior works, or the importance of the results for a setting.

I would note that other reviewers pick up on the same issue:

"The paper would benefit from giving a stronger motivation for why frustration is an important or useful thing to measure. ... Broadly speaking, it would be useful to see some kind of demonstration of why local support incompatibility matters." (Reviewer hSeA)

"The claimed added perspective also needs stronger justification. ... recovery of accuracy does not establish that the proposed statistic reveals a consequential difference missed by standard behavioral measurements" (Reviewer vKUw)

On your third point, thank you for pointing this out--the low soundness score was a mistake and I have increased it.

Thank you for the detailed clarification and for correcting the soundness assessment. We think your question about what the experiments teach us points to a distinction that the manuscript should make more explicit.

Why study the relationship between geometry and activation support?

A learned representation must organize similarities among examples while retaining distinctions needed by the task. Geometrically distant inputs may require the same output, while nearby inputs may require different outputs.

What we are interested in is this question of similarity and distinguishability and its place in the mechanics of learning. ReLU networks make this easier to study, since they offer two ways of comparing examples: geometric closeness and activation-support overlap. There is related work, such as ReLU space folds (Lewandowski et al., 2025), which investigates how paths in input space map into activation space, with both theoretical and empirical analysis. Ours happens to be a more empirical investigation.

Studying internal representations to understand learning, rather than focusing only on the final outcome, is not without precedent. Rumelhart et al. (1986) explored how hidden representations reconcile differences between input and output similarity; Grossman et al. (1988) treated internal representations themselves as objects of learning. Both sought to understand the mechanics of learning.

What do the experiments teach us?

What caught our attention was not simply that frustration changed, but how differently it behaved. Under cross-entropy, the closest pairs initially showed a smaller change than pairs overall, but a larger one by epoch 200. Under MSE, the closest pairs already showed a larger change during the measured pulse. Full-batch and minibatch training showed opposite relative responses, as did two layers of the same network.

What we find intriguing is that the same measurement reveals different patterns across objectives, optimization settings, geometric scales, and layers. The changes are not uniform, and a single average would miss much of this organization. This suggests that a training perturbation can redistribute activation-support differences across neighborhoods and layers, rather than simply changing their overall amount.

We do not yet know what produces these differences, but we think they are worth understanding.


Lewandowski et al. (2025). On space folds of ReLU neural networks. https://arxiv.org/abs/2502.09954

Rumelhart et al. (1986). Learning internal representations by error propagation.

Grossman et al. (1988). Learning by choice of internal representations. NeurIPS.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.