- Rating
- 1
- Confidence
- 2(−1)
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper introduces “representational frustration,” measured as Jaccard dissimilarity between ReLU activation supports among pairs selected by hidden-state distance. It studies how this statistic responds to a temporary learning-rate increase, using control-selected pairs and subtracting the all-pair response to characterize local excess. Experiments primarily use shallow, width-512 MLPs on 10,000 CIFAR-10 training examples, with mostly two seeds per condition. The reported differences persist through epoch 200 and vary across objectives, optimizer settings, pulse timings, and layers. These results establish an exploratory descriptive phenomenon.
Critical strengths
-
Selecting pairs using the control geometry avoids conflating a change in pair membership with a change in support dissimilarity in the same comparison. This is a useful methodological choice for interpreting the reported differences.
-
Opposite retained local-excess signs in the two hidden layers recur in both reported seeds. This supports studying the phenomenon beyond a single aggregate statistic.
Critical weaknesses
-
Strict positivity gives arbitrarily small activations the same status as large, consequential activations. Indeed, the example in Section 2 demonstrates that substantial “frustration” can arise from vanishingly small activation differences. This exposes a central ambiguity: the statistic may largely reflect threshold crossings with negligible computational consequences. The paper provides no threshold-sensitivity analysis, magnitude-aware comparison, occupancy-matched null model, or functional intervention that resolves this ambiguity. Subtracting the all-pair response does not control for support occupancy, as Appendix B acknowledges. The claimed added perspective also needs stronger justification. At epoch 200, test cross-entropy already distinguishes the reference runs substantially: 2.716 versus 2.312. Thus, recovery of accuracy does not establish that the proposed statistic reveals a consequential difference missed by standard behavioral measurements. The memory connection remains explicitly hypothetical, and Appendix E uses a different cycle-based statistic without validating a bridge to the main observable.
-
Appendix D states that pulse and control runs are trained separately rather than branched from a common pre-pulse checkpoint, and that the source of their pre-pulse differences remains unresolved. For reference seed 43, the maximum pre-pulse excess across checkpoints and scales is 0.001593, while the reported epoch-200 excess at the 1% scale is 0.00394. These are different summaries, but their magnitudes make clear that baseline discrepancies are not negligible relative to some headline effects. There is no control–control distribution establishing how much persistent difference arises without an intervention. Interpretation is further limited because selected pairs can cease to be nearby in the perturbed representation, and their identities change over time. Consequently, persistent aggregate displacement does not establish persistent incompatibility among pairs that remain locally comparable. Mostly two seeds, one principal dataset and model family, unmatched update counts across batch sizes, and unequal recovery durations across pulse timings further restrict the conclusions. The authors disclose these limitations clearly, but disclosure does not supply the missing validation. The resulting contribution remains a small descriptive case study.
Questions for authors
-
What causes the pre-pulse divergence reported in Appendix D? Beyond initialization and training examples, were minibatch orders, random-number-generator states, and numerical execution settings matched?
-
Do the retained effects survive branching both arms from exactly the same pre-pulse model and optimizer checkpoint, using matched subsequent minibatch sequences? How do their magnitudes compare with independently repeated control–control differences?
-
Do the main signs and persistence results survive reasonable positive activation thresholds, magnitude-aware support comparisons, and occupancy-matched controls? For control-selected nearby pairs, how much of the effect remains when requiring proximity in both representations?
2 comments
Official Comment by Reviewer vKUw
Thank you for the response. It clarifies that the reported persistence is an aggregate pattern over control-selected pairs that can change across checkpoints, and that the link to memory remains a hypothesis. My main concerns remain unresolved. The source of the pre-pulse differences is still unclear. I will maintain the rating but reduce the confidence.
We appreciate the reviewer's detailed technical assessment and thoughtful suggestions. Several of these raise questions we find genuinely interesting beyond the present evaluation.
On experimental controls: We appreciate the suggested checkpoint-branching and control–control comparisons. Beyond strengthening attribution, they raise an interesting possibility: persistent, scale-dependent differences may reflect not only learning-rate perturbations but a more general dependence of learned representations on training history. We find this distinction worth investigating.
On measurement: Binary activation support is a deliberate choice: we measure which ReLU units are active, rather than their magnitudes or functional contributions. We agree that near-zero crossings and support occupancy may contribute to the response. Threshold-sensitive, magnitude-aware, and occupancy-matched comparisons could help distinguish these effects. If the response is concentrated near activation boundaries, that opens a deeper question about why such differences emerge and persist, and whether they carry functional significance. We find this a particularly interesting direction to explore.
On locality and persistence: Our measurement compares identical control-selected pairs across runs at each checkpoint, while allowing neighborhoods to evolve over training.The reported persistence therefore concerns an aggregate, scale-dependent response, not the continued incompatibility of the same pairs across training. Requiring proximity in both representations and tracking fixed pairs over time could offer complementary perspectives on how this organization evolves.
On significance: Test cross-entropy does distinguish the runs, as reported in the paper. Our interest is not in claiming that frustration reveals an otherwise undetectable difference, but in characterizing how activation-support disagreement is organized across geometric scales and layers. Understanding how these structural and behavioral differences relate remains an open question.
The earlier cycle-based observations under label corruption in Appendix E motivated the present investigation; they are not intended as validation of the pairwise observable. Similarly, the connection to memory remains a hypothesis. Understanding whether and how these structural differences carry task-relevant information is an important direction for further investigation.
Given the paper's current standing, we do not intend to undertake a rushed experimental campaign solely for this rebuttal. We nevertheless consider the suggested controls and comparisons important for understanding the phenomenon more rigorously.
We genuinely appreciate the reviewer's careful engagement and the specificity of these suggestions. They help sharpen the questions we are interested in pursuing, and we are curious to see what emerges from investigating them more carefully.