acceptodds
Under review as a conference paper at ICLR 2027

Traversing the solution space of neural networks with Hessian Null Space Continuation

Abstract

On a single task, deep neural networks can learn a wide range of solutions, depending on their optimizer, training data, architecture, and hyperparameters. Many of these solutions are surprisingly : rather than isolated points in the weight space, they are connected by low loss regions. Despite this observation, the diversity of solutions in terms of their internal computation in these regions has not been characterized. A parallel line of work has identified the : many neural network solutions exist with similar training loss yet distinct internal structures. However, it is unclear how these diverse solutions are related in weight space. Here, we unify these subfields and demonstrate for the first time that there exist many different internal mechanisms within a local mode connected region in weight space. To do so, we introduce Hessian Null Space Continuation (HNC), a scalable method that uses local curvature information to traverse regions of weight space that preserve network function. HNC can additionally be steered toward solutions with specified properties. In RNNs trained on a memory task, HNC drives the networks to learn drastically different representations and dynamics, even with maintained behavior. In ImageNet-trained Vision Transformers, HNC finds alternative representations which differ more from the original network than any independently trained models of different architectures and training objectives. In reinforcement-learning agents, HNC uncovers a distinct behavioral strategy at comparable return in a navigation task, and exposes a reward-hacking strategy in an AI Safety Gridworld environment. Finally, HNC provides local geometric information about the solution distribution, showing how model size and task complexity shape its dimension and functional sensitivity. Together, our results show that a surprisingly large amount of representational diversity exists near a single trained solution, which is unseen by standard gradient-based optimization techniques. Our domain-agnostic method, HNC, can identify and quantify this diversity, opening new possibilities for mechanistic understanding of solution spaces and providing a principled basis for model merging, editing, and fine-tuning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
solution degeneracy, loss landscape geometry, mode connectivity, representational similarity, Hessian

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_WeTWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Hessian Null Space Continuation (HNC) which explore alternative neural network solutions while approximately preserving the input-output behavior of an anchor model. The method uses Hessian steering and restoration steps to traverse the solution space and demonstrations it utility across recurrent neural networks, vision transformers, and reinforcement learning. The approach is conceptually interesting and provides a potentially useful tool for studying the geometry of approximately function-equivalent solutions. However, I have concerns regarding the depth of the experimental evidence, comparisons with existing methods, and the generality and interpretation of the findings.

Critical strengths

Strengths

  1. Interesting and well-motivated approach. The paper introduces a Hessian-based continuation framework for exploring approximately function-preserving neural network solutions. The idea of explicitly steering along low-curvature directions while using restoration steps to maintain functional similarity is interesting and provides a systematic approach to solution-space exploration.

  2. Connection between solution-space geometry and representational diversity. The paper investigates an important question: to what extent can internal representations differ while preserving model behavior? The connection to representational convergence and the Platonic Representation Hypothesis provides an interesting motivation, even if the broader interpretation requires further investigation.

  3. Experiments across multiple domains. The authors demonstrate HNC in recurrent neural networks, vision transformers, and reinforcement learning. Although the depth of individual experiments is limited, the breadth of applications provides useful preliminary evidence that the framework can be applied beyond a single architecture or task.

  4. Multiple measures of representational and functional similarity. The paper does not rely solely on one representation similarity metric. The use of different representational and dynamical measures, together with functional-preservation evaluations, provides a more informative characterization of the resulting solutions than a single metric would offer.

  5. Potential as a research tool. Beyond the specific applications considered, HNC may provide a useful framework for investigating the geometry of neural network solution spaces and studying the relationship between functional equivalence and internal representations.

Critical weaknesses

Weakness

1. Broad Claims Without Sufficient Experimental Depth

My primary concern is the mismatch between the breadth of the paper's claims and the depth of the supporting evidence. Although the authors demonstrate HNC across multiple domains, many experiments serve primarily as proof-of-concept demonstrations rather than systematic investigations of those claims.

In particular:

  • 3-bit flip-flop task: The experiment provides a useful controlled demonstration of representational diversity. However, the task requires maintaining only three binary variables, while the recurrent network uses 64 hidden units. It remains unclear whether the observed diversity persists in more complex tasks or depends substantially on redundant representational capacity.

  • Reinforcement learning: The reward-hacking experiment is conducted in a simplified environment with a known exploit. While this demonstrates a potential application of HNC, it does not establish whether the method can identify previously unknown reward-hacking behaviors in more realistic environments or provide advantages over existing exploration methods.

  • Vision experiments: The authors evaluate representations on Places365 using an ImageNet-pretrained model, following the evaluation setting motivated by the Platonic Representation Hypothesis (PRH). Although this choice is reasonable for studying cross-dataset representation similarity, it makes the interpretation of functional preservation relative to the original training task less direct. Evaluating both representation similarity and functional preservation on held-out ImageNet data would provide a clearer comparison.

  • Optimizer-induced bias: The paper suggests that representational convergence among independently trained models may partly reflect optimizer-induced bias. However, this explanation is not directly isolated by the experiments. Discovering alternative representations through HNC does not establish why conventional training produces similar representations. More controlled comparisons across training seeds, optimizers, architectures, and model scales would strengthen this interpretation.

I acknowledge that the paper includes a pretrained vision transformer, multiple representation and dynamical similarity measures, and separate investigations of solution-space geometry across model scales. Nevertheless, these experiments do not fully resolve the questions above. Overall, the evidence demonstrates that HNC can discover alternative representations in selected settings, but does not yet sufficiently establish the generality or broader implications of these findings. I believe a more focused and systematic investigation of the central claims would substantially strengthen the work.

2. Insufficient Comparisons with Existing Methods and Computational Efficiency

The idea of traversing connected low-loss regions has been studied, including by [1] and [2], which are acknowledged in the paper. Although HNC introduces a distinct Hessian-based continuation procedure with an explicit functional preservation objective, its additional advantages over existing solution-space exploration methods remain unclear. Furthermore, prior work on deep ensembles, including [3], provides useful reference points for understanding the diversity obtainable through independently trained models. Diversity-promoting ensemble learning, representation regularization, and explicit dissimilarity objectives also provide relevant alternative approaches. Although these methods do not necessarily target the same function-preservation objective, comparisons would help establish the utility of HNC.

The existing comparisons with undirected steering and penalized optimization are useful, but I believe a more systematic evaluation of representational divergence, functional preservation, and computational cost is necessary. In particular, HNC relies on Hessian-vector products, which may introduce substantial computational overhead compared with conventional first-order methods, even when their asymptotic complexities are comparable. Reporting actual wall-clock time, GPU resources, and memory consumption would help clarify the practical trade-offs. Without such comparisons, it remains difficult to determine whether HNC offers meaningful advantages over existing approaches, particularly given its additional computational requirements.

[1] Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred A. Hamprecht. Essentially no barriers in neural network energy landscape. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1309–1318. PMLR, 2018. URL https://proceedings.mlr.press/v80/draxler18a.html.

[2] Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P. Vetrov, and Andrew Gordon Wilson. Loss surfaces, mode connectivity, and fast ensembling of DNNs. In Advances in Neural Information Processing Systems, volume 31, 2018.

[3] Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspective. arXiv preprint arXiv:1912.02757, 2019.

3. Functional Preservation and Numerical Reliability

The function preservation objective is evaluated using a finite probe set. Consequently, the empirical Hessian null space may contain directions that preserve outputs on selected samples without necessarily preserving the model's behavior across the broader data distribution. Although the paper includes held-out evaluations and probe-set scaling analyses, a more systematic investigation of the relationship between probe-set size, representational divergence, and functional preservation would strengthen the conclusions.

Questions for authors
  1. Motivation for Hessian-based continuation: What is the specific advantage of using Hessian null-space continuation over first-order constrained optimization? Is there a theoretical or empirical reason why the Hessian-based direction is particularly effective at discovering representational diversity? Prior deep ensemble methods utilize the gradients or large learning rate as basis to diversify the anchor models along the low loss channel or connectivity. I am curious about what can be explained in this work as it is also using those information as basis.

  2. Steering and restoration: How do the steering and restoration steps interact? In particular, could restoration favor certain regions of the solution space or reverse the changes introduced by steering?

  3. Role of the null-space geometry in representation: What properties of this space make substantial representational changes possible while preserving the model's outputs?

7 comments
AuthorsOpenReview ↗

Response 1/4: More experiments demonstrating generality of HNC

We thank the reviewer raising those concrete points. We address each weakness and question below.

 

W1 (Broad claims without sufficient experimental depth), 3-bit flip-flop task: "It remains unclear whether the observed diversity persists in more complex tasks or depends substantially on redundant representational capacity."

To test if HNC can find diverse solutions in more complex tasks, and if such ability depends substantially on redundant representational capacity, we applied HNC to trained RNNs with 3 to 256 hidden units on the 3-bit task, and RNNs on flip-flop tasks with 3 to 32 bits (Appendix J.9). The tables give the distance from the anchor that HNC reaches at held-out accuracy of at least 0.99. With 3 units, which is the theoretical minimum for the task, HNC finds no more divergence than exists between independently trained networks ("retraining"). However, as long as the network has one spare unit, the mean DSA distance between HNC alternatives and the anchor exceeds the variability across seeds, so the diversity does not depend on large redundant capacity.

Representational diversity with respect to different network capacity: 3-bit task, mean over 3 anchors.

Hidden units348163264128256
1 - CKA0.030.100.210.420.280.370.540.59
DSA0.230.270.170.470.460.590.851.04

 

The diversity also persists on harder tasks. With more bits, the CKA distance stays above retraining up to 16 bits and the DSA distance at every difficulty. We also tested two more complex tasks, where different input dimensions interact with each other: contextual decision making [Mante et al. 2013], where a context cue selects which of two evidence streams to integrate over time, and a multitask setting where one RNN learns 20 cognitive tasks [Yang et al. 2019]. HNC again changes the dynamical distance several times more than independent seeds do.

HNC applied to harder tasks: mean over 12 anchors.

Bits34581632
1 - CKA0.440.420.380.340.270.06
DSA0.740.660.700.570.530.20
TaskDSA, HNCDSA, retraining
Contextual decision making (128 units)0.610.08
20 cognitive tasks in one RNN (256 units)0.360.05

 

W1, reinforcement learning: "[...] it does not establish whether the method can identify previously unknown reward-hacking behaviors in more realistic environments or provide advantages over existing exploration methods."

Our RL experiments aim to show that reward-matched but behaviorally distinct policies exist around a trained policy. To validate that HNC can find a reward-hacking policy, we need a setting where the hacking policy is known in advance. We therefore ran it on the boat race task from AI Safety Gridworlds, which was designed for this purpose and has a documented exploit. Finding previously unknown reward hacks in realistic environments, and comparing them with exploration methods, is beyond the scope of the current paper and remains an exciting direction for future work.

AuthorsOpenReview ↗

Response 2/4: More experiments demonstrating generality of HNC (continued)

W1 (continued), vision experiments: "Evaluating both representation similarity and functional preservation on held-out ImageNet data would provide a clearer comparison."

We repeated the ViT-S walk with probe images from the ImageNet training set and evaluated both similarity and function on held-out ImageNet validation images (Appendix K.12). The result of Section 4 still holds: "HNC produces representations that differ more from the anchor than those of any other trained model in our comparison set, even those of an untrained network". We added this result to Section 4 (lines 340-341): "The same holds when the walk is steered on ImageNet images and evaluated on held-out ones (Appendix K.12)."

Max-over-layers CKA to the anchor ViT-S on held-out ImageNet validation images.

ModelMax-over-layers CKA to anchor
ViT-B (ImageNet-21k)0.95
ViT-S (ImageNet-1k)0.92
ViT-L (ImageNet-21k)0.92
ViT-S (DINO)0.87
DeiT-III-S0.84
ResNet-500.84
Untrained ViT-S0.61
ConvNeXt-S0.51
HNC endpoint0.42

 

W1, optimizer-induced bias: "More controlled comparisons across training seeds, optimizers, architectures, and model scales would strengthen this interpretation."

To test whether standard training favors a narrow set of solutions, we trained 20 flip-flop RNNs per condition, varying one factor at a time: seed, width, optimizer (Adam or SGD), and architecture (RNN, GRU, LSTM) (Appendix J.10). All of these networks land in the same family of solutions, with one stable fixed point per memory state. The HNC endpoints instead implement memory with drastically different mechanisms: 15 to 35 stable fixed points, slowly drifting activity, or a limit cycle. HNC therefore reaches dynamical solutions that none of these training choices produced. This suggests that standard training is biased toward a narrow family of solutions, even though the task admits many others. Explaining why standard training has this preference is beyond the scope of the current study, and there has been an existing line of work in deep learning theory on the implicit bias of optimization [Gunasekar et al. 2017; Soudry et al. 2018; Arora et al. 2019; Vardi 2023].

We have also updated the wording in the main paper. It now refers to the bias of standard training choices, not of the optimizer alone. For example, the Introduction now reads (lines 94-98): "Characterizing the broader set of solutions that a model admits helps separate the structure that the task demands from the structure that a particular training procedure selects." In Section 3, we added (lines 257-259): "By contrast, nearly all trained 3BFF networks converge to the same solution with eight stable fixed points despite changes in seed, width, optimizer, or architecture (Appendix J.10), suggesting that standard training favors a narrow set of solutions."

 

We believe that the generality across supervised learning and reinforcement learning, and across recurrent neural networks, vision transformers, and convolutional neural networks, makes a strong case for the broad applicability of our method. Across all settings, we find that substantial representational and mechanistic diversity exists in the low-loss, connected region around a single trained network. Meanwhile, overparameterized networks are known to have a large null space. Our scaling result in Section 6 shows that the null fraction increases with model size, and HNC can exploit this large null space to find representationally and mechanistically diverse solutions.

AuthorsOpenReview ↗

Response 3/4: Comparison with existing methods and computational cost; probe-set size effect

W2. Insufficient comparisons with existing methods and computational efficiency: "[...] its additional advantages over existing solution-space exploration methods remain unclear. [...] Reporting actual wall-clock time, GPU resources, and memory consumption would help clarify the practical trade-offs."

On 3BFF RNNs, we compared HNC with low-loss curves between independently trained networks ([1], [2]), an ensemble method (independently trained networks), orthogonal gradient descent [Farajtabar et al. 2020] (a first-order method used in continual learning to preserve a network's outputs), and explicit dissimilarity objectives (both local and global). Every method starts from an already trained network and explores diverse solutions from that starting point, except independent training and the global dissimilarity objective, which train from scratch. Among all methods that start from the anchor, HNC moves representations farthest at matched function preservation. In terms of wall-clock time, HNC is 2 to 30x slower than the other methods. Its peak memory is comparable to theirs. HNC therefore trades running time for diversity of solutions without requiring more memory.

On ViT, we also compared HNC with a first-order method (OGD). HNC changes the representation about two to three times more than OGD. Here, HNC is 5x slower per step than OGD with the restore step but needs a third of its memory.

Overall, HNC is uniquely suited when (1) the objective is to find solutions as diverse as possible around a given trained network, and (2) the scientific question concerns the local loss landscape rather than the global set of solutions.

Divergence from the anchor (3BFF RNN and ViT-S).

MethodRNN, Δ\DeltaCKARNN, Δ\DeltaDSAViT-S, Δ\DeltaCKA
HNC0.83 ±\pm 0.020.44 ±\pm 0.120.48 ±\pm 0.12
OGD0.04 ±\pm 0.010.18 ±\pm 0.040.22 ±\pm 0.01
OGD + restore step0.15 ±\pm 0.020.19 ±\pm 0.040.00 ±\pm 0.00
Similarity penalty0.02 ±\pm 0.000.25 ±\pm 0.00n/a
Low-loss curve to another seed0.12 ±\pm 0.010.19 ±\pm 0.04n/a
Different seed (reference)0.11 ±\pm 0.010.06 ±\pm 0.00n/a

Compute cost on the CKA-steered 3BFF RNN, on one NVIDIA H200.

MethodWall-clock time (min)HNC slowdownPeak GPU memory (MB)
HNC571x89
OGD1.832x125
OGD + restore step5.011x125
Similarity penalty2.325x89
Low-loss curve to another seed262.2x86
Different seed (reference)4.014x81

Compute cost on ViT-S, on one NVIDIA H200.

MethodTime per step (s)Time per walk (GPU-hours)Peak GPU memory (GiB)
HNC573.741
OGD0.60.02120
OGD + restore step10.50.14120

 

W3. Functional preservation and numerical reliability: "[...] a more systematic investigation of the relationship between probe-set size, representational divergence, and functional preservation would strengthen the conclusions."

We thank the reviewer for raising this point. The relationship between probe-set size, representational divergence, and function preservation is examined in Appendix K.9 ("Scaling with the size of the probe set", Figure 37). A larger probe set preserves the function better on held-out ImageNet images that the walk never constrained. Because the function is held more firmly, the walk stays under its loss ceiling for more steps, so it runs longer and moves the representation farther from the anchor. The compute per step grows about linearly with the probe size.

Reviewer_dgqtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Hessian Null Space Continuation (HNC) to uses local curvature information and a steer method to traverse over the loss landscape to optimize a subgoal (e.g. diversity of representation) without sacrificing the primitive loss. HNC is validated on various tasks including RNNs, vision transformers and reinforcement learned models, which are used to show that a large amount of representational diversity exists near a single trained solution.

Critical strengths
  1. This paper presents an interesting perspective beyond conventional mode connectivity. Previous mode-connectivity work only asks two trained solutions can be connected, and this work extends the scope to how much the internal realization of the same function change within a connected region.
  2. HNC moves beyond static loss landscape and proposes the continuation procedure to convert a local linear redundancy into a traversable region of parameter space.
  3. The steering component of HNC is interesting and could possibly bring a multi-goal optimization tool. The steering is effectively optimizing a secondary goal while subjecting to the same functional output.
  4. The experimental results in multiple domains, e.g. RNNs, vision transformers and RL models, strengthen the paper's possible application substantially. For example, the reward-hacking example is illustrative as HNC can act as a stress test of a trained model by searching for different policies that retain a similar proxy reward.
Critical weaknesses
  1. The manuscript mentions model merging as a potential implication. It seems more necessary to directly add the model merging experiment in the current version. If HNC offers a particular method to discover substantially different representations, a direct application using model soup among these representation could bring in realistic impact and also directly validate the connection between HNC and model merging.
  2. From a Jacobian/NTK-style perspective, substantial local parameter redundancy should be expected in heavily overparameterized networks. Therefore, I personally do not view the presence of many low-curvature/function-insensitive directions itself as a major discovery.
  3. Another concern is the much narrower solution space from one local minima solution than there could possibly exist. For example, if we would like to fit a simple XOR function, the function choice can be either a 2-layer network or a 4-layer network. The solutions between these two architectures are clearly not connected. Therefore, I believe the scope should be stated more precisely.
  4. Representational diversity should not automatically be treated as meaningful diversity in real scenarios. For example, parameter symmetries and other equivariant transformations could be dissimilar in representation but not represent mechanistic dissimilarity. The experiments that demonstrate the behavioral differences in RNN and RL mitigate this concern, but a general notion about what defines dissimilarity is still missing.
Questions for authors

Other questions:

  1. The current scope of experiments can also be better explained in the manuscript. For example, what obstacles need to be fixed when applying HNC to an LLM of around ~10B scale and what new phenomenon there could be.
3 comments
AuthorsOpenReview ↗

Response 1/3: Model merging; clarifying the contribution of HNC

We thank the reviewer for the positive feedback and the thoughtful reviews! We respond to each point below.

W1. Directly add the model merging experiment in the current version. If HNC offers a particular method to discover substantially different representations, a direct application using model soup among these representations could bring in realistic impact and also directly validate the connection between HNC and model merging.

Thank you for the suggestion. We have added the model merging experiment in Appendix O. We merged the 3BFF anchor with its HNC endpoints by averaging their weights, either with one HNC network or with several HNC networks at once. Since HNC alternatives stay in the same low-loss region as the anchor, all of these merged networks still solve the task, with at least 99% of output bits correct.

To merge the anchor θ0\theta_0 with one HNC endpoint θHNC\theta_{HNC}, we use θ(α)=(1−α)θ0+αθHNC\theta(\alpha) = (1-\alpha)\theta_0 + \alpha\theta_{HNC}, with α\alpha between 0 and 1. We then tested the noise robustness of the merged networks against their parent networks. We added Gaussian noise to the hidden state during a 1000-step memory period without input, and measured the fraction of trials in which the network still outputs the stored memory. The merged networks retain the stored memory better than both the anchor and the HNC endpoint over a wide range of α\alpha (Table 1).

This robustness also grows as more HNC endpoints θ1,…,θk\theta_1, \dots, \theta_k are included in the merge, θ=(θ0+θ1+⋯+θk)/(k+1)\theta = (\theta_0 + \theta_1 + \dots + \theta_k) / (k + 1) (Table 2). Model merging between a trained network and its HNC alternatives therefore improves the noise robustness of the resulting networks.

 

Table 1. Fraction of memory retained after a 1000-step hold with hidden-state noise (σ=0.14\sigma = 0.14). Bold: higher than both parents. Mean ±\pm s.e.m. over 47 anchor-endpoint pairs.

HNC endpointα=0\alpha = 0 (anchor)α=0.25\alpha = 0.25α=0.5\alpha = 0.5α=0.75\alpha = 0.75α=1\alpha = 1 (HNC endpoint)
Undirected0.61 ±\pm 0.030.70 ±\pm 0.020.73 ±\pm 0.020.54 ±\pm 0.030.13 ±\pm 0.00
DSA-steered0.61 ±\pm 0.030.71 ±\pm 0.030.68 ±\pm 0.030.66 ±\pm 0.030.55 ±\pm 0.02

 

Table 2. Fraction of memory retained after a 1000-step hold with hidden-state noise (σ=0.15\sigma = 0.15), when the anchor is merged with kk other networks (k=0k = 0 is the anchor alone). Independently trained networks are merged after matching their hidden units to the anchor's. Mean ±\pm s.e.m. over three anchors.

Networks merged with the anchork=0k = 0k=1k = 1k=2k = 2k=4k = 4k=8k = 8
Undirected HNC endpoints0.43 ±\pm 0.110.59 ±\pm 0.050.66 ±\pm 0.080.81 ±\pm 0.070.86 ±\pm 0.09
DSA-steered HNC endpoints0.43 ±\pm 0.110.63 ±\pm 0.090.72 ±\pm 0.070.75 ±\pm 0.050.74 ±\pm 0.03
Independently trained (aligned)0.43 ±\pm 0.110.19 ±\pm 0.010.22 ±\pm 0.020.22 ±\pm 0.010.21 ±\pm 0.03

 

W2. I personally do not view the presence of many low-curvature/function-insensitive directions itself as a major discovery.

We agree that local parameter redundancy is expected in overparameterized networks, and we do not claim its existence as a contribution. HNC builds on this established observation to construct multi-step, functional-preserving paths through weight space. We clarify that our contribution is to characterize the representational and dynamical diversity accessible along these paths and to provide a method for deliberately finding diverse solutions. The existence of locally function-insensitive directions alone does not establish how far they can be followed while preserving function, or how substantially internal representations and dynamics can change along the resulting paths. We have added a Related Work section (Appendix C) that relates HNC to prior null-space methods, to better contextualize and motivate HNC.

AuthorsOpenReview ↗

Response 2/3: Scope of the solution space; how to define diverse solutions

W3. "Another concern is the much narrower solution space from one local minima solution than there could possibly exist. [...] I believe the scope should be stated more precisely."

Thank you for your comment. HNC explores parameter space for a fixed architecture, starting from a trained anchor and tracing a continuous, function-preserving path in the parameter space. We agree that the same learning problem can be implemented by networks with different architecture and parameterization. While it is interesting how such architectural inductive bias can shape learned solutions, connecting them in parameter space requires additional construction beyond our current framework. We have clarified this in the Discussion (lines 492-494): "HNC is also a local search algorithm that operates within a fixed architecture. It therefore cannot reach solutions in disconnected basins, or solutions that implement the same function with a different architecture."

However, within a fixed architecture, there are multiple ways to broaden the search of HNC. In Appendix I, we combined HNC with a global search method: iteratively finding alternative solutions locally, then retrain the network to be repelled from all existing alternative solutions. This allows us to reach greater distance in the weight space, comparable to the weight distance between independently trained networks from different seeds. Besides, we can loosen the loss ceiling during the walk and filter the visited networks by their loss afterwards, which allows HNC to cross loss barriers and basin boundaries.

Finally, we could steer against solutions trained with other architectures or optimizers with architecture-agnostic representational distances. This would encourage HNC to discover representations distinct from a broader collection of existing solutions, while the explored networks themselves remain within the chosen architecture.

 

W4. Representational diversity should not automatically be treated as meaningful diversity in real scenarios. For example, parameter symmetries and other equivariant transformations could be dissimilar in representation but not represent mechanistic dissimilarity. [...] a general notion about what defines dissimilarity is still missing.

Thank you for raising this important point. In our experiments, changes in representations are often accompanied by changes in temporal dynamics, behavior, and out-of-distribution (OOD) performance. Characterizing these properties jointly provides stronger evidence that the observed diversity extends beyond changes in representational coordinates alone.

We have also added a ViT experiment showing that the observed representational differences cannot be explained by parameter symmetries alone (Appendix K.11). We added on lines 484-485: "In ViTs, the endpoints also differ from the anchor on measures that parameter symmetries cannot change (Appendix K.11)."

Similarly, for RNNs, one symmetry-invariant notion of mechanistic dissimilarity is the set of dynamical objects that implement the computation, which is how the RNN reverse-engineering literature defines a mechanism (Sussillo & Barak, 2013; Driscoll et al., 2024). Trained networks store memories at stable fixed points, while HNC endpoints also use ongoing motion in the readout-null space (undirected), slowly decaying trajectories (CKA-steered), and limit cycles (DSA-steered) (lines 254-257, Appendix J.11). Parameter symmetries preserve these objects, so a network that stores memory on limit cycles or slowly decaying trajectories instead of fixed points implements a distinct mechanism.

We share the reviewer's concern that optimizing any single diversity metric can produce changes in that metric without corresponding changes in the properties of interest (Goodhart's principle). We therefore emphasize evaluating representations, dynamics, and behavior together, particularly under perturbations or distribution shifts to discriminate functionally comparable solutions. In practice, we believe it also helps to define dissimilarity in a more application- or scenario-specific way, based on which aspects of the network's computation are of practical or scientific interest.

We have clarified our operational definition of "diverse solutions" in Section 2 (lines 129-132): "The appropriate notion of difference depends on the aspect of computation of interest: here we use Centered Kernel Alignment and Dynamical Similarity Analysis (Kornblith et al., 2019; Ostrow et al., 2026) to quantify differences in network's representations and dynamics, but properties such as generalization or robustness to perturbations are equally valid criteria."

AuthorsOpenReview ↗

Response 3/3: Applying HNC to a 7B-parameter LLM

Q1. The current scope of experiments can also be better explained in the manuscript. What obstacles need to be fixed when applying HNC to an LLM of around ~10B scale and what new phenomenon there could be?

Thank you for the question. We now state the scope of our experiments in the Introduction (lines 119-120): "Our experiments cover networks of up to tens of millions of parameters. We discuss the potential to apply HNC to billion-parameter language models in Appendix P." We have added an experiment at this scale in Appendix P. We applied HNC to Qwen2.5-7B-Instruct (7.6B parameters) to reduce sycophancy, the tendency of the model to agree with a wrong answer suggested by the user. The walk changed only the MLP weights of the last four transformer blocks (0.8B parameters) and preserved the model's next-token decisions on ordinary chat. It was steered to reduce how much the model's answer depends on an internal sycophancy direction [Fel et al. 2026]. Each step took about 30 minutes on one H200 GPU. After five steps, the model repeated the wrong suggestion about half as often in short replies, while its accuracy, perplexity, and refusal rate were unchanged and 99% of its held-out next-token decisions were kept.

 

Scaling HNC to this size required addressing several obstacles:

  • Memory: Moving all parameters does not fit on one GPU beyond a few billion parameters. We therefore walk only the MLP weights of a few layers. This is standard practice in LLM model editing, where methods such as ROME [Meng et al. 2022] and MEMIT [Meng et al. 2023] edit only a few MLP layers. HNC could likewise walk within a low-rank adapter such as LoRA [Hu et al. 2022]. Moving all parameters would require sharding the parameter vectors across GPUs.
  • Runtime: Each step took about 30 minutes on one H200 GPU. Half-precision products are several times faster, but they cannot resolve the sharp directions in the steering gradient, and the projection diverges. We therefore keep the projection in single precision and accept the slower step.
  • What quantity to preserve: At every position, the output of an LLM is a distribution over more than 150,000 tokens, so matching all logits is both expensive and stricter than needed. We preserve only the anchor's top-8 next-token logits on a probe of chat conversations. We then check on held-out conversations that the top-8 next-token set is still kept. The choice of which inputs and outputs to preserve defines the function that HNC keeps fixed, so it has to be made for each application.

 

At LLM scale, we find that the ability to change a behavior while preserving the constrained function depends on the steering objective. We also observe that an internal proxy can continue to improve after the corresponding behavioral change has plateaued.

 

We thank the reviewer again for the constructive suggestions and feedbacks, which have greatly improve our paper!

Reviewer_TV7ZOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work focuses on understanding the functional space of solutions found by gradient-based optimizers. Recent techniques for large models, such as model merging or model editing, make a proper characterization and traversal of the functional solution space a timely and relevant topic. There is a growing body of literature around the observation that seemingly unrelated solutions in parameter space found by SGD are in fact connected by a region of low loss, i.e., mode connectivity. Unlike these works, which typically train multiple models with SGD then connect them post hoc, the authors propose using a single trained network as an "anchor" then exploring the local solution space around it. To do this, they develop the Hessian Nullspace Continution (HNC) which linearly approximates the model around the anchor using its Hessian. Notably, this is done to minimize a function-matching loss instead of the task loss. HNC makes steps along the nullspace of the Hessian to explore models that are ~functionally equivalent to the anchor. To mitigate the error of their approximation, HNC adds a corrective function matching step as well as periodic relinearizations. Through three experiments, the authors show the utility of HNC in exploring the space of solutions for small RNNs trained on a memory task, large image classification models, and RL policy networks on plume-tracking and an AI safety grid world. The results show that even around a single trained solution, there exist diverse functional changes. The authors also demonstrate that they results have implications for the Platonic Representation Hypothesis (PRH) and reward hacking. Overall, this is an interesting and useful paper that is well-written, justified, and evaluated.

Critical strengths
  • Clear writing Overall, I find that the paper is well-written and easy to follow. I think the community will benefit from the presentation of this paper even if they do not directly build on its core method.
  • Robust evaluation The authors do a good job of justifying and ablating various decisions, specifically, the use of a function-matching loss instead of task loss, the computationally tractable algorithm for the Hessian nullspace, systematic study of the sub-steps of HNC. This speaks to the soundness of the results.
  • Novel and important contributions A diverse set of tasks demonstrates the practical utility of their technique. In particular, I found the results of the image classification models and PRH especially interesting and of great importance to the community. There has been speculation that the scale of modern models is responsible for their generalization capabilities and accurate reflection of structure in the world. Though not conclusive, this experiment showed that those properties may be an artifact of our training procedures---an important distinction to draw.
Critical weaknesses
  • The authors make a strong claim: "demonstrate for the first time that there exist many different internal mechanisms within a local mode connected region in weight space". While I agree that this work is most comprehensive towards this finding, I do believe there are related prior works that show similar findings, see [1, 2, 4]. Oddly, the authors do cite [1] to note the existence of diverse experts near a solution (perhaps the distinction is that in this work the task is implicitly the same?).
  • One concern about HNC is its computational cost due to the Hessian, which the authors spend significant effort ameliorating. Furthermore, the second-order approach is what distinguishes their work for prior first-order methods. Accordingly, I expect a comparison to first-order methods [2, 3, 4] to show what is gained by their approach. Similarly, a random baseline like [1] would be an easy comparison.
  • Another key distinction of this work is the focus on a function-matching loss. As a result, the third RL experiments confused me. Isn't searching for different behavioral policies under the same reward like a task-matching loss? To me, a function-matching loss would be producing the same state-action trajectories but the policy network's internal mechanisms changing. While I think the results are interesting for this experiment, it seems out of place in context of the paper.

[1] Neural Thickets: https://arxiv.org/abs/2603.12228

[2] Mechanistic Mode Connectivity: https://proceedings.mlr.press/v202/lubana23a.html

[3] Numerical Exploration of Training Loss Level-Sets in Deep Neural Networks: https://arxiv.org/abs/2011.04189

[4] Orthogonal Gradient Descent: https://proceedings.mlr.press/v108/farajtabar20a.html

Questions for authors
  1. It isn't clear to me that the different mechanisms in the 3 bit FF task are distinct. At least the undirected case appears to be just a noisy version of the trained network. Could you help me understand what the 7 and 15 FP solutions are doing? Basically, I don't think there are multiple distinct ways to memorize 2n2^n random values. For example, a network could traverse through 2 FP for every logical state and then it is a "16 FP solution." But is that really a different mechanism?
  2. How does HNC compare to low-order methods? Can those methods find similar solutions in a local region? In particular, [4] uses the Jacobian nullspace which is equivalent to nullspace of Gauss-Newton matrix used by HNC. I do not think you need to robust comparison for every result, but some comparison should be made.
5 comments
AuthorsOpenReview ↗

Response 1/3: Novelty relative to prior work

We thank the reviewer for their positive evaluation and thoughtful questions! Here we address all the weaknesses and questions raised in the review.

W1. The authors make a strong claim: "demonstrate for the first time that there exist many different internal mechanisms within a local mode connected region in weight space". While I agree that this work is most comprehensive towards this finding, I do believe there are related prior works that show similar findings, see [1, 2, 4].

We clarify that the key novelty of our work is that starting from a trained network, HNC obtains alternative solutions that share the same input-output function but have distinct internal representations, all within the same connected low-loss region of weight space. To our knowledge, none of [1, 2, 4] establishes these properties jointly (see table below), and we have added a Related Work section in the appendix (Appendix C) to further contextualize and clarify the contribution of HNC.

  • While Neural Thickets [1] shows that random perturbations around pretrained weights can improve performance on downstream tasks, these diverse experts differ from the pretrained network in their input-output function. Moreover, [1] does not examine the representations of these solutions, nor does it establish that the perturbed networks lie in the same low-loss region as the pretrained network without a loss barrier in between. We therefore view [1] as complementary to our work since [1] characterizes diversity in the input-output function around a solution, while HNC characterizes diversity in internal representations when the input-output function is held fixed.
  • [2] identifies mechanisms through model behavior, namely which input attributes a model's predictions are invariant to, rather than through internal representations, and does not analyze the representations or activations of the resulting solutions.
  • [4] keeps the network's outputs on earlier-task inputs approximately fixed so that it can learn a new task with less forgetting. Its goal is to change some outputs while protecting others, and it does not characterize the internal structure of the resulting solutions.
HNC (ours)[1] Neural Thickets[2] Mechanistic Mode Connectivity[4] OGD
Explicitly preserves the network's input-output functionYesNoNoOn earlier task inputs only
Analyzes internal activations or representations of the resulting solutionsYesNoNoNo
How alternative solutions are obtainedPost trainingPost trainingTraining or fine-tuningDuring training
AuthorsOpenReview ↗

Response 2/3: Comparison with lower-order methods; clarifying function matching in RL

W2+Q2: How does HNC compare to low-order methods? Can those methods find similar solutions in a local region? In particular, [4] uses the Jacobian null space which is equivalent to the null space of the Gauss-Newton matrix used by HNC. I do not think you need a robust comparison for every result, but some comparison should be made.

We have added such a comparison to lower-order methods to Appendix G. Starting from the same anchors, we compared HNC with random perturbation [1], with orthogonal gradient descent [4], and with task training plus a penalty on similarity to the anchor. We also compared it with low-loss curves that connect the anchor to an independently trained network [Garipov et al. 2018]. For each method, we measured how far the representation moves from the anchor while the function on held-out inputs stays preserved.

Lower-order methods do find some alternative solutions near the anchor, but HNC changes representations the most at matched function preservation. On the 3BFF RNN, HNC moves farthest at every level of function preservation, whether steered by CKA or DSA. On ViT-S, HNC reaches over twice the representational divergence of OGD when 10% of the network's predictions are allowed to change. These gains come with more compute per step, since HNC is about ten times slower than OGD on the RNN and five times slower on ViT-S. However, HNC uses much less (about 1/3) GPU memory than OGD on ViT-S. OGD must store a gradient for every constrained output, so it soon runs out of memory on larger models. In the table, we briefly summarize the representational divergence for different methods at a fixed level of function-matching (1e-3 function loss for RNNs, 10% prediction change for ViT). We have added the full experimental setup, figures of representational divergence against function loss, and the run time and memory of each method to Appendix G.

MethodRNN, Δ\DeltaCKARNN, Δ\DeltaDSAViT-S, Δ\DeltaCKA
HNC0.83 ±\pm 0.020.44 ±\pm 0.120.48 ±\pm 0.12
OGD [4]0.04 ±\pm 0.010.18 ±\pm 0.040.22 ±\pm 0.01
OGD [4] + restore step0.15 ±\pm 0.020.19 ±\pm 0.040.00 ±\pm 0.00
Similarity penalty0.02 ±\pm 0.000.25 ±\pm 0.00n/a
Low-loss curve to another seed0.12 ±\pm 0.010.19 ±\pm 0.04n/a
Different seed (reference)0.11 ±\pm 0.010.06 ±\pm 0.00n/a

 

W3. "Isn't searching for different behavioral policies under the same reward like a task-matching loss? To me, a function-matching loss would be producing the same state-action trajectories but the policy network's internal mechanisms changing."

Thanks for pointing it out! Rather than a strict dichotomy between function matching and task matching, we find it helpful to view both settings as preserving the network's function at different resolutions. Preserving the full input-output mapping allows us to isolate the diversity in internal representations without it being confounded by diversity in the input-output mapping. Meanwhile, underspecification in RL presents a unique opportunity to use HNC to find behaviorally distinct policies, including reward-hacking ones, which is of high practical value. In the main-text RL experiments, we therefore deliberately preserve a coarser readout of the policy's function, which is the return.

In Appendix M.8, we explicitly preserve the state-action mapping for ten PPO policies trained on HalfCheetah, and found that during HNC, the hidden representations of the policy network becomes increasingly dissimilar to the original ones (Δ\DeltaCKA = 0.15 ±\pm 0.04 at the end of the walk, mean ±\pm s.e.m.), while keeping the return at the anchor level.

In the updated paper, we explicitly motivated this choice with the updated text (lines 361-365): "This underspecification can admit many behaviorally distinct policies with similar reward level, even unintended strategies that exploit the reward function rather than accomplish the intended task, a phenomenon known as reward hacking. We therefore preserve the return of a policy rather than its actions, and use HNC to search for behaviorally distinct policies with comparable return. When we instead preserve the actions, as in the supervised setting, HNC changes the hidden representation of a policy while keeping its return (Appendix M.8)."

AuthorsOpenReview ↗

Response 3/3: memory mechanisms of the 3BFF networks

Q1. The undirected case seems to be a noisy version of the trained network. What are the 7 and 15 FP solutions doing? Basically, I don't think there are multiple distinct ways to memorize random values. For example, a network could traverse through 2 FP for every logical state and then it is a "16 FP solution." But is that really a different mechanism?

Thanks for the question. We agree that fixed-point counts alone do not establish distinct mechanisms. Our distinction concerns how each network maintains a memory and updates its hidden state over time in the absence of input: the dynamics of RNN hidden activities implementing the memory. We therefore distinguish between memory maintained by fixed points, slow transients, or periodic activity, rather than simply by different numbers of fixed points. Memory held in sustained activity versus memory held at a resting point is the standard distinction between dynamic and persistent coding in the working-memory literature, and it is how the RNN reverse-engineering literature defines a mechanism [Sussillo & Barak 2013; Turner, Dabholkar & Barak 2021; Driscoll, Shenoy & Sussillo 2024].

To test whether the undirected network is simply a noisy version of the anchor, we added Gaussian noise to the anchor's pre-activations at every time step, tuning its scale to match the undirected network's steady-state hidden speed. The noisy control had twice the task loss of the undirected network. After 3000 steps without input, its mean output drift was 30%, compared with 5.7% for the undirected network, and its accuracy was 27%, compared with 99.3%. Thus, adding noise does not reproduce the undirected network's ability to maintain memory despite ongoing hidden-state motion. Our dynamical analysis shows that this motion occurs in the readout-null space, allowing hidden activity to change while the output remains stable.

For the CKA-steered network with 7 stable fixed points, seven memory states each have a stable fixed point. The eighth has no stable fixed point and instead maintains memory through a slow transient in the readout-null space, as shown in Appendix J.11.

For the DSA-steered network with 15 stable fixed points, the hidden state does not settle at these fixed points during the task. Five of the eight memories are maintained by limit cycles: after 3000 steps without input, the state continues along a periodic orbit, with 99% of its motion in the readout-null space. Several fixed points are never visited, so their total number does not adequately describe how this network maintains memory. We now describe the dynamics of these two networks in the main text (lines 255-257): "Meanwhile, the CKA-steered network holds its memories in a mix of fixed points and a slowly decaying trajectory, while the DSA-steered network implements most of its memories as limit cycles in the readout-null space (Appendix J.11)."

 

We thank you again for your thoughtful questions, which have helped us improve the quality of the paper!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.