acceptodds
Withdrawn from ICLR 2027

Exact Functional Solution Space: Auditing Neural Analyses Across Exact-Equivalent Parameterizations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural-network analyses are typically computed from a particular parameterization but often interpreted as properties of the learned solution. This interpretation is problematic because different parameterizations can realize exactly the same input-output function while changing internal quantities and the outcomes of downstream procedures. We address this problem using Exact Functional Solution Space (EFSS), which identifies a neural solution with its realized function rather than with a particular parameter vector. This yields a simple criterion: a solution-level claim should remain consistent across exact-equivalent parameterizations under the appropriate semantics of its output. We turn this criterion into an output-aware audit that distinguishes direct invariance, correspondence under relabeling or transport, and consistency of model-producing procedures. Across Transformer interventions, pruning, and model editing, we find that similar task performance can conceal representation-dependent scientific judgments, while unstable internal measurements can leave reported decisions unchanged. These findings show that consistency must be assessed at the level of the claim being made. EFSS makes exact functional equivalence a practical basis for tracing such dependence through an analysis and guiding targeted repairs under the tested transformations. By separating the consistency of scientific conclusions from task utility, the framework helps researchers assess what their analyses establish about a learned function.

Primary area:
interpretability and explainable AI
Keywords:
functional equivalence, neural network symmetries, representation dependence, invariance and equivariance, interpretability, model pruning, model editing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_AWH2OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that analyses like pruning scores or edit-acceptance checks should give the same answer on two networks that compute exactly the same function. It provides a formal framework and then it tests this by applying function-preserving changes to the weights (permutations, rescaling) and rerunning the analyses.

Critical strengths

S1. The motivating framework is sensible. Asking which claims based on previous techniques (pruning scores, importance rankings, etc.) are invariant to a function-preserving reparameterization is an interesting question in itself, and the paper's framing is not without interest. For instance, a rescaling that preserves the function can still change which layers get pruned (Fig. 2) or whether an edit is accepted (Table 1), while the final quality stays almost the same. These results should have been the main ones of the paper.

Critical weaknesses

W1. What the framework presented serves for is unclear. The full fibers and quotients are never computed (to be fair, acknowledged at l.485), as it only uses some specific transformations and permutations. Moreover, the results (Prop 1 and 2) are elementary and do not add much. Finally, it is not clear how it helps with knowing which claims should hold at the function level. The conclusion does not help much with this -- it just says again what the framework is.

W2. The experiments and the guidelines in 4.6 do not add much insight. Figure 1(a) is a sanity check not for experiments, and the NLL differences (the most interesting results) are small. Importantly, the guidelines in 4.6 are generic: there is no way to know how they result from the framework and the experimental insights. They should at the very least be linked to some quantitative results.

W3. Much of the information important for the paper is hidden in the appendix, so a reading of the first 9 pages is not enough. For instance, "selector" and "intervention gap" are used in Section 4.2 without being defined in the main text.

Questions for authors

Q1. How can one find a certified transformation on a new architecture without listing all possible transformations?

Q2. What is the decision procedure to know whether a claim should hold for a function only or for a given representation? (take the example of explaining the architecture itself which depends on the representation)

Reviewer_Zw9kOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper formalises the idea that neural networks with different parameters but identical input–output functions represent the same solution (sec 2), then proposes an “audit” (sec 3) of whether analyses and procedures give consistent results across such equivalent representations. The paper's experimental section (sec 4) tests whether analyses and procedures such as neuron selection, pruning, and model editing produce consistent results when applied to different parameterisations of a neural network that compute exactly the same function.

Critical strengths

None

Critical weaknesses

The theoretical part of the paper (up to and including section 3) is completely tautological, stating completely elementary mathematical facts that everyone is aware of, but that are of such broad generality that they are essentially useless to investigate concrete properties neural networks. Propositions 1, 2 are not propositions, what you state is definitionally true, there is nothing to prove there.

The empirical part of the paper, in contrast, focuses on super-specific situations but without really explaining what they have to do with the elementary theoretical discussion that precedes it. Not one of the labelled equations in sec 2, 3 is mentioned in sec 4, which illustrates this disconnect.

Vocabulary: I find the choice of wording really problematic. What is a "certified" permutation? Are any permutations "uncertified"? This sounds like nonsense. Similarly, the words "scientific" and "audit" are used in a very unspecific way and could mean a whole range of things. This vagueness makes them essentially meaningless. What is a "labelled or structured output" (l216)? It is completely vague, and as a consequence I cannot see any mathematically meaningful difference between (16) and (17). Making the notion of group action explicit might perhaps have helped, but it's not clear.

The self-reported usage of AI is worryingly extensive.

Questions for authors

In which way does sec 4 illustrate any of the ideas of sec 2, 3? And in which way are sec 2, 3 essential to make these points?

Reviewer_kz29OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Exact Functional Solution Space (EFSS), a framework for auditing whether neural network analyses produce consistent conclusions across functionally equivalent parameterizations. The authors construct equivalent networks using permutations and reciprocal scaling, and evaluate neuron importance, pruning, and model editing. Experiments on GPT-2, Qwen2.5, and TinyLlama show that identical functions can lead to different pruning decisions, downstream predictions, and model-editing outcomes. They also investigate transformation-aware methods to improve consistency.

Critical strengths

Important research problem. The paper addresses whether neural network analyses reflect properties of the learned function or artifacts of a particular parameterization.

Unified framework. EFSS provides a systematic approach to evaluating consistency across intermediate measurements, downstream decisions, and model-producing procedures.

Interesting empirical findings. Experiments demonstrate that functionally equivalent models can produce different pruning allocations and editing decisions, even when downstream performance remains similar.

Practical insights. The authors investigate transformation-aware repairs that improve consistency across equivalent representations.

Critical weaknesses

Limited novelty. The main ideas overlap with existing work on Implementation Invariance (Sundararajan et al., 2017) and reparameterization-aware pruning (Asiaee, 2026). The quotient-space formulation also builds on established neural network symmetry theory. The methodological novelty beyond these works is not sufficiently clear.

Limited reparameterization diversity. Experiments primarily focus on permutations and reciprocal scaling. An important missing case is Query–Key reparameterization in Transformer attention, which preserves attention logits under appropriate invertible transformations. Evaluating this case would better support the framework's generality.

Unclear necessity of invariance. Some algorithms, such as magnitude pruning, are inherently parameter-dependent. Different outcomes across equivalent representations do not necessarily indicate a methodological flaw.

Limited practical benefits. Improved consistency does not necessarily translate into better downstream task performance, leaving the practical value of the proposed repairs somewhat unclear.

Questions for authors

What are the main methodological contributions of EFSS beyond Implementation Invariance (Sundararajan et al., 2017) and Causal Mechanism Reduction (Asiaee, 2026)?

Could the authors evaluate Query–Key reparameterization in Transformer attention, for example (Q'=QA,\ K'=KA^{-\top}), to demonstrate generality beyond permutations and reciprocal scaling?

How should one determine whether representation invariance is necessary for a particular algorithm, especially for inherently parameter-dependent procedures such as magnitude pruning?

Can the authors provide stronger evidence that improving representation consistency leads to meaningful practical benefits beyond stable intermediate measurements and decisions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.