acceptodds
Under review as a conference paper at ICLR 2027

Certifying Factual Information Surplus in Noisy Transformer States

Abstract

How is information about a learned fact distributed across the parameter groups of a transformer? We study this question using small transformers trained on synthetic data. Within each experimental repetition, models are trained on all possible assignments of a fixed set of binary facts, with shared initialization and data-order randomness across assignments. We model observations of saved parameter groups by adding independent Gaussian noise, which specifies measurement precision. We derive a computable lower bound on joint information surplus: the mutual information about an individual fact in a pair of groups observed together, minus the sum of the mutual informations in their separate observations. The method combines entropies of Gaussian distributions matched to the means and covariances of the observed states with analytic Rényi divergence corrections. We establish a sufficient condition, expressed through covariances between parameter groups, for detecting positive surplus when differences between trained states are small relative to observation noise. Evaluation using Gram matrices of parameter states and rigorous interval arithmetic certifies positivity without Monte Carlo integration. In a confirmatory experiment with a protocol fixed in advance, the method establishes small positive lower bounds across independent training repetitions, whereas the tested pairwise entropy and transport bounds do not certify positivity on the same data. Thus, for the studied source and noise level, jointly observing embeddings and the output group comprising final normalization and the answer head provides more information about an individual fact than the sum available from their separate observations. This result shows that characterizing distributed factual information at a specified observation precision requires accounting for dependencies between parameter groups.

Primary area:
learning theory
Keywords:
transformers, factual knowledge, mutual information, information synergy, Gaussian mixtures, information-theoretic bounds

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_jkozOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I found the paper extremely difficult to read and could not understand most of the parts of the main text. Symbols and technical terms are repeatedly used before they are explained, leaving the reader to piece together their meaning from later sections. Below I report a couple of examples. This is a review of the writing, not a judgment about the validity of the mathematics. I think the paper needs a substantial rewrite to make the setup and argument understandable before judging its technical quality.

Critical strengths

See questions

Critical weaknesses

See questions

Questions for authors

A few examples of lines I was not able to follow from the main text:

  • Lines 43-44: "We formalize this question using a controlled finite distribution over training-label assignments and an explicit Gaussian observation channel." What is sampled? What does the observer receive?

  • Lines 46-47: "The resulting eight states define a finite source distribution, allowing us to condition on one target association while averaging over the other two." What are these "states"? What exactly is conditioned on? Why eight?

  • Lines 53-54: "We derive a"lower bound BF≤ΔFB_F \leq \Delta_F using moment-matched Gaussian entropies and analytic Rényi corrections." What are ΔF,BF\Delta_F, B_F, and FF? What are moment-matched Gaussian entropies? What is an analytic Rényi correction?

  • Lines 53-55: "For a fixed finite source at low signal-to-noise ratio (SNR) tt, the gap satisfies 0≤ΔF−BF=O(t3)0 \leq \Delta_F-B_F=O(t^3), while the leading interaction can be of order t2t^2." What is tt? What is the “leading interaction”? What is the intended message of this sentence?

  • Lines 79-82: "The signed difference between joint and separate information is negative co-information. In a nonnegative bivariate partial information decomposition (PID), it equals synergy minus redundancy" What do co-information, PID, synergy, and redundancy mean here?

  • Lines 364-365: "Alternate-format queries yield 68/480 correct answers at initialization and 473/480 at step 24 (14.17% and 98.54%)." What are "alternate-format queries"? What does 480 count?

  • Lines 380-381: "The transport/Fano composite tightens single-group upper bounds by pairing opposite-class components and joint lower bounds by classification overlap." What is the "transport/Fano composite"? How does it produce these bounds?

  • Lines 452-454: "The ordering reverses for synthetic parity at amplitude 2 with fixed unit noise: Taylor gives 0.08428519 bit, while our bound gives −0.84306302, despite positive true surplus." What is the synthetic parity setup? Amplitude of what? Why unit noise?

  • Lines 456-458: "On a calibration replicate, the conditional-product control preserves both single-group endpoints and gives BF≈−0.00045801B_F \approx -0.00045801 bit, consistent with its nonpositive true surplus." Endpoints of what? What exactly does the control preserve?

Reviewer_DgS6OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors investigates how information is stored jointly across transformer model parameters. Specifically, they derive a lower bound on the extra information about a fact available from looking at two parameter groups together (relative to sum of their individual contributions), when Gaussian noise is added to these parameter values. The paper details how the bound behaves as the noise increases. On experiments with small transformers (around 15,000 parameters) and eight possible answer assignments, the authors analyze various parameter groups, including the embeddings, and their corresponding joint information bound; this bound is positive in all twenty of their runs, while existing bounds compared with are inconclusive.

Critical strengths

The overarching questions of where and how factual information is stored across model parameters is interesting, and whether the storage is synergistic, and to what extent. This has broader implications for mechanistic understanding of transformers. The paper establishes that the joint effect can exist, atleast for the synthetic setting and noise model tested.

Critical weaknesses

The training and evaluation settings are very small-scale (~15,000 parameter transformer, three questions, two possible answers per question). Additionally, the conditions of the analysis are limited to Gaussian noise for observing parameter groups. Given that the paper is relying on synthetic experiments, I recommend showing that these results are consistent across significantly larger-scale and varied settings. Currently, the results support a narrow claim about joint information when Gaussian noise is added to parameter values and not some overarching insight, e.g., whether the additional joint information contributes to factual recall or affects the editability of stored facts. On this note, it would be useful to explicate what the practical implications are of the information-theoretic bound.

The prose is hard to follow, as well as the logical progression of ideas. It is difficult to identify what the explicit contributions are; the paper would be stronger with a dedicated contributions sections, as well as reforming the conclusion to focus more on broader impact and less technical details. I recommend revising certain terminology ("outward-rounded interval arithmetic") that is overly complicated.

Questions for authors

Are there broader implications of the information-theoretic bound, e.g., for larger model effects, factual recall, editability of stored facts? How should general practitioners interpret the bound?

In particular what does the setting (artificial loss of precision in parameter values via the Gaussian noise model) tell us about how a transformer stores or uses facts? Also how sensitive are the conclusions to noise model choice?

Reviewer_XXREOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a "fact-specific information surplus" between two predefined parameter groups of a small transformer. It is defined as ∆_F = I(F;Y_i,Y_j) − I(F;Y_i) − I(F;Y_j) (Eq. 5), which is negative co-information. Here Y_i is the parameter group observed through an analyst-chosen Gaussian channel Y_i = m_i(U) + N_i with N_i ~ N(0, ρS_i² I) (Eq. 3). S_i² is the spread of the group's own eight trained states (Eq. 2).

The source works as follows. A one-block transformer (14,784 parameters) is trained for 24 AdamW updates on three binary query–label associations. Each replicate trains all 2³ = 8 label assignments with shared initialization and data order. This gives an 8-state uniform source U, and the target is F = U_3 − 1 (Sec. 3, Sec. 7).

Contributions:

  • Lower bound (Prop. 1, Eq. 12). B_F ≤ ∆_F. Each mixture entropy is written as a moment-matched Gaussian entropy minus a KL deficit (Eq. 9). Only the deficits that enter with a negative sign are bounded, using the order-2 Rényi divergence (Eq. 10). That divergence has a closed form in terms of Gram matrices (Eq. 18–19, App. B).
  • Low-SNR analysis (Lemma 1, Prop. 2, App. C). Both ∆_F and B_F equal t²Γ_F/(2 ln 2) + O(t³), with t = 1/ρ, and their gap is O(t³). Γ_F (Eq. 16) compares the squared within-class cross-covariance with the unconditional cross-covariance.
  • Exact certification (Sec. 6, App. E). FP32 inputs are treated as exact dyadic numbers. Determinants and inverses are computed in exact rational arithmetic, and exp/log use outward enclosures.
Critical strengths
  1. A correct certificate that keeps the second-order interaction term. The construction is: moment matching (Eq. 9); bounding only the subtracted deficits by D_2 (Eq. 10); the closed-form Gram expression (Eq. 18); exact rational arithmetic (App. E). Together these give a lower bound on negative co-information whose sign can be trusted for the stated channel. The matrix dimension depends on the number of source states, not the number of parameters; extracting the Gram matrices is linear in d, at O(dn²) (Sec. 6, p. 7). Sec. 8 shows why KT bounds have first-order slack that hides an O(t²) surplus. This is the paper's main technical contribution, and it is sound.
  2. A clean confirmatory protocol with full reporting. The pair, target, ρ, checkpoint and threshold were fixed before the 20 new seeds were drawn. Seeds were allocated once with no optional stopping (Sec. 7, App. H.1). The statistical unit is the replicate, and the exact Clopper–Pearson criterion is stated in advance (Eq. 20). All 20 outcomes are reported, including the replicate (12) that misses the threshold (Table 6).
Critical weaknesses

What the positive sign means is unclear, and the paper does not show what drives it.

The noise scale is set by the signal itself. In Eq. 2–3 the noise variance is ρ times the spread of the group's own eight states. The paper's "low SNR" (p. 6) therefore does not correspond to a physical measurement precision. The paper notes that equal ρ is not equal precision across dimensions (Sec. 3, p. 3), and states in Sec. 9 (p. 9): "Exact parameter access or a different metric can change the sign." If each noiseless group determines F, the noiseless surplus is −H(F) (Sec. 3, p. 4). That holds generically when the eight states are distinct. The positive sign is therefore a property of heavy observation noise.

Tightness is never measured. The 8-component mixtures lie in an affine span of dimension at most 7 (App. D.3 uses k_P = 7). The true ∆_F, I(F;Y_E) and I(F;Y_H) could therefore be estimated cheaply, by Monte Carlo with error bars or by quadrature. The paper already uses quadrature as a diagnostic in 1–2 dimensions (App. E.4). None of these values is reported. As a result:

  • there is no strong non-certified reference against which to judge how loose B_F is;
  • there is no way to tell whether the surplus is non-negligible relative to the separate informations.

The certified comparisons do not fill this gap. The paper explains the KT failure itself (Sec. 8). For the transport/Fano composite it gives only the Table 1 results (0 of 20 replicates certified; lower endpoints −0.0748955 to −0.0592105 bit, p. 8) and the remark that it "still leaves a negative surplus bound" (p. 8). Given that the ingredients are named by the authors (pp. 2, 6), the methodological novelty is careful but incremental. That makes the missing empirical context more important.

Questions for authors

see above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.