acceptodds
Under review as a conference paper at ICLR 2027

Local Substitutions Do Not Compose

Abstract

A model can be nearly insensitive to individual identity substitutions while still depending on the correspondence among many inputs. When those identities are forgotten jointly, the resulting behavioral loss need not remain small. We formalize this distinction through behavioral quotients, which identify concrete states under declared transformations and measure the irreducible prediction error induced by that abstraction. Under uniform coalition queries, we derive sharp bounds on block quotient error from singleton contextual substitutions and finite query certificates that remain valid after the quotient is selected from the same measurements. We then show a fundamental failure of composing individually safe substitutions: every singleton discrepancy can vanish as the number of bound objects grows, while any accurate block quotient still requires states at a fixed positive error tolerance. Coordinated renaming that preserves the bindings instead gives an exact quotient with states. Controlled Transformer experiments reproduce the corresponding finite-size separation in learned responses. Finally, we decompose shared prediction error across aligned instances into residual identity dependence, cross-instance disagreement, and fitting error, with direct diagnostics for the first two terms.

Primary area:
learning theory
Keywords:
learning theory, behavioral abstraction, contextual interventions, relational binding, symmetry, model interpretability, representation reuse

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NUM5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates an interesting problem: a model can be nearly insensitive to individual identity substitutions while still depending on the correspondence among multiple inputs. To study this phenomenon, the authors introduce behavioral quotients to formalize identity abstraction and quantify the information lost when multiple identity distinctions are removed jointly. Through theoretical analysis, the authors establish bounds connecting local substitution discrepancies to global quotient errors and demonstrate a complexity separation between independent block quotients and binding-preserving coordinated quotients. The findings are further supported by controlled Transformer experiments. And the paper provides useful insights into how relational dependencies can be distributed across many weak local effects, highlighting the importance of evaluating joint abstractions rather than relying solely on individual substitutions.

Critical strengths
  1. This paper is well-written and clear. All the mathematical derivations are rigorous. It establishes explicit bounds between local substitution discrepancies and global quotient errors, provides finite-sample certification guarantees, and carefully derives the main theoretical results.

  2. The paper investigates an interesting and meaningful research problem. It highlights an important distinction between local identity substitutability and joint abstraction, showing that small changes under individual substitutions do not necessarily imply that multiple identity distinctions can be safely removed together. This provides a useful perspective for understanding relational dependencies and entity binding.

Critical weaknesses
  1. Although Theorem C establishes a rigorous and interesting complexity separation between independent block quotients and binding-preserving coordinated quotients, its broader implications for relational abstraction in practical models could be further discussed.
Questions for authors
  1. There are several cross-reference formatting issues throughout the manuscript. For example, Line 178 has a typo: "Equation equation 9," and more cases occur in the Appendix. Please modify it.

  2. I have mentioned this as Weakness 1. I appreciate that the authors explicitly define the scope of their theoretical results and acknowledge the challenges of extending them to pretrained LLMs. However, a more concrete discussion of how these quotient families and transformation assumptions might correspond to entity binding or relational reasoning in LLMs would strengthen the practical relevance of the theoretical findings. While additional formal results or LLM experiments may be beyond the scope of this paper, clarifying these connections would provide useful guidance for future research.

  3. I think the theoretical novelty of Theorem A could be better distinguished from existing work. For example, Filmus [1] has already studied functions over fixed-Hamming-weight slices of the Boolean hypercube and their associated spectral properties, which are closely related to the derivation of Theorem A. Although the authors cite this work in the Related Work section, I suggest providing a more explicit discussion and relevant citations alongside the theorem and its derivation to better clarify which results follow from existing theory and what constitutes the new contribution of this paper.

Reference

[1] Filmus, Y. (2016). An orthogonal basis for functions over a slice of the Boolean hypercube. The Electronic Journal of Combinatorics, 23(1), P1.23.

3 comments

Dear Reviewer NUM5,

We would like to respectfully offer a reminder and invite you to join the further discussion. We believe that most concerns have been reasonably addressed or substantially resolved. If you have a different view, we can give further clarification. Alternatively, we would also welcome you to raise the score or adjust the confidence.

We would also like to share important updates that may change your view of the paper. Following your suggestions, we have strengthened the discussion of practical implications and clarified our theoretical contributions.

  • Practical significance of relational quotients. Theorem C establishes that concrete identities can be discarded while their relational bindings must be preserved. Individually safe substitutions can require exponentially many states when combined, whereas binding-preserving quotients require only polynomially many states.
  • Connection to LLM relational reasoning. We added a concrete entity-binding example showing how coordinated renaming preserves relationships across mentions, while independent substitutions can disrupt coreference and relational behavior.
  • Clarification of theoretical novelty. We distinguish the established spectral foundations from our contributions, including sharp bounds connecting local substitution measurements to joint quotient error and finite-query certificates that remain valid after partition selection.

We have also corrected the cross-reference issues and expanded the discussion of related work.

We look forward to your reply and will be happy to answer any further questions.

Reviewer_NUM5OpenReview ↗

Thank you for response

Dear authors,

Thank you so much for the thoughtful response. I think extending this work to large language models as future work will be interesting. I will maintain my score.

One small suggestion: in your revised manuscript, please try to use a different color to highlight the additions or modifications made during the rebuttal phase, so that it is more convenient for our discussion.

Thank you.

Dear Reviewer NUM5,

Thank you again for your suggestion and positive assessment of our work. We appreciate your recognition of the paper's contributions.

Given your overall positive evaluation and the clarifications provided during the discussion, we would greatly appreciate it if you could reconsider whether the paper now merits a Clear Accept (4). If you still have any concerns preventing a higher rating, we would be happy to address them.

Thank you very much for your time and consideration.

Reviewer_GwwKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies when identities that appear individually interchangeable can be forgotten jointly without losing a model's behavior. It defines behavioral quotients under declared transformations and measures their error for a fixed scalar response and query distribution. Under uniform coalition queries, the authors derive sharp bounds relating singleton substitution discrepancies to block quotient error, together with finite sample certificates that remain valid after partition selection.

The central result constructs a bounded response whose singleton discrepancies decrease as O(1/r)O(1/r), while accurate block quotients require exponentially many states at a fixed positive error tolerance. Coordinated renaming that preserves the underlying bindings instead permits an exact quotient with polynomially many states. Five training seeds of a small Transformer illustrate this separation. Exhaustive evaluation of all 115,975115{,}975 partitions of ten slots finds a minimum feasible block state count of 243243, while the declared coordinated quotient has 5656 states and negligible error. The paper also decomposes prediction error across aligned instances into residual identity dependence, disagreement between instances, and estimation error.

Critical strengths
  1. The main separation establishes a meaningful limitation of local substitution tests. Theorem C operates at a fixed positive error tolerance: singleton discrepancies vanish as the number of objects increases, while accurate block quotients still require exponentially many states. Coordinated transformations preserve the bindings and avoid this cost. Proposition 1 explains the mechanism through an explicit expression for the signal retained by an arbitrary partition. This is the strongest contribution, because it identifies a concrete reason why individually acceptable substitutions need not justify a collective abstraction.

  2. The paper provides constructive certificates alongside the separation. The bounds E(Π)≤J(Π)≤κΠE(Π)E(\Pi) \le J(\Pi) \le \kappa_\Pi E(\Pi) connect local measurements to the error of a complete proposed quotient, and the simultaneous concentration argument permits partition selection using the same pairwise measurements. Within the controlled setting, the validation is thorough: the authors enumerate every partition, report five training seeds, and compare the true pairing against all 945945 perfect pairings. These checks make the finite size example more convincing than a comparison against a few selected alternatives.

Critical weaknesses
  1. The learned experiment cannot test the theory independently, and the empirical headline is narrower than presented. The training target is mapped by Eq. 35 exactly onto the analytic response vrv_r, and all five models reach ∥vθ−vr∥2=(3.34±1.03)×10−4\|v_\theta - v_r\|_2 = (3.34 \pm 1.03) \times 10^{-4}. By the paper's own stability bounds (Eqs. 32 and 33), any response this close to vrv_r must inherit its quotient structure, so the experiment could not have produced a different outcome once the model fit the target. The input also supplies object identity embeddings shared by the bound fields, so the model is given the binding rather than required to discover it. The abstract describes this as the corresponding finite size separation in learned responses, but at r=5r = 5 Theorem C guarantees only K≥2c0r≈13K \ge 2^{c_0 r} \approx 13; the reported 243243 comes from exhaustive search. That value also holds only in a narrow tolerance window: all isolated merges are feasible only when ε≥1/320\varepsilon \ge 1/320, and the minimum stays at 243243 only while ε<1/240\varepsilon < 1/240 (at ε=0.0042\varepsilon = 0.0042 it falls to 135135). On the other side, adding within object swaps fi↔rif_i \leftrightarrow r_i to the coordinated group gives (r+22)=21\binom{r+2}{2} = 21 orbits on which vrv_r is constant, so the 5656 state count reflects the declared group rather than a minimum. No learned result is reported for any rr other than 55. The empirical evidence consists of a single synthetic task at one problem size (r=5r = 5, 1,0241{,}024 states), with no pretrained models, natural inputs, or baselines, and Section 7 has no experiment; Appendix G itself marks results on pretrained language models as not yet established. The experiment therefore illustrates the construction but provides no independent evidence that the phenomenon arises, or can be detected, in learned models.

  2. The significance and practical reach are limited relative to the paper's framing. The technical tools are established: the Johnson slice spectrum, Hoeffding with a union bound over pairs, orbit averaging as conditional expectation, and the law of total variance for Theorem D. The paper does not engage the approximate state abstraction literature, where quotients with explicit error bounds are the central object. The title also overstates the result relative to the paper's own Theorem A: aggregated local measurements do compose, since J(Π)J(\Pi) upper bounds the joint error; what fails is reasoning from the largest single discrepancy. The exponential separation holds between block exchange and one hand chosen coordinated group, and Appendix D.1 states that the framework does not discover such a group, so a practitioner auditing a real model is left without the step the result makes necessary. Every result assumes uniform coalition queries over a response defined on 2N2^N, which for language models requires a masking protocol that Appendix D.6 concedes need not transfer to natural inputs. The finite query certificate needs about 95,94395{,}943 sampled contexts per pair to certify the paper's own 1,0241{,}024 state example at ε=1/256\varepsilon = 1/256, and the 400400 repetition audit in Appendix F.5 is not reported in enough detail to assess it. For a top venue, the paper needs either a new technical tool or evidence that the phenomenon arises in models where the binding is not supplied as input.

Questions for authors
  1. Clarification of the existing finite query audit: Please report the response functions, problem sizes, sample sizes, confidence levels, and partition selection procedure used in the 400400 repetitions mentioned in Appendix F.5. What were the observed coverage and the fraction of runs that certified a nontrivial quotient at the target tolerance? Please distinguish sampled comparisons from distinct model evaluations when reporting the budget. If the existing results permit it, a comparison with Appendix D.4 would clarify the practical advantage of the local certificate.

  2. Clarification of the finite state learning bound: Please define the lookup estimator explicitly in Section 7 or Appendix D.8, including its prediction when Nt=0N_t = 0. The sample mean on observed states, with prediction 1/21/2 on unseen states, appears to give the stated bound. Its conditional estimation error is at most 1/(4max⁡{1,Nt})1/(4\max\{1, N_t\}), which is bounded by both 1/41/4 and 1/[2(Nt+1)]1/[2(N_t + 1)], and Equation (113) then gives the claimed estimation term after averaging over abstract states. Is this the intended estimator? The present cross references leave the definition unstated.

  3. Clarification of the relationship to earlier abstraction results: Please discuss the connection to Li, Walsh and Littman, "Towards a Unified Theory of State Abstraction for MDPs" (ISAIM 2006), and Abel, Hershkowitz and Littman, "Near Optimal Behavior via Approximate State Abstraction" (ICML 2016), which study what an abstraction preserves and how approximation affects behavior. Chen, Dobriban and Lee, "A Group-Theoretic Framework for Data Augmentation" (JMLR 2020), is also directly relevant to orbit averaging as a conditional expectation (Theorem 2). A short comparison would help distinguish these established foundations from the substitution certificates and relational separation developed here.

  4. Validation request: Can you rerun the experiment without object identity embeddings, so that the binding must be inferred from position or content? Does a block quotient still fail while the coordinated quotient succeeds?

  5. Validation request: Can you report learned results at r=6r = 6, where all partitions can still be enumerated, or apply the certificate of Theorem B at larger rr, to show that the separation grows in learned responses?

3 comments

Dear Reviewer GwwK,

We would like to respectfully offer a reminder and invite you to join the further discussion. We believe that most concerns have been reasonably addressed or substantially resolved. If you have a different view, we can give further clarification. Alternatively, we would also welcome you to raise the score or adjust the confidence.

We would also like to share important new experimental results that may change your view of the paper. To address your concerns about the learned experiments and practical significance, we conducted additional validation without object identity embeddings and across multiple problem sizes.

  • Separation without explicit identity embeddings. All five No-ID Transformers achieved 100% held-out accuracy. Exhaustive evaluation confirmed that the minimum feasible block quotient requires 243 states, compared with only 56 states for the binding-preserving quotient, with MSE of 9.29 × 10⁻⁹.
  • Consistent separation across problem sizes. Across 15 trained models at r = 4, 5, 6, all achieved 100% held-out accuracy. The minimum block state counts were 81, 243, and 405, compared with only 35, 56, and 84 binding-preserving states, respectively.
  • Validated finite-query certificates. All 400 audit repetitions satisfied simultaneous confidence-interval coverage, and all 100 runs at the largest sampling budget certified the minimum 9-state quotient.

We have also clarified the theoretical contributions, their relationship to prior abstraction literature, and the finite-state learning bound.

Full experimental details will be included in the revision. We look forward to your reply and will be happy to answer any further questions.

Dear Reviewer GwwK,

The rebuttal period ends in less than 24 hours, so we wanted to follow up. We believe our responses and new experiments have addressed most of your concerns. We would really appreciate it if you could take another look, share your thoughts, or reconsider your score.

Thank you again for your time and consideration.

Thank you for the clarification. I appreciate the distinction between the observed gains and their possible explanations. The proposed replication is a useful next step toward understanding how much the findings depend on the particular scoring and evaluation setup. With that question still open, I will maintain my current assessment. I encourage the authors to pursue this validation, which would strengthen the contribution and help others build on these observations.

Reviewer_yff6OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether language-model personas can be decomposed into sparse combinations of more primitive personality-trait directions. The authors construct a dictionary of 60 traits and extract activation directions for 37 personas across three models. They fit each persona direction using a small set of selected traits, termed a “trait recipe,” finding a median of roughly six traits that captures about half of the persona direction.

The paper then tests whether these decompositions are behaviorally meaningful through activation steering. Adding trait recipes increases recognition of the corresponding persona, subtracting them from prompted personas reduces recognition, and exchanging recipe/core components shifts behavior toward target personas. The authors further apply the framework to safety-relevant behaviors, showing that a sycophancy recipe modulates false-claim endorsement and capitulation, while a ten-trait reconstruction of the refusal direction can substantially reduce refusal and increase harmful assistance. Overall, the work provides evidence that sparse trait mixtures capture a causally active, though incomplete, component of persona representations.

Critical strengths
  1. The paper provides a compelling compositional perspective on persona representations. Rather than treating each persona direction as an indivisible feature, it identifies sparse mixtures of reusable trait directions and demonstrates substantial consistency across three models. This is a useful conceptual and empirical step toward understanding the internal structure of persona representations.

  2. The causal intervention experiments are broad and convincing. The paper goes beyond correlational decomposition by testing addition, removal, and exchange of trait recipes, with matched random controls. The extension to sycophancy and refusal is particularly consequential, showing that the proposed decomposition captures components that influence safety-relevant behavior rather than only persona recognition.

Critical weaknesses
  1. The interpretation of individual traits as meaningful primitives is not yet fully established. The trait dictionary is correlated and linearly dependent, so the decomposition is not unique, and individual coefficients cannot necessarily be interpreted causally. The interventions establish that the joint recipe is behaviorally active, but not that the selected traits constitute a uniquely meaningful or mechanistically privileged decomposition.

  2. Several evaluation choices make the reported effect sizes somewhat difficult to interpret. Persona recognition relies on a single LLM judge and fixed candidate lists, without human or alternative-judge validation. In addition, the creation results report the best of three steering doses rather than a dose fixed prior to evaluation. These choices do not invalidate the main qualitative results, but stronger independent evaluation and pre-specified intervention strengths would improve confidence in the quantitative conclusions.

Questions for authors
  1. How stable are the selected trait recipes to changes in the trait dictionary? For example, if the dictionary is expanded, subsampled, or replaced with semantically related adjectives, do the same underlying trait concepts remain selected and does the resulting recipe retain similar causal effectiveness?

  2. Have the authors evaluated persona recognition with an independent judge model or human annotators? Evidence that the main creation/removal effects persist under a substantially different evaluator would increase my confidence in the behavioral validation.

  3. For the creation experiments, how much do conclusions change if the steering dose is selected entirely on calibration data and fixed before evaluating the held-out scenes, rather than reporting the best of three doses?

1 comment

Dear Reviewer yff6,

The rebuttal period ends in less than 24 hours. Unfortunately, we have not yet received a response from you regarding this paper, and your previous response appears to address another paper. We would greatly appreciate it if you could take another look at our paper and our responses to the other reviewers, and reconsider your score.

Thank you very much for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.