acceptodds
Under review as a conference paper at ICLR 2027

Predictive Geometry in Language Models

Abstract

For a given token, a language model's prediction may be more or less robust to changes in the preceding context. We ask how training exposure changes this context robustness. To capture that change, we trace the full next-token distribution as the preceding context grows. We apply Fisher-Rao geometry to this path of predictive distributions, viewing context growth as motion on a statistical manifold. We develop a theory of these paths that measures the total predictive change and decomposes it into the part associated with the observed token's probability and the remaining change in the distribution over alternative tokens. Across four model-data settings, we show that the theoretical decomposition captures systematic effects of repeated exposure: both components of the predictive change increase with exposure. We then use the theory in two ways. For privacy auditing, we use Fisher-Rao context paths to distinguish documents presented during continued training from those omitted from it. For context-scoped editing, we use the geometry to strengthen a continuation after a short context while keeping longer-context predictions near their pre-edit values. It rises from % to -%, while prediction changes on held-out longer extensions are - smaller than without protection. Together, these results show that training exposure leaves a measurable geometric signature in context-dependent predictions that can be used both to detect exposure and to control where its effects appear.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
language models, training exposure, context dependence, Fisher–Rao geometry, statistical manifolds, next-token prediction, membership inference, model editing, predictive distributions, context robustness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_o8M1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work investigates how exposure to training data influences the predictive behavior of LLMs across varying context lengths. The authors propose an analytical framework based on Fisher–Rao information geometry, which decomposes the shift in the full predictive distribution for a fixed target token—occurring when transitioning between short and long contexts—into a displacement R along the direction of the target token and a residual geometric gap. They further establish the decomposition R = B + G. Through controlled continued-training experiments on Pythia-70M, Pythia-160M, and SmolLM2-135M—varying the number of times documents are presented during training—the authors analyze changes in the predictive distributions. The experiments reveal a consistent positive statistical correlation between training exposure and geometric shifts in predictions, with the component B accounting for 75.90%–88.37% of the average slope of the R response. Furthermore, the authors demonstrate the framework's potential utility by applying it to membership inference and context-constrained model editing.

Critical strengths

The paper proposes a predictive geometric decomposition with rigorous mathematical properties, offering a new metric for predictive change that distinguishes between shifts in the probability of a target token and the redistribution of probabilities among other tokens. The experimental design in the paper is comprehensive, the conclusions are stated rigorously, and the scope of applicability is clearly defined.

Critical weaknesses

The experiments in the paper do not sufficiently demonstrate that incorporating full distribution geometry consistently improves practical detection performance across various model settings. Furthermore, related work mentioned in the main text is not directly compared against the results obtained for membership inference, such as ReCaLL and likelihood-array regression.

Questions for authors

What is the scope of applicability for context-constrained editing? The results presented in the paper are derived primarily from the use of a single manually triggered prompt and a target token within the Pythia-70M model; furthermore, both the protected and evaluation contexts are drawn from a limited set of textual sources. Please clearly distinguish between theoretical guarantees for the precise protection of constructed contexts and empirical evidence regarding generalization to unseen contexts. Additionally, please clarify the extent to which the current experiments support conclusions about the effectiveness of these edits. How should the incremental value of Fisher–Rao features—relative to likelihood/context features—be understood? Given that the paper mentions ReCaLL and likelihood-array regression in the "Related Work" section, could you clarify why these methods were not directly included in the current evaluation protocol, or perhaps provide comparable results?

4 comments

Thank you for your review. We have uploaded an updated version of the paper. The theorem numbering and all section, table and appendix references below refer to that version.

We have added the requested LAR-2 and ReCaLL comparisons and made the editing guarantee and held-out evaluation explicit. The responses below connect each concern to the completed analysis and the corresponding paper changes.

W1. The experiments do not establish consistent practical detection improvements from full-distribution geometry across model settings.

We tested the practical added value against both a strong vocabulary-informed baseline and a new target-probability-only baseline. This separates two questions that the original baseline name obscured: whether geometric features improve a rich predictor, and whether they improve prediction from target probabilities alone.

The rich likelihood/context baseline includes vocabulary-standardized likelihood, token rank and the strongest-alternative margin. Adding geometry changes AUC by +.0045+.0045, +.0018+.0018, −.0029-.0029 and −.0232-.0232 in initial 70M, fresh 70M, Pythia-160M and SmolLM2, respectively. The new target-only baseline includes binary displacement BB; adding a fixed G,N,1−hG,N,1-h block changes AUC by −.0013-.0013, +.0046+.0046, +.0023+.0023 and −.0129-.0129 in the same order. In both comparisons, the first three paired conditional 95% intervals include zero and the SmolLM2 interval is negative, holding fitted classifiers and existing targets fixed. Thus the comparisons do not establish a positive incremental AUC benefit. The revised abstract and Section 5 state the supported result: context features rank controlled inclusion labels, with rich likelihood/context features capturing most performance obtained with augmentation.

The full-distribution analysis also supplies an exposure conclusion beyond these fitted classification results. All 24 observed gap slopes are positive, while three corresponding direct alternative-dissimilarity slopes are negative. Target probabilities cannot measure that alternative dissimilarity; the theorem explains how endpoint weighting permits these patterns to coexist. Section 4.4 now makes this measured consequence explicit. It is the scientific reason to distinguish binary motion, changing alternatives and endpoint weighting when interpreting exposure response.

W2. ReCaLL and likelihood-array regression appear in Related Work but were not directly compared in the membership evaluation.

We have now evaluated both named procedures on the same held-out document groups, target checkpoints and controlled labels. The rich likelihood/context baseline has higher observed AUC than the retained-grid LAR-2 adaptation and the specified five-shot ReCaLL procedure in all four settings. This supplies the missing comparison and tests the strength of the baseline against which geometry is assessed. Appendix F.6 reports the results with each method's supervision and query requirements; Q3 gives the table and implementation details.

Q1. What is the scope of applicability for context-constrained editing?

The guarantee is exact preservation on the constructed protection contexts; locality on unseen contexts is evaluated empirically. The restricted parameterization makes the guarantee tractable. With untied input/output embeddings, changing only the target output row keeps hidden states fixed. If UU stacks the protected states and dd is the update direction, a step αd\alpha d changes the protected target logits by αU d\alpha U\,d. Enforcing U d=0U\,d=0 therefore preserves all logits, and hence the complete predictive distribution, on those contexts for any finite step in exact arithmetic. Projecting the target gradient onto this null space gives a feasible target-changing direction when the projection is nonzero. Appendix G derives the minimum-norm direction and separately checks the realized finite-precision update.

The experiment tests locality beyond those constraints on sources separate from the construction sets. It uses one association in one Pythia-70M checkpoint and two context splits sharing the short target prompt. Target probabilities reach 0.09149 and 0.06441. Held-out longer-context 90th-percentile distances are 0.01215 and 0.01624; clean-context RMS distances are 0.00556 and 0.00196. At matched target strength, the post-study unprotected comparison has 41–58 times greater held-out longer-context movement. This comparison supports the protection method's locality on the sampled context families. The two splits test new context sources for the same association and prompt.

We have aligned the paper's claim with that evidence. Section 6 is titled “Illustration: a protected output-row edit,” and the conclusion identifies the single association and two context splits. Changing only one row also preserves conditional alternatives, giving G=0G=0 and L=∣B∣L=|B| at each fixed context. The example therefore realizes a protected intervention in the equality case.

Q2. How should the incremental value of Fisher–Rao features—relative to likelihood/context features—be understood?

For membership classification, incremental value means the held-out performance change from adding geometric features to a specified comparator. W1 reports that test against both the rich likelihood/context baseline and target probabilities alone. The distinction matters because the rich baseline already uses vocabulary information, so it is not a target-probability-only comparator.

For interpreting exposure, the added value is a distinction that those probabilities leave undetermined. The full-distribution measurements reveal that gap growth can accompany decreasing alternative dissimilarity, as observed in three target–phase evaluations. The decomposition explains this through the combination of alternative affinity and endpoint weighting. Section 4.4 presents that empirical consequence, while Section 5 reports the separate predictive-value tests. This gives the geometric representation a concrete measurement role without treating access to additional information as evidence that a fitted classifier necessarily improves.

Q3. Given that the paper mentions ReCaLL and likelihood-array regression in the “Related Work” section, could you clarify why these methods were not directly included in the current evaluation protocol, or perhaps provide comparable results?

We have completed comparable evaluations; the results are below. The original table compared representations under a shared classifier search. A direct named-method comparison additionally required LAR-2's own feature construction and estimator, and new prefixed likelihood queries for ReCaLL. Appendix F.6 now reports those procedures on the original held-out groups:

ProcedureInitial 70MFresh 70M160MSmolLM2
Rich likelihood/context.7460.7610.8568.6280
Rich likelihood + geometry.7505.7628.8539.6048
LAR-2, retained-grid adaptation.6987.7353.8118.5787
ReCaLL, five-shot fold prefixes.7016.7055.8218.6016

LAR-2 uses the published working estimator and penalty with training-only normalization. Its objective is mean logistic loss plus 0.01∥β∥22/20.01\|\beta\|_2^2/2, with an unpenalized intercept. Acquisition reuses the retained 24-context grid, so we label it a retained-grid adaptation.

ReCaLL uses its relative conditional-likelihood score with one fixed prefix per target/fold, concatenating five training-fold continued-training nonmember records. Those records are excluded from every role in that target's training, including background presentations. Conditional and unconditional likelihood are newly evaluated on the same target span.

These procedures have explicit access and supervision differences. ReCaLL uses known continued-training nonmembers and additional queries without fitting a membership classifier. The learned representations use labeled training groups and retained context measurements. The paired intervals condition on fitted procedures and existing targets, and ReCaLL's also condition on the selected prefixes. The higher observed AUC of the rich comparator in all four settings supports its strength on this panel, with the tested acquisition and supervision of each method specified.

Dear Reviewer,

We have posted responses to all your comments and uploaded a revised manuscript. We added direct comparisons with ReCaLL and likelihood-array regression, together with target-probability-only ablations to assess the incremental value of geometric features. We also distinguish the editing construction’s exact protection guarantee from its empirical locality on unseen contexts and clarify the scope of the single-association illustration.

As the discussion period draws to a close, we would be grateful if you could let us know whether these responses and revisions address your concerns. We would be happy to provide any further clarification.

Thank you for your time and consideration.

Reviewer_o8M1OpenReview ↗

Response to the Authors' Rebuttal

Thank you for your rebuttal and for addressing my concerns. After considering your responses, I have decided to maintain my original score.

Reviewer_FGkHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper develops an information-geometric account of how a language model's next-token prediction changes as the preceding context grows, and uses it to study the effect of training exposure. For a fixed target token, it traces the full next-token distribution as the context is extended one token at a time (nested contexts), treats this as a path on a statistical manifold, and measures the Fisher–Rao motion. In conventional terms, the "context" is a prefix and the "target" is a single-token suffix, so the nested contexts amount to sweeping the prefix length while reading the teacher-forced next-token probability, rather than checking whether a multi-token suffix is reproduced.

Contribution:

  1. A novel token-directed Fisher–Rao decomposition of how a prediction moves as context grows, splitting each step exactly into a binary target-versus-rest term and a nonnegative gap.
  2. A randomised continued-training study, with design-based inference, that measures and decomposes the exposure effect.
  3. Two applications of the framework, membership inference from the final checkpoint and a context-scoped edit.

Validation: four continued-training settings over small models (Pythia-70M in two studies, Pythia-160M, SmolLM2-135M) with injected documents at repetition counts D∈0,…,16D\in{0,\dots,16}. Exposure reliably increases the geometric motion (dominated by the binary term, small gap); membership AUCs are modest (~0.60–0.83) and not consistently above a likelihood baseline; the single editing case study raises a target continuation from ~0.002% to 6–9% while keeping held-out longer-context predictions 41–58× more stable than an unprotected edit.

Critical strengths
  • The information-geometric framing is novel and clean. Representing accumulating-context predictions as a path on the simplex and deriving an exact decomposition of the Fisher–Rao motion into binary target-versus-rest movement plus a nonnegative gap is a principled contribution, and it gives a precise language for what scalar target probability captures and omits. I do not know of prior work that characterised full-distribution motion across nested contexts this way.
Critical weaknesses
  • The method's significance is unestablished: in Table 2 the Fisher–Rao features do not consistently beat an observed-token likelihood baseline, and the exposure response is dominated by the binary term with a small gap. Combined with a narrow regime (repetition ≤16, single-token teacher-forced targets, 70M–160M models, below where memorisation is typically realized), whether the geometry adds value beyond a scalar on the membership-inference and exposure-measurement tasks it is proposed for is left open.
Questions for authors
  • Does the Fisher–Rao signal correlate with a conventional verbatim-memorisation measure (Rouge-L on greedy continuations), and how does this vary with repetition?
  • At the studied regime the Fisher–Rao signal only ties the scalar likelihood baseline. Is there evidence that the signal, and the geometry's advantage over the scalar, strengthens toward higher repetition, larger models, or multi-token targets?
4 comments

Thank you for your review. We have uploaded an updated version of the paper. The theorem numbering and all section, table and appendix references below refer to that version.

We have added the requested greedy-continuation study and made its connection to the token-level exposure measurements explicit. The response below gives the empirical interpretation supplied by the geometry and the evidence on continuation recovery, repetition and scale.

W1. The significance beyond scalar likelihood is unclear: geometric features do not consistently improve membership AUC, the binary component dominates, and the experiments use repetition up to 16, single-token teacher-forced targets and 70M–160M models.

Binary dominance is an empirical result of the decomposition; it does not make the full-distribution measurement redundant. The distinction matters because an increase in the gap can reflect changing endpoint weights even when the conditional alternatives become more similar. We observe exactly this pattern: all 24 target–phase gap slopes are positive, while three corresponding direct alternative-dissimilarity slopes are negative. A target-probability profile cannot measure that alternative dissimilarity. The theorem explains how the two changes can coexist through R=B+GR=B+G and the gap's endpoint weighting. Section 4.4 now makes this consequence central.

The controlled regime is appropriate for testing exposure response because presentation counts are assigned and the same documents and contexts are measured before and after training. Complete-allocation inference respects the joint assignment structure of documents sharing a checkpoint. The primary global sharp no-response null is rejected in all eight phases, with positive observed target slopes. Thus the studied regime already yields measurable exposure response, even though the new continuation probe yields no complete 32-token recoveries. This distinguishes predictive response to exposure from exact generation of a reference continuation.

We also addressed the practical comparison directly. The submitted likelihood/context baseline uses vocabulary-standardized likelihood, rank and the strongest-alternative margin. The revision makes those inputs explicit and adds a pure target-probability baseline, including BB, under the same grouped protocol. Neither augmentation comparison establishes a positive AUC increment at the reported conditional 95% level, holding fitted classifiers and existing targets fixed. The abstract and Section 5 now state the supported application result: context features rank controlled inclusion labels, with rich likelihood/context features capturing most performance obtained with augmentation. The theorem and exposure study establish the additional interpretation of categorical change described above.

Finally, Section 4.5 and Appendix E.5 supply the missing behavioral connection through greedy continuation, dose-resolved overlap, exact recovery and held-out overlap prediction. Q1 reports that evidence; Q2 distinguishes the observed scale comparison from extensions beyond the tested regime.

Q1. Does the Fisher–Rao signal correlate with a conventional verbatim-memorisation measure (ROUGE-L on greedy continuations), and how does this vary with repetition?

The new study finds weak positive pooled correlations between final-minus-base ROUGE-L and the original geometric response ΔSR\Delta S_R, with variation across doses. We evaluated all 28,800 original document–target cells at matched base and final checkpoints. The first 16 retained tokens prompt greedy generation of up to 32 tokens, compared with the next 32 reference tokens. This protocol was fixed before the new acquisition. ROUGE-L F1 measures partial textual overlap; complete recovery requires matching all 32 reference token IDs.

SettingMean ROUGE-L change at dose 0Mean ROUGE-L change at dose 16Pooled correlation with geometric response
Initial 70M0.00880.02080.048
Fresh 70M0.00570.01610.033
Pythia-160M0.00360.03500.121
SmolLM20.02750.03720.063

At dose 16, mean overlap changes exceed their zero-dose counterparts in all four settings. The full dose curves and dose-stratified correlations are reported in Section 4.5 and Appendix E.5. Pooled Spearman correlations range from 0.033 to 0.121, while within-dose signs are mixed in Pythia; the association therefore does not imply that geometry tracks recovery uniformly at each repetition level. Zero-dose changes also reflect shared continued training on other documents. The group-resampling intervals retain repeated document appearances and condition on the observed targets and assignments.

There are zero complete 32-token matches among 9,600 base and 28,800 final generations under this prompt and decoder. Together with the partial-overlap results, this shows why the behavioral evaluation must distinguish overlap from exact continuation recovery.

We also tested whether geometric augmentation improves held-out prediction of final ROUGE-L. All four paired conditional 95% intervals for the augmentation-minus-target-only MSE difference include zero, conditioning on fitted regressors and targets. These predictors use whole-document summaries. A separate position-aligned analysis measures geometry and likelihood at the same 32 reference positions as the generation outcome. Appendix E.5 keeps the aligned correlations and whole-document prediction test distinct.

Q2. At the studied regime the Fisher–Rao signal only ties the scalar likelihood baseline. Is there evidence that the signal, and the geometry's advantage over the scalar, strengthens toward higher repetition, larger models, or multi-token targets?

The existing comparison supports a larger geometric exposure response at 160M, but does not establish a growing classification advantage. Primary RR response slopes are larger in Pythia-160M than in fresh Pythia-70M, while adding geometry changes the 160M membership AUC from 0.8568 to 0.8539. Section 5 now discusses these results together: response magnitude and incremental detection performance are separate measured properties.

For multi-token behavior, we have added the requested 32-token continuation evaluation on the same documents and checkpoints. It finds the weak pooled associations and dose-resolved overlap changes reported under Q1. This extends the behavioral evidence beyond the original teacher-forced target measurement. The theorem itself characterizes per-token categorical predictions; it does not assert a result about the joint continuation distribution.

Higher repetition than 16 and substantially larger models remain open extensions. The abstract and conclusion now state the tested 70M–160M model range and doses 0,1,2,4,8,160,1,2,4,8,16 explicitly. We also clarified the context construction: acquisition samples 24 nested context lengths, extending leftward while holding the target token fixed. These changes identify the regime in which the exposure result is established and the comparisons needed to test whether an advantage emerges at greater scale or repetition.

Dear Reviewer,

We have posted responses to all your comments and uploaded a revised manuscript. We added the requested greedy-continuation evaluation, including dose-resolved ROUGE-L results, exact-recovery measurements, and comparisons with target-probability predictors. We also clarify the geometry’s contribution to interpreting exposure response and distinguish the observed scale effects from the open questions concerning higher repetition and substantially larger models.

As the discussion period draws to a close, we would be grateful if you could let us know whether these responses and revisions address your concerns. We would be happy to provide any further clarification.

Thank you for your time and consideration.

Thank you for the detailed response and the additional greedy-continuation study.


Q1. The new results answered my question, but further strengthened my concern. The correlation with Rouge-L score is pretty weak, the sign even flips within some doses for Pythia, and augmenting with G,N,1−hG, N, 1-h doesn't improve held-out Rouge-L prediction (all MSE intervals include zero). Together with Table 17, the geometry still doesn't add much over target probabilities in practice.

Regarding the 3/24 cases where GG grows while alternative dissimilarity decreases: App. E.2 reports that at the document–target level only 2 of 28,799 increasing-gap cells reverse, and states that these cases "should not be presented as typical training responses." It is therefore unclear what general conclusion about exposure this distinction supports.


Q2. Scale remains my main concern. At 70M–160M and D≤16D \le 16, there are 0 exact 32-token recoveries across 38,400 generations, and the mean Rouge-L change at D = 16 is only 0.016–0.037. At this scale, greedy decoding is prone to repetition and degeneration, so ROUGE-L may mostly reflect generation quality rather than recall, please also report Type Token Ratio to confirm this. The paper additionally missing these key informations: token-per-parameter or flop, and training sequence length.


I see value in the R=B+GR = B + G decomposition, but given these concerns I will keep my score.

Reviewer_AbzkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how a language model’s predictions change as more context is supplied, and how repeated training exposure affects these changes. It uses Fisher–Rao geometry to compare next-token distributions and separates a token-directed measure of change into a part determined by the target token’s probabilities and an additional gap involving the alternatives. It tests these quantities under randomized continued training, then applies them to membership inference and context-specific editing.

Critical strengths
  • The distinction between changes in the target token’s probability and changes among the alternatives is useful. The geometric decomposition gives a precise way to describe it (§3).
  • The exposure study controls document presentation counts and compares the same documents and contexts before and after training. Its analysis accounts for documents sharing a trained model (§4; Appendix C.3).
  • The paper includes relevant likelihood-based comparisons and reports settings where adding geometry does not help.
Critical weaknesses

My main concern is what the full-distribution geometry adds beyond simpler likelihood-based measurements. The target-versus-rest term explains roughly 76–88% of the mean exposure response. The remaining gap also depends on the target probabilities, so a larger gap does not necessarily mean that the alternatives have become more different across contexts (§4.3).

The paper’s own diagnostics make this concern concrete. Under its chosen attribution, alternative-token changes account for about 0.3–10% of the mean R response (Table 4). In the initial 70M study, a model using only observed-token inputs predicts R with R² around 0.96 (Appendix E.2). This does not explain all of the exposure effect, but it suggests that much of the measured signal is already available from the target probabilities.

The membership results do not show a consistent practical gain. Likelihood-based context features outperform geometry-only features in all four settings. Adding geometry changes AUC by approximately +0.005, +0.002, −0.003 and −0.023 (Table 2). Individual path-length, turning-angle and excess-length scores are also near chance (Table 7). These results support a context-dependent membership signal, but provide limited evidence that the new geometry improves its detection.

The editing experiment changes only the target token’s output weights, leaving all other logits unchanged. The conditional distribution over alternative tokens therefore stays the same, and the proposed gap is zero. This experiment does not test the new alternative-token component of the theory.

The improvement over the unprotected edit comes from the null-space constraint that protects other contexts. The paper also shows that using the target probability or its log gives the same update direction. The result supports the protection method, but does not establish an added benefit from the proposed geometry.

Questions for authors
  • What conclusion about training exposure becomes available through the full-distribution gap that could not be obtained from target-probability context profiles?
  • Since the one-row editing experiment has unchanged conditional alternatives and a zero gap, what does it demonstrate about the new decomposition?
  • Should the membership results be interpreted mainly as evidence for likelihood-based context features, given the small and inconsistent gains from adding geometry?
5 comments

Thank you for your review. We have uploaded an updated version of the paper. The theorem numbering and all section, table and appendix references below refer to that version.

The responses below distinguish what the full-distribution analysis reveals, how large the measured contributions are, and what the application experiments establish.

W1. The added value beyond likelihood is unclear: the binary term explains roughly 76–88% of the mean exposure response, and the remaining gap also depends on the target probabilities.

Target probabilities alone cannot determine the full token-directed change. For example, compare p=(0.2,0.3,0.5)p=(0.2,0.3,0.5) with either q=pq=p or q=(0.2,0.6,0.2)q=(0.2,0.6,0.2), taking the first token as the target. Both comparisons have identical target probabilities and B=0B=0, but the first has G=0G=0 and the second has G>0G>0. The alternatives supply information that the target-probability endpoints leave undetermined.

Theorem 1 establishes how that information enters: R=B+GR=B+G, with G≥0G\geq0 and equality precisely when the conditional alternatives agree. It also derives the gap GG's dependence on endpoint target probabilities. This dependence is essential to interpretation: the gap combines alternative dissimilarity with its probability-dependent weighting.

The exposure study demonstrates the consequence of this. All 24 observed gap slopes are positive, while three corresponding direct alternative-dissimilarity slopes are negative. The full-distribution analysis distinguishes these outcomes and explains how they can occur together. A likelihood-only analysis cannot measure the accompanying alternative dissimilarity or establish this distinction.

W2. Alternative-token changes account for about 0.3–10% of the mean response, and target-probability inputs predict RR with R2R^2 around 0.96.

Despite its high prediction R2R^2, this fitted scalar control leaves exposure-related structure in the transition residuals. After predicting RR from the two target probabilities and aggregating those residuals into an upper-tail statistic, their observed exposure-response slopes are 10.9% and 11.5% of the original upper-tail response slopes. The reported block-bootstrap ratio intervals are 9.6–12.2% and 10.3–12.7%, respectively. Thus high predictive accuracy coexists with measured exposure dependence in the residual statistic in both held evaluations. In Appendix E.3, we report the control and its ratio intervals.

The decomposition in Theorem 1 then explains the composition of the measured change. Binary motion accounts for 75.90–88.37% of retained mean-response slopes. The gap's 11.63–24.10% share combines changing alternatives with endpoint weighting, and the attribution assigned specifically to changing affinity is 0.26–9.74% under the order-averaged convention. These measurements show that much of the gap response is assigned to endpoint weighting, rather than to changing affinity alone. We make this empirical conclusion explicit in Section 4.4. The residual upper-tail analysis and these mean-attribution shares use different aggregations and retain their separate interpretations.

W3. Likelihood/context features outperform geometry-only features in all four settings; augmentation gains are small and inconsistent, and individual path-length, turning-angle and excess-length scores are near chance.

We addressed the comparison directly by adding a target-probability-only baseline and testing it under the same grouped fitting protocol. This separates the question of added information beyond target probabilities from augmentation of a predictor that already uses vocabulary information.

The rich likelihood/context representation has higher observed AUC than the geometry-only representation in all four settings. It includes vocabulary-standardized likelihood, token rank and the strongest-alternative margin. We made those inputs explicit and renamed the baseline accordingly. The new target-only baseline uses transformations of target probabilities and binary displacement BB. The measured augmentation differences are:

SettingGeometry added to rich likelihood/contextG,N,1−hG,N,1-h added to target-only features
Initial 70M+0.0045−0.0013
Fresh 70M+0.0018+0.0046
Pythia-160M−0.0029+0.0023
SmolLM2−0.0232−0.0129

In both comparisons, the first three paired conditional 95% intervals include zero and the SmolLM2 interval is negative, and the target-only SmolLM2 difference remains negative after the four-comparison adjustment. The intervals hold the fitted classifiers and target checkpoints fixed. The updated abstract and Section 5 state the resulting application claim precisely: context features rank controlled inclusion labels, and rich likelihood/context features capture most of the performance obtained with augmentation.

The property controls further identify which measurements carry marginal membership signal. The strongest marginal geometric scores are the token-directed quantities A,R,B,GA,R,B,G; length, turning and excess-length scores are close to chance. Appendix F.3 reports these controls, and Appendix F.5 gives the new target-only comparison. Together they provide a direct assessment of predictive utility. W1–W2 establish the separate exposure conclusions that require the full-distribution analysis.

W4. The one-row edit preserves conditional alternatives and therefore has a zero gap; it does not exercise the alternative-token component.

The edit demonstrates a feasible protected intervention in the theorem's equality case. Updating one output row preserves conditional alternatives, so G=0G=0 and L=∣B∣L=|B| at each fixed context. The complete distribution's before/after distance is therefore exactly accounted for by binary movement.

We made this evidential role explicit in the revision. Editing is excluded from the abstract's main empirical summary and the two headline contributions; Section 6 is titled “Illustration: a protected output-row edit.” Appendix G derives the conditional-alternative invariance. The theorem and exposure study supply the nonzero-gap characterization and measurements, while this example realizes the target/locality contract in the equality case.

W5. Null-space protection explains the editing improvement, and probability and log-probability objectives yield the same update direction.

The construction makes the protection mechanism explicit. Restricting the edit to an untied output row keeps hidden states fixed, so preserving construction-context logits becomes an exact linear constraint. Appendix G derives the minimum-norm feasible direction when the projected target gradient is nonzero and checks the realized finite-precision update.

The matched-strength comparison tests whether that protection matters empirically. The post-study unprotected update has 41–58 times greater held-out longer-context movement. This is the evidence supporting the protected intervention. Probability and log-probability yield the same projected direction because their gradients differ by a positive scalar; the invariance concerns the direction, while normalization and step units can differ. Section 6 assigns protection to the constraint and uses geometric coordinates to express the target change and locality in common units.

Q1. What conclusion about training exposure becomes available through the full-distribution gap that could not be obtained from target-probability context profiles?

An exposure-associated increase in the gap can accompany a decrease in unweighted alternative dissimilarity. The full-distribution measurements identify this pattern in three target–phase evaluations, and the theorem explains how endpoint weighting permits it. Target probabilities determine the binary contribution but leave the alternative dissimilarity undetermined. W1 gives the mathematical and empirical evidence; W2 quantifies the response's composition and the fitted scalar control's residual.

Q2. Since the one-row editing experiment has unchanged conditional alternatives and a zero gap, what does it demonstrate about the new decomposition?

It demonstrates how a protected context-specific change can be constructed in the equality case, where the full distance equals ∣B∣|B|. Its contribution is the realized target/locality contract and the measured benefit of protection. We changed its role in the paper accordingly: Section 6 and Appendix G present a supporting illustration, while the abstract and headline contributions center the theorem and controlled exposure study. W4–W5 give the derivation, manuscript changes and empirical comparison.

Q3. Should the membership results be interpreted mainly as evidence for likelihood-based context features, given the small and inconsistent gains from adding geometry?

For practical ranking in these settings, rich likelihood/context features capture most of the performance obtained with augmentation. We now test that conclusion against both a vocabulary-informed comparator and target probabilities alone, as detailed under W3. The exposure conclusion is different and independently measured: scalar profiles leave alternative dissimilarity undetermined, and the decomposition explains how that dissimilarity and endpoint weighting enter predictive change. The paper reports both the classification outcome and this additional account of the exposure response.

Comments on the paper

5 comments

Thank you for your review. We have uploaded an updated version of the paper. The theorem numbering and all section, table and appendix references below refer to that version.

We have made the empirical consequence of the decomposition more prominent and revised the framing to distinguish the general mathematical result from the controlled settings in which we measure it.

W1. The empirical evidence is narrower than the broad framing: controlled continued training of small models and one editing association in one checkpoint.

The controlled design makes exposure measurable as an intervention: we assign document presentation counts and compare the same documents and contexts before and after training. Complete-allocation inference draws complete allowed assignments rather than independently permuting documents that share a trained checkpoint. These choices separate assigned exposure from pre-existing document predictability and account for the dependence induced by joint training.

We have brought that design's scope into the abstract and conclusion. Both now identify randomized continued training of 70M–160M models at doses 0,1,2,4,8,160,1,2,4,8,16. The abstract also identifies membership ranking as a held-out-document analysis under existing target checkpoints. The conclusion describes editing as one association with two context splits, and Section 6 is explicitly titled “Illustration: a protected output-row edit.”

This framing gives each result its supported role. The theorem characterizes categorical predictive change independently of model size or training history; the randomized study measures how that relationship responds to assigned exposure. Original-pretraining membership, substantially larger models and broader editing effectiveness are stated as extensions, rather than treated as conclusions of these experiments.

W2. The incremental value of full-distribution features varies across applications, leaving the practical conditions for using them unclear.

The clearest demonstrated use is to interpret categorical predictive change: the full-distribution measurements distinguish changes in conditional alternatives from changes in the target probabilities that weight their geometric contribution. This distinction matters empirically: all 24 observed gap slopes are positive, while three corresponding direct alternative-dissimilarity slopes are negative. Gap growth alone would give the wrong interpretation of alternative redistribution in those cases.

For membership ranking, we directly tested whether this extra information improves a fitted predictor. The revision adds a target-probability-only baseline, including binary displacement BB, under the same grouped fitting protocol. Adding a fixed G,N,1−hG,N,1-h block changes AUC by −.0013-.0013, +.0046+.0046, +.0023+.0023 and −.0129-.0129 in initial 70M, fresh 70M, Pythia-160M and SmolLM2, respectively. The first three paired conditional 95% intervals include zero; the SmolLM2 interval is negative. Augmentation of the rich likelihood/context baseline likewise establishes no positive increment at that level. These intervals condition on the fitted classifiers and existing target checkpoints. Section 5 therefore identifies rich likelihood/context features as capturing most of the ranking performance obtained with augmentation.

The controls give useful guidance about the heterogeneity. The all-family selector chooses rich likelihood alone in 19 of 20 folds. Ordinary shape/divergence augmentation also deteriorates in SmolLM2, so the observed degradation is not specific to Fisher–Rao features. These findings are consistent with overlapping predictive information and costs from fitting larger feature sets. SmolLM2 changes model family and corpus together, so it does not isolate an architecture effect. Section 5 and Appendix F now discuss these controls together: geometry supplies the demonstrated interpretation of categorical change, while the grouped comparisons assess whether to add it for classification.

Q1. Which empirical conclusions about exposure or membership inference are most specifically enabled by the additional full-distribution information, beyond the likelihood-path account? A short discussion of why the incremental gains differ across model settings would help readers understand when to use the proposed representation.

The additional exposure conclusion is that an increasing geometric gap can accompany decreasing unweighted alternative dissimilarity. We observe this in three target–phase evaluations. Target-probability profiles determine BB but cannot measure the conditional-alternative affinity hh; the theorem explains how that affinity and endpoint weighting jointly determine GG. Measuring both quantities is what lets us distinguish these exposure patterns. Section 4.4 now puts this finding alongside the component attribution, making the scientific consequence explicit.

For membership inference, the additional measurements do not establish an AUC improvement in the reported comparisons. The revision resolves an important ambiguity in that comparison: the rich likelihood/context baseline already includes vocabulary-standardized likelihood, token rank and the strongest-alternative margin. The added pure target-probability ablation separately tests augmentation beyond target probabilities alone. W2 gives its results and the controls that bear on heterogeneity. Together, these analyses explain why access to additional categorical information and improved membership ranking are separate empirical questions.

Q2. Could the authors make the scope of the controlled continued-training and one-association editing results more explicit in the abstract and conclusion, and state which extensions to original pretraining or larger models remain open?

We have made those distinctions in the uploaded paper. The abstract and conclusion name the controlled continued-training regime, model sizes and dose range. The conclusion identifies original-pretraining membership, unseen-target transfer, substantially larger models and broader editing effectiveness as open extensions.

For editing, Section 6 separates two forms of evidence. With fixed hidden states and an untied output-row update, the null-space constraint preserves all logits on the construction contexts for any finite step in exact arithmetic. Locality on held-out contexts is measured separately. The conclusion identifies this as one association and two context splits; the checkpoint and short target prompt are shared. This states what the construction guarantees and what the experiment tests, while keeping the general categorical theorem distinct from the scope of this illustration.

We thank all the reviewers for assessing our work. To strengthen the paper and address the reviews, we have uploaded a revised manuscript with the following updates:

  • Added a target-probability-only membership ablation.
  • Added comparisons with retained-grid LAR-2 and five-shot ReCaLL.
  • Added a greedy-continuation study using the existing documents and checkpoints.
  • Clarified the distinction between endpoint weighting and alternative-token redistribution.
  • Made the likelihood/context baseline’s vocabulary-informed inputs explicit.
  • Stated the experimental scope more directly in the abstract and conclusion.
  • Presented editing as a supporting illustration, distinguishing exact protection on constructed contexts from measured locality on held-out contexts.

The original geometric results and controlled exposure study are retained. All references in our responses refer to the revised manuscript. We hope these revisions and our responses address the reviewers’ concerns and clarify the contribution and supporting evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.