- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work investigates how exposure to training data influences the predictive behavior of LLMs across varying context lengths. The authors propose an analytical framework based on Fisher–Rao information geometry, which decomposes the shift in the full predictive distribution for a fixed target token—occurring when transitioning between short and long contexts—into a displacement R along the direction of the target token and a residual geometric gap. They further establish the decomposition R = B + G. Through controlled continued-training experiments on Pythia-70M, Pythia-160M, and SmolLM2-135M—varying the number of times documents are presented during training—the authors analyze changes in the predictive distributions. The experiments reveal a consistent positive statistical correlation between training exposure and geometric shifts in predictions, with the component B accounting for 75.90%–88.37% of the average slope of the R response. Furthermore, the authors demonstrate the framework's potential utility by applying it to membership inference and context-constrained model editing.
Critical strengths
The paper proposes a predictive geometric decomposition with rigorous mathematical properties, offering a new metric for predictive change that distinguishes between shifts in the probability of a target token and the redistribution of probabilities among other tokens. The experimental design in the paper is comprehensive, the conclusions are stated rigorously, and the scope of applicability is clearly defined.
Critical weaknesses
The experiments in the paper do not sufficiently demonstrate that incorporating full distribution geometry consistently improves practical detection performance across various model settings. Furthermore, related work mentioned in the main text is not directly compared against the results obtained for membership inference, such as ReCaLL and likelihood-array regression.
Questions for authors
What is the scope of applicability for context-constrained editing? The results presented in the paper are derived primarily from the use of a single manually triggered prompt and a target token within the Pythia-70M model; furthermore, both the protected and evaluation contexts are drawn from a limited set of textual sources. Please clearly distinguish between theoretical guarantees for the precise protection of constructed contexts and empirical evidence regarding generalization to unseen contexts. Additionally, please clarify the extent to which the current experiments support conclusions about the effectiveness of these edits. How should the incremental value of Fisher–Rao features—relative to likelihood/context features—be understood? Given that the paper mentions ReCaLL and likelihood-array regression in the "Related Work" section, could you clarify why these methods were not directly included in the current evaluation protocol, or perhaps provide comparable results?
4 comments
Q2. How should the incremental value of Fisher–Rao features—relative to likelihood/context features—be understood?
For membership classification, incremental value means the held-out performance change from adding geometric features to a specified comparator. W1 reports that test against both the rich likelihood/context baseline and target probabilities alone. The distinction matters because the rich baseline already uses vocabulary information, so it is not a target-probability-only comparator.
For interpreting exposure, the added value is a distinction that those probabilities leave undetermined. The full-distribution measurements reveal that gap growth can accompany decreasing alternative dissimilarity, as observed in three target–phase evaluations. The decomposition explains this through the combination of alternative affinity and endpoint weighting. Section 4.4 presents that empirical consequence, while Section 5 reports the separate predictive-value tests. This gives the geometric representation a concrete measurement role without treating access to additional information as evidence that a fitted classifier necessarily improves.
Q3. Given that the paper mentions ReCaLL and likelihood-array regression in the “Related Work” section, could you clarify why these methods were not directly included in the current evaluation protocol, or perhaps provide comparable results?
We have completed comparable evaluations; the results are below. The original table compared representations under a shared classifier search. A direct named-method comparison additionally required LAR-2's own feature construction and estimator, and new prefixed likelihood queries for ReCaLL. Appendix F.6 now reports those procedures on the original held-out groups:
| Procedure | Initial 70M | Fresh 70M | 160M | SmolLM2 |
|---|---|---|---|---|
| Rich likelihood/context | .7460 | .7610 | .8568 | .6280 |
| Rich likelihood + geometry | .7505 | .7628 | .8539 | .6048 |
| LAR-2, retained-grid adaptation | .6987 | .7353 | .8118 | .5787 |
| ReCaLL, five-shot fold prefixes | .7016 | .7055 | .8218 | .6016 |
LAR-2 uses the published working estimator and penalty with training-only normalization. Its objective is mean logistic loss plus , with an unpenalized intercept. Acquisition reuses the retained 24-context grid, so we label it a retained-grid adaptation.
ReCaLL uses its relative conditional-likelihood score with one fixed prefix per target/fold, concatenating five training-fold continued-training nonmember records. Those records are excluded from every role in that target's training, including background presentations. Conditional and unconditional likelihood are newly evaluated on the same target span.
These procedures have explicit access and supervision differences. ReCaLL uses known continued-training nonmembers and additional queries without fitting a membership classifier. The learned representations use labeled training groups and retained context measurements. The paired intervals condition on fitted procedures and existing targets, and ReCaLL's also condition on the selected prefixes. The higher observed AUC of the rich comparator in all four settings supports its strength on this panel, with the tested acquisition and supervision of each method specified.
Reminder
Dear Reviewer,
We have posted responses to all your comments and uploaded a revised manuscript. We added direct comparisons with ReCaLL and likelihood-array regression, together with target-probability-only ablations to assess the incremental value of geometric features. We also distinguish the editing construction’s exact protection guarantee from its empirical locality on unseen contexts and clarify the scope of the single-association illustration.
As the discussion period draws to a close, we would be grateful if you could let us know whether these responses and revisions address your concerns. We would be happy to provide any further clarification.
Thank you for your time and consideration.
Response to the Authors' Rebuttal
Thank you for your rebuttal and for addressing my concerns. After considering your responses, I have decided to maintain my original score.
Thank you for your review. We have uploaded an updated version of the paper. The theorem numbering and all section, table and appendix references below refer to that version.
We have added the requested LAR-2 and ReCaLL comparisons and made the editing guarantee and held-out evaluation explicit. The responses below connect each concern to the completed analysis and the corresponding paper changes.
W1. The experiments do not establish consistent practical detection improvements from full-distribution geometry across model settings.
We tested the practical added value against both a strong vocabulary-informed baseline and a new target-probability-only baseline. This separates two questions that the original baseline name obscured: whether geometric features improve a rich predictor, and whether they improve prediction from target probabilities alone.
The rich likelihood/context baseline includes vocabulary-standardized likelihood, token rank and the strongest-alternative margin. Adding geometry changes AUC by +.0045, +.0018, −.0029 and −.0232 in initial 70M, fresh 70M, Pythia-160M and SmolLM2, respectively. The new target-only baseline includes binary displacement B; adding a fixed G,N,1−h block changes AUC by −.0013, +.0046, +.0023 and −.0129 in the same order. In both comparisons, the first three paired conditional 95% intervals include zero and the SmolLM2 interval is negative, holding fitted classifiers and existing targets fixed. Thus the comparisons do not establish a positive incremental AUC benefit. The revised abstract and Section 5 state the supported result: context features rank controlled inclusion labels, with rich likelihood/context features capturing most performance obtained with augmentation.
The full-distribution analysis also supplies an exposure conclusion beyond these fitted classification results. All 24 observed gap slopes are positive, while three corresponding direct alternative-dissimilarity slopes are negative. Target probabilities cannot measure that alternative dissimilarity; the theorem explains how endpoint weighting permits these patterns to coexist. Section 4.4 now makes this measured consequence explicit. It is the scientific reason to distinguish binary motion, changing alternatives and endpoint weighting when interpreting exposure response.
W2. ReCaLL and likelihood-array regression appear in Related Work but were not directly compared in the membership evaluation.
We have now evaluated both named procedures on the same held-out document groups, target checkpoints and controlled labels. The rich likelihood/context baseline has higher observed AUC than the retained-grid LAR-2 adaptation and the specified five-shot ReCaLL procedure in all four settings. This supplies the missing comparison and tests the strength of the baseline against which geometry is assessed. Appendix F.6 reports the results with each method's supervision and query requirements; Q3 gives the table and implementation details.
Q1. What is the scope of applicability for context-constrained editing?
The guarantee is exact preservation on the constructed protection contexts; locality on unseen contexts is evaluated empirically. The restricted parameterization makes the guarantee tractable. With untied input/output embeddings, changing only the target output row keeps hidden states fixed. If U stacks the protected states and d is the update direction, a step αd changes the protected target logits by αUd. Enforcing Ud=0 therefore preserves all logits, and hence the complete predictive distribution, on those contexts for any finite step in exact arithmetic. Projecting the target gradient onto this null space gives a feasible target-changing direction when the projection is nonzero. Appendix G derives the minimum-norm direction and separately checks the realized finite-precision update.
The experiment tests locality beyond those constraints on sources separate from the construction sets. It uses one association in one Pythia-70M checkpoint and two context splits sharing the short target prompt. Target probabilities reach 0.09149 and 0.06441. Held-out longer-context 90th-percentile distances are 0.01215 and 0.01624; clean-context RMS distances are 0.00556 and 0.00196. At matched target strength, the post-study unprotected comparison has 41–58 times greater held-out longer-context movement. This comparison supports the protection method's locality on the sampled context families. The two splits test new context sources for the same association and prompt.
We have aligned the paper's claim with that evidence. Section 6 is titled “Illustration: a protected output-row edit,” and the conclusion identifies the single association and two context splits. Changing only one row also preserves conditional alternatives, giving G=0 and L=∣B∣ at each fixed context. The example therefore realizes a protected intervention in the equality case.