acceptodds
Under review as a conference paper at ICLR 2027

Quantifying the Agreement Between Data-Influence and Data-Similarity to Understand LLM Behavior

Abstract

One way to understand LLM behavior is to trace its output back to the training data. Two types of measures are commonly used for output tracing: data-similarity and data-influence. The former is cheaper while the latter is believed to be more accurate. Even though many works have compared them for ground-truth tasks, no such comparisons exist for output tracing. Here, we fill this gap and precisely quantify the commonalities and differences between the two measures. We do this by first ranking the training documents according to each measure and then computing the overlap between the two rankings. Our main finding is that the two rankings agree significantly, but there is an asymmetry between them: The top documents of data-similarity are assigned more consistent ranks by data-influence than the other way around. This result is valid across a range of experiments involving OLMo2-1B, Qwen3-1.7B, LlaMa3.2-1B, Gemma3-1B, and GPT2. We exploit the asymmetry to obtain a favorable cost-accuracy trade-off by using the costly data-influence to refine the results of data-similarity.

Primary area:
interpretability and explainable AI
Keywords:
Influence, Similarity, Understanding, Agreement, Interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tbeHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the agreement between data-similarity and data-influence methods for tracing LLM outputs to training documents. The authors compare their document rankings and observe an asymmetric relationship. Based on this finding, they propose combining similarity-based retrieval with influence-based re-ranking. Experiments are conducted across six language models.

Critical strengths
  • The paper investigates a potentially interesting problem concerning data-similarity and data-influence for LLM output tracing.

  • The experiments cover multiple language models and influence estimators.

Critical weaknesses
  • The paper does not sufficiently explain why agreement between data-similarity and data-influence rankings provides meaningful insights into LLM behavior. The connection between the research question and its practical significance remains unclear.

  • It is unclear whether the sampled documents were included in the actual training corpora of the evaluated models, raising concerns about interpreting the influence scores as training-data attribution.

-The paper reports asymmetric ranking agreement but provides limited explanation of its underlying causes or implications. Such agreement does not necessarily indicate better attribution quality.

Questions for authors
  • What meaningful insights into LLM behavior can be derived from the observed ranking agreement?

  • Were the evaluated documents part of the actual training corpora?

  • What explains the observed asymmetry between the two rankings?

  • Can the authors directly evaluate attribution quality and computational efficiency rather than relying on proxy-based metrics?

Reviewer_QUrQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper compares BM25 data-similarity rankings with several estimated data-influence rankings for LLM output tracing. It uses 100 prompts, six models, and a pool of 100,000 sampled DCLM documents. The rankings overlap at the top but asymmetrically: documents with higher BM25 ranks often also rank relatively highly under influence, more often than in the opposite direction. The authors suggest first retrieving candidates with BM25 and then refining them with influence scores.

Critical strengths

(1) The paper raises a useful question about two common tracing methods.

Critical weaknesses

(1) The paper does not establish that its rankings trace outputs back to the models’ training data. The experiment generates 100 prompts with Gemini, collects model responses, and ranks 100,000 documents randomly sampled from DCLM. These documents are treated as training data, but the paper does not establish that they belong to each model’s training corpus. It also provides no comparison with a real document-removal experiment or with each model’s actual training corpus, so the validity of the conclusion is not verified. The ranking comparison remains a valid descriptive analysis of the sampled pool, but interpreting it as tracing outputs back to training data relies on a provenance assumption not established in the paper. Agreement alone does not prove causal attribution. The paper further does not test whether the resulting ranking is useful. The hybrid experiment does not show that the proposed pipeline works. It scores only 100 positives and 100 random negatives, so the AUC only asks whether one measure separates those 200 documents. It does not check whether the retrieved top 100 are the top 100 over the full pool. For OLMo2-1B, a BM25 shortlist of 20,000 out of 100,000 documents keeps only about 65% of the influence top 100 (Section 3.F). The pipeline should be judged by how many high-influence documents it retains and how much computation it saves, not mainly by positives defined by BM25. The study also uses only BM25, so it remains open whether the same pattern holds for other similarity analysis.

(2) The experimental setup and evaluation limit how broadly the conclusions can be applied. Specifically, the scoring choices and prompts may confound the results. BM25 scores the concatenated prompt and response, while in src/ifsim/llm.py the influence loss masks prompt-token labels and scores the response conditioned on the prompt. The implementation also uses selected MLP projections and sums absolute tokenwise scores. These choices can change the rankings. The 100 prompts were generated with Gemini and manually checked, but were not balanced or categorized by topic. It is unclear whether the direction of the asymmetry holds across prompt types, or whether a few queries drive the mean. Thus, the observed results are therefore specific to this setup, not a general relationship between similarity and influence.

Questions for authors

Please see the weaknesses.

Reviewer_WW49OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the relationship between two families of output tracing methods on language models: data similarity and data influence. Data similarity is modeled primarily using BM25, while data influence is estimated using EK-FAC, with diagonal-Fisher and raw-gradient variants considered in an ablation. For 100 sentence-completion prompts, the authors generate responses from six language models and use each prompt-response pair as a query. They then rank a fixed pool of 100,000 DCLM documents according to BM25 similarity and estimated influence on the autoregressive loss. The resulting rankings are compared using overlap at different depths, rank-biased overlap, asymmetric containment, paired statistical tests, qualitative examples, and a proof-of-concept ROC analysis. The authors find non-random but incomplete and query-dependent agreement. Documents ranked highly by BM25 tend to receive reasonably favorable influence ranks more often than highly influential documents receive favorable BM25 ranks. Agreement is also higher for more detailed curvature approximations. Based on these observations, the paper proposes using similarity retrieval as a computationally inexpensive prefilter before applying influence estimation

Critical strengths
  1. The subject of study, the relationship between data similarity methods and data influence methods, is practically important and insufficiently studied. Similarity retrieval and influence estimation are often used for related attribution or tracing purposes, but they model different meanings of relevance. Measuring whether and when their rankings agree is useful for understanding under what scenarios the two methods are complementary or interchangeable. The use of asymmetric containment is especially appropriate because the proposed pipeline depends on the direction of retrieval, not merely symmetric overlap.
  2. The breadth of empirical experiments is good. The study covers six language-model checkpoints, 100 queries, a relatively large candidate collection, several ranking metrics, qualitative examples, and multiple influence approximations in the OLMo ablation. The asymmetry is observed reasonably consistently.
Critical weaknesses
  1. The reported asymmetry does not appear to support the proposed similarity-prefiltering pipeline. Let IKI_K denote the top-KK influence-ranked documents and SMS_M denote the top-MM BM25-ranked documents. If influence is treated as the expensive target and BM25 as the inexpensive prefilter, the relevant recall is
= \Pr(d \in S_M \mid d \in I_K).$$ For $K=100$, this is exactly the paper's **Sim-to-Inf** direction, which is the weaker of the two directions. For OLMo, retaining the top 1,000 BM25 documents recalls only approximately 28% of the top-100 influence documents, while retaining 20,000 of the 100,000 candidates recalls approximately 65%. Thus, even a prefilter retaining 20% of the corpus misses around 35% of the target influence set. Conversely, the stronger Inf-to-Sim result shows that influence ranking can recover BM25-selected documents; it does not show that BM25 preserves the most influential documents. Similarly, the reported AUC of 0.83 labels BM25-selected documents as positives and uses influence scores to recover them, which evaluates the reverse prediction problem. The more relevant opposite-direction AUC is substantially lower. Therefore, the current results do not establish that BM25 is a high-recall proxy for influence or that the proposed pipeline provides a favorable accuracy-computation trade-off. A direct evaluation of the complete prefilter-and-rerank pipeline is needed. 2. It is unclear whether the candidate DCLM documents are genuine training examples for the evaluated models. The paper defines influence through a counterfactual in which document (j) is removed from training and repeatedly describes the candidates as training documents. However, it applies the same random sample of 100,000 DCLM documents to six independently trained checkpoints without documenting exact dataset versions, record identifiers, or membership in each model’s training corpus. The candidate must still be an actual training point if the resulting score is interpreted as historical training-data attribution. For a document that was not used to train a model, the gradient-based quantity is better interpreted as a hypothetical sensitivity to adding or upweighting that document, rather than the effect of removing it from the model’s training data.
Questions for authors
  1. Evaluation of the proposed pipeline: If the intended application is to use BM25 to prefilter candidates before influence reranking, could the authors report RK(M)=∣TopK(Influence)∩TopM(Similarity)∣KR_K(M) = \frac{|\mathrm{TopK}(\mathrm{Influence}) \cap \mathrm{TopM}(\mathrm{Similarity})|}{K} over a range of K and M, followed by the quality of the final influence-reranked results relative to full-corpus influence computation? How do the authors reconcile the claimed favorable cost-accuracy trade-off with the reported approximately 28% recall at M=1000 and approximately 65% recall at M=20000 for K=100?
  2. Training-data provenance and interpretation: Are the 100,000 sampled DCLM records verified, exact training examples for each of the six evaluated checkpoints? If so, please provide the DCLM version, record identifiers or membership-verification procedure for each model, and clarify what data were used to estimate the EK-FAC or diagonal curvature statistics. If these documents cannot be verified as training examples, how should the removal counterfactual involving θ−j\theta_{-j} be interpreted, and should the experiments instead be framed as hypothetical influence over an external reference corpus?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.