acceptodds
Under review as a conference paper at ICLR 2027

From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution

13 views

Abstract

Training data attribution (TDA) aims to identify training examples that shape model behavior, but its intervention value depends on both which examples are selected and how they are modified. Influence functions (IF) estimate behavioral changes under infinitesimal reweighting, yet IF-selected examples often show limited advantages over random selection under conventional weight-based interventions. This raises the question of whether influential examples lack intervention value or whether reweighting fails to realize their behavioral leverage. We introduce influence-guided response rewriting, which uses IF to identify intervention targets and replaces their responses with behavior-aligned or behavior-opposed supervision while keeping instructions fixed. Across four open-weight LLMs, we compare rewriting and reweighting on the same influence-selected examples using epistemic abstention as our primary testbed. Although the same influence-selected examples exhibit weak and inconsistent effects under weight-based interventions, changing their supervision content reveals strong, persistent and bidirectional behavioral leverage. Further analyses show that influence-selected examples provide greater rewriting leverage than alternative selectors, with changes remaining concentrated on target-relevant behaviors. The same qualitative contrast extends to safety refusal. These results distinguish the local reweighting effects captured by influence estimates from the broader intervention leverage of the examples they identify, motivating intervention-aware evaluation of TDA methods.

Primary area:
interpretability and explainable AI
Keywords:
Large Language Models, Data Attribution, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dPBFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Summary:

The paper asks why training examples selected by influence functions (IF) often perform no better than random examples when they are upweighted or deleted during fine-tuning. Is IF picking the wrong examples, or does changing their weight fail to bring out what they could teach the model? To separate the two, the authors keep the IF selection fixed and vary only the intervention. They introduce influence-guided response rewriting: the instruction of each selected example is kept, and its response is replaced with a template that either supports the target behavior (aligned) or opposes it (opposed).

Contributions:

  1. Influence-guided response rewriting. The paper separates choosing training examples from modifying them: influence functions identify which SFT examples to target, and replacing their responses determines what those examples teach the model. The authors also propose intervention leverage to measure how well a selection method and an intervention work together.

  2. Controlled retraining experiments on four models. The authors retrain OLMo2-1B, Qwen3.5-2B, Gemma3-4B and OLMo2-7B under identical settings, varying only the intervention applied to the same IF-selected examples. They report that rewriting produces strong, lasting effects in both directions, while upweighting and deletion do not.

  3. Further analyses. The authors compare IF with other ways of selecting examples (a probe for unanswerability, TRAK, gradient and embedding similarity, and others), examine how rewriting changes the influence scores of the selected examples, check whether gains stay on the targeted types of abstention rather than increasing refusal overall, and extend the study to safety refusal on OLMo2-7B.

Validation:

The authors retrain four models (OLMo2-1B, Qwen3.5-2B, Gemma3-4B, OLMo2-7B) on a 64k Tulu 3 SFT subset, applying each intervention separately to the top 2.5% and the bottom 2.5% of IF-ranked examples (1,600 each), and compare against an unmodified baseline and randomly selected examples. They measure abstention recall on AbstentionBench across training checkpoints, and report that rewriting beats matched random selection in both directions on all four models, while upweighting and deletion are weak and inconsistent, sometimes moving the behavior the wrong way. Additional experiments cover other selection methods, scenario-level results, and safety refusal on OLMo2-7B.

Critical strengths

A clear research question. The paper asks whether the limited gains from upweighting or deleting influence-selected examples come from poor selection or from reweighting not being able to change what those examples teach the model. Separating those two is a useful way to think about data attribution, and the question is worth asking even if the motivating claim rests on thin evidence (see Questions). The authors also released their code, which let me check the implementation.

Critical weaknesses

Weakness 1: The upweighting in the code does not match the paper. Section 3.2 (lines 183–184) defines upweighting as multiplying a selected example’s loss by α\alpha. However, the released code (patches/open-instruct_local_changes.patch, lines 50–52) also divides by the weighted token count, so α\alpha partly cancels. With the per-device batch sizes in Table 7 (Appendix H), α\alpha = 2 gives only about 1.6$\times$ weight for OLMo2-1B, and, if the same loss was used for OLMo2-7B (batch size 1), would have no effect at all there. Since the main claim compares rewriting against reweighting, this understates reweighting in the comparison.

Weakness 2: The paper is inconsistent about which templates were used for behavior-opposed rewriting. Table 10 (page 30) says the abstention-opposed rewrite uses the Abstention-answer pool, but Appendix H (lines 1454–1456) says a comply pool was used, and the released code (build/build_interventions.py, lines 27–29) uses the comply templates, which Table 10 lists for safety. The answer templates are included in the release but never used by default. As a result, it is unclear what supervision produced the weaken results in Fig. 2, 5, 9 and Table 9, and whether those templates are actually the opposite of abstention.

Questions for authors

Q1. The released OLMo2-1B training script (train/run_experiments.sh, lines 14 and 38) uses the weighted loss in patches/open-instruct_local_changes.patch (lines 50–52), which divides by the weighted token count. With a per-device batch size of 4, α\alpha = 2 then corresponds to only about 1.6$\times$ weight. Was the same loss used for Qwen3.5-2B, Gemma3-4B and OLMo2-7B? For OLMo2-7B (per-device batch size 1, Table 7), α\alpha would cancel completely. Does the conclusion that reweighting is weak still hold with a loss where α\alpha directly scales each example’s weight?

Q2. Table 10 says abstention-opposed rewriting uses the Abstention-answer pool (answer_templates_20.jsonl), but this file is never used by default. Instead, the code (build/build_interventions.py, lines 27–29) uses comply_templates_20.jsonl, which Table 10 lists as the Safety-comply pool, and Appendix H also mentions a comply pool. Which pool produced the reported abstention-opposed results, and do the results hold with the other pool?

Q3. IF-selected examples are more often abstention-related than the overall training data (Table 3: about 43–46% vs. 28%), so part of IF’s advantage over random selection could come from topic relevance. Apart from Probing (Fig. 5), the alternative selectors in Appendix C.3, including the topic-matched Judge Matched control, are evaluated only on OLMo2-1B (lines 1077–1079). Could the authors repeat these comparisons on the other three models (Qwen3.5-2B, Gemma3-4B, OLMo2-7B) to show that IF’s selection advantage holds across all four?

Q4. The paper’s motivation (lines 043–046) is that upweighting or removing IF-selected examples (such weight-based interventions) sometimes works no better than random selection. The two cited works give limited support for this. Dai et al. (2025) do not upweight or remove examples; they select a small subset to train on, and their main result is that an improved influence-based method (BIDS) beats random selection. Lee et al. (2026a) is a blog post, not a peer-reviewed paper, with no released code. It tests removal (and one content edit), finding removal no better than random for most behaviors but effective for refusal, which is close to the behaviors studied here. Neither work tests upweighting. Can the authors cite work that actually tests upweighting of IF-selected examples?

Reviewer_eNbvOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper tackles the ineffective intervention of current training data attribution methods on SFT of language models. They use influence functions to select intervention target data, then change the response to one that induces the target behavior, rather than reweighting the data as in standard influence functions. The paper assesses its intervention on epistemic abstention with four language models. The result shows that response rewriting has a greater effect on changing the target behavior than upweighting or removing the data from the training trajectory. Further, the result shows that selecting intervention target data using influence functions is better than random selection or other TDA methods. Further analysis shows that reweighting has an inconsistent effect on the target behavior, that selected data from influence functions tend to align with the behavior’s linear representation, and that the result extends to safety refusal.

Critical strengths
  • I think the paper is valuable in terms of showing how we could use influence functions on language models. To my knowledge, influence functions do not work well for steering the behavior, and this paper solves this problem by altering the intervention of influence functions. Although I think the method does not have theoretical support, it has huge practical utility.
  • The paper shows interesting analysis of their method. Figure 4 shows that influence-selected data aligns with the linear unanswerability direction in the representation space, and Figure 7 shows that the Recall difference comes from improvement on the target category.
Critical weaknesses
  • The paper should provide evidence on whether the improvement in Recall is due to a naive increase in abstention or to a truly improved abstention ability of the model. The paper reports precision in Table 4 of the appendix, and rewriting seems to underperform the random or baseline settings.
  • Figure 2 only provides confidence intervals for random interventions. Without considering the confidence interval of response rewriting, we cannot determine the statistical significance of the result.
Questions for authors
  • I think one possible reason why probing only works well for reducing abstention is that it already selects data whose responses are abstentions. Could the authors provide evidence on whether it is true?
  • Could the authors explain why precision is worse than the random baseline? Is this due to the proposed method naively increasing the abstention rate?
  • Is the performance increase in Figure 2 statistically significant compared to the random baseline?
  • Epistemic abstention is a behavior with clear target responses for rewriting. I’m curious how we could rewrite the response for behaviors without a clear target response, such as improving math-solving ability.
Reviewer_ZizTOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies whether training examples selected using influence functions are more useful when their responses are rewritten rather than reweighted or removed. The study is conducted in the context of SFT of LLMs, focusing on abstention and safety refusal. The core idea is to rewrite the responses using templates that encourage or discourage various behaviors. Through thorough experiments across four models, the authors show that rewriting produces stronger and more persistent changes in abstention behavior. They also show experimentally that influence-based selection paired with rewriting is superior to the other data-selection methods evaluated with rewriting.

Critical strengths

I think the insights from the experiments are useful. They show that influential examples for abstention and safety refusal can be more effective at steering these behaviors when their responses are rewritten rather than removed or reweighted.

Critical weaknesses

While I appreciate the empirical evaluation, I think the advantage of rewriting over deletion and rewiting is not surprising. Several related works support this intuition through relabeling and response replacement in other settings, such as classification, abstention tuning, and behavioral interventions in LLMs. So I would not consider the core insight particularly novel, surprising, or counterintuitive.

Also, the comparisons with stronger alternative selectors are limited to OLMo2-1B. The larger-model and cross-model transfer results are encouraging, but they do not establish whether influence-based selection retains its advantage over those alternatives at larger sizes.

The experiments also focus on abstention and safety refusal using generic templates, so they do not establish the same advantage for other tasks or instruction-specific corrections.

Questions for authors

Does influence-based selection retain its advantage over the stronger alternative selectors on larger models?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.