- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper explores a connection between fine-tuning and in-context learning. They show that having a certain library of sources and a retriever, and using some of the retrieved content in the prompt, can theoretically recover a fine-tuned model whose parameters have changed.
Critical strengths
- I believe this paper offers what a lot of other submissions do not, and that is clarity. The paper is clear to read, and the illustrative figure helps a lot. The math is clear and intuitive, and the terms defined, like induced shifts and oscillation, are well-defined. The assumption about bilinear decomposition is clearly stated. The authors also mention the failure cases.
- The theoretical result, specifically Proposition 1, carrries not only mathematical rigor but also key insights, shown by the the three terms that outline how to inform in-context alignment, namely the fitness of the clause, the relaization of the inted effects of those clasues, and divesity of the rules.
- the experiemnts give a hint towards the validity of the method.
Critical weaknesses
- The experimental results are limited, and I cannot exactly capture the benefits of this method from one example mentioned in the figures, especially while using an oracle context. We would be more interested in a more realistic context that is not guaranteed to be perfect. I also think we need more diverse datasets and applications.
- The theoretical results have strong assumptions about the linearity of the attention heads.
- I don't think the lietratre was sufficiently addressed, and I would liek to know how the paper's proposed problem is different from what is in [1], [2], [3], [4], [5], [6].
[1] https://aclanthology.org/2023.findings-acl.779/
[2] https://aclanthology.org/2026.acl-long.1932/
[4] https://aclanthology.org/2024.findings-emnlp.182/
Questions for authors
- Between fine-tuning and in-context learning, there is a family of methods that do inference-time alignment without modifying the weights of the model, as in controlled decoding [1], [2], [3]. Can we recover the same performance of controlled decoding using in-context learning as well?
- How much overhead does the retriever addition add to the inference latency?
[1] https://arxiv.org/abs/2310.17022
6 comments
Linear Attention Assumption
The reviewer remarks that our theoretical results have strong assumptions about the linearity of the attention heads. However, the core of our results, i.e., the decomposition of the approximation error and its connection to self-distillation, does not assume linear attention heads. The results hold for general transformers. Only the KV-prefix section does make the linearity assumption, and it serves as theoretical motivation rather than as a main contribution. To make this clearer, we have moved most of the KV-prefix section, including its attention-linearity assumption, to the appendix. The main text keeps only its key findings, which motivate approximating training updates with clauses in context. We thank the reviewer for bringing this to our (linear) attention.
More diverse datasets and applications
The reviewer requests evaluations on more diverse datasets and applications than the initial synthetic setting. In addition to our initial dataset, we introduce one more synthetic setup (logic riddles) as well as a realistic alignment setup that involves training on SafeChain and evaluating on a SafeChain hold-out set, as well as HarmBench, StrongREJECT, and XSTest (↓) for safety, and MMLU-Pro, MATH-500, and GPQA-Diamond (↑) for reasoning and general capabilities.
Thus, we have expanded our investigations from one to eight datasets spanning harmfulness, overrefusal, reasoning, general knowledge, and logical in-context learning evaluations.
Crucially, we observe that the explanatory power of our theory holds across the tested settings, as detailed here: https://openreview.net/forum?id=FDitFfgFgK¬eId=If8lLMbdGL.
Empirical Validation of the Approximation Error Decomposition: Bound Tightness, In-Context Safety Alignment, and Constitutional On-Policy Self-Distillation
TL;DR
The reviewer appreciates both the clarity and insight provided by the core result of our paper (approximation error decomposition), but also requests more empirical evaluations to establish the results’ explanatory and predictive value. To this end, we have expanded our experimental evaluation, which now comprises three complementary parts:
1. Relationship between context and training updates. We extend the experiment from our initial submission, illustrating the overall relationship between fine-tuning and context-induced distribution shifts with an experiment to measure the tightness of the proposed theoretical bound and the impact of the different error terms.
2. Explanatory power in realistic alignment. We isolate the effects of the three approximation errors on a held-out SafeChain split and show that controlling them allows context-induced performance to approach, and under privileged information even exceed, the corresponding SFT performance in safety alignment.
3. Connection to on-policy self-distillation. We extend our theoretical analysis to characterize when self-distillation can recover a desired target policy despite the context-induced teacher only approximating it. This establishes a direct connection to our error decomposition and explains how controlling the individual errors can improve constitutional self-distillation. For more details on the theory see here: https://openreview.net/forum?id=FDitFfgFgK¬eId=YozD0dRtL9.
Below we summarize the main new experimental results, which will all be included in the revised paper.
New empirical results
1. The empirical tightness of the theoretical bound on DR.
We empirically measure the three terms of our approximation error decomposition and find that the proposed upper bound becomes increasingly tight with model performance, supporting its informativeness. For Llama-3.1-70B and Qwen3.8-27B the bound lies 0.3 to 0.6 log-odds above the measured error. For comparison, in our experimental setup, a model at chance is 2.94 away from the correct answer. Thus, the bound is tight for strong models. For more details on the experiment, see here: https://openreview.net/forum?id=FDitFfgFgK¬eId=C1Jq0W8gcn.
2. Approximation error decomposition in real alignment with imperfect context
We additionally observe the impact of all three approximation errors on a held-out SafeChain split. All three have statistically significant effects (p < 0.001 for both model sizes): improving clause selection increases composite overrefusal and safety performance by +5.51/+3.31 pp, reducing clause interactions results in improvements of +2.52/+2.13 pp, and providing privileged interpretation support leads to +8.06/+3.34 pp higher safety performance (1.7B/8B, respectively, safety & overrefusal measured by Qwen3Guard on a SafeChain). For more details on the experiment, see here: https://openreview.net/forum?id=FDitFfgFgK¬eId=YwvXWcNlfE.
3. Explaining improved self-distillation from controlled in-context approximation
Tables 1 and 2 demonstrate the practical implications of our decomposition for on-policy self-distillation. Both Qwen3 1.7B and 8B are trained on SafeChain using teacher configurations that ablate the effects of controlling context approximation errors. All models, including the SFT baseline, are trained with a batch size of 64 and 350 training steps. We then evaluate the trained models on HarmBench, StrongREJECT, and XSTest (↓) for safety, and MMLU-Pro, MATH-500, and GPQA-Diamond (↑) for reasoning and general capabilities, all at pass@1 and the same generation parameters. The tables report the corresponding safety, reasoning, and overall composite scores. Controlling the teacher's approximation errors improves the overall composite by +4.78 pp (1.7B) and +2.55 pp (8B) compared with uncontrolled self-distillation. Crucially, the resulting policies also outperform the corresponding SFT baselines by +2.85 pp and +2.93 pp, respectively. For more details on the experiment, see here: https://openreview.net/forum?id=FDitFfgFgK¬eId=YwvXWcNlfE.
Table 1. Qwen3-1.7B: Performance after self-distillation.
Table 2. Qwen3-8B: Performance after self-distillation.