acceptodds
Under review as a conference paper at ICLR 2027

How Test-Time Updates Become Behavior: Objectives, Training Scope, and Update Paths

Abstract

Can large language models turn information in deployment-time support into behavior that remains accessible after the support is removed? We study a Support-Derived Behavioral Objective (SDBO) that constructs one question–answer target from the support and a fixed schema, then takes four local stochastic-gradient steps. On 32 independent QAEdit confirmation episodes, SDBO improves joint attained success for Llama-3.1-8B-Instruct over raw-support training by 37.5 percentage points at both primary KL budgets. Paired analyses establish an objective-by-training-scope interaction; content controls and held-out formulations show that the effect depends on the correct support content and extends beyond the training wording. Two fixed update policies achieve positive native-query net gains and pass relative-preservation non-inferiority tests. To probe why scope matters, we decompose full-model updates and intervene on 32 separate fresh episodes. Scaling each full-update projection to the effective radius of its independently trained local counterpart leaves a 62.5-percentage-point behavioral gap at both policy points. Path comparisons further support a learning-rate-driven multistep contribution. The complete gain-and-preservation pattern is not reproduced on Qwen3-8B-Base, marking a model-setting boundary. These results show that final parameter support and update magnitude do not determine behavior: the process that forms the update also matters.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time training, large language models, behavioral generalization, training scope, optimization dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.71 over 7 reviews, above 25% of the others.
Reviewer_Qw6LOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies whether a language model can learn information from deployment-time support and retain the corresponding behavior after the support is removed. The authors propose Support-Derived Behavioral Objective (SDBO), which converts a support statement into a canonical question-answer target and performs four SGD updates on the down-projection matrices of the last eight layers of Llama-3.1-8B-Instruct. Experiments on QAEdit compare SDBO with raw-support training and a four-view objective under local and full-model parameter scopes. The authors report that local SDBO improves behavioral success over raw-support training on Llama-3.1-8B-Instruct, while the effect does not reproduce on Qwen3-8B-Base. Through additional projection, radius-matching, and optimization-path experiments, the paper further argues that post-update behavior depends on how an update is formed rather than only on its final parameter support or magnitude.

Critical strengths
  1. The authors remove the support after the update and evaluate the resulting model instead of relying only on training loss or teacher-forced likelihood. This provides a more meaningful test of whether the information has actually affected the model's behavior. I also appreciate that the negative result on Qwen3-8B-Base is reported.
Critical weaknesses
  1. I do not see a clear methodological contribution in SDBO. The method trains on a single question-answer pair derived from the support and computes the loss only on the answer tokens. The experiments are also conducted in a single-fact editing setting on QAEdit, but SDBO is mainly compared with variants designed by the authors themselves. There is no comparison with existing editing methods such as FT-M, ROME, or MEMIT, which have already been evaluated on QAEdit by Yang et al. (2025). I also find the ablations insufficient. Several design choices are bundled together in SDBO, such as converting the support into a canonical QA format and computing the loss only on the answer. The paper does not separate these choices, so it is hard to tell what actually causes the improvement. The authors' four-view objective O1 performs better, but one of its training prompts is identical to the acquisition probe in every released episode, so this comparison is hard to interpret. I think these comparisons and ablations are more important than the extensive statistical analysis currently presented in the paper. Without them, it is difficult for me to understand what SDBO contributes over existing methods, and this is the main reason for my recommendation.

  2. I also find the explanation of the main result unclear. Sections 6 and 7 emphasize the importance of the training scope, but Section 7.3 shows that the learning rate has a very large effect. The SDBO learning rates used for the local and full-model settings differ by roughly 300 times after KL calibration. In the same-learning-rate comparison, there is no observed advantage for the local setting over the full-model setting, but both settings perform poorly. Therefore, I do not think this experiment clearly separates the effect of training scope from the effect of learning rate and the resulting optimization path. Table A8 also makes the interpretation less clear: on the replay prompts, the component of the full-model update outside the late layers retains almost all of the original KL, while the projected late-layer component has almost none. Overall, I found it difficult to determine exactly what these experiments establish about training scope. The large number of labels and newly introduced terms, such as S0, S3, OB, and Full-slow, also makes this analysis unnecessarily difficult to follow.

Questions for authors
  1. Appendix C.1 shows that the transform T parses a fixed support template with a predefined relation schema, and all 104 supports in the released data follow this format. What happens when the support does not follow this template? For example, can SDBO handle a differently phrased fact or a task rule of the kind mentioned in the introduction?

  2. The paper states that evaluation queries never enter target construction. However, the relation label required by T does not appear to be directly provided by the public QAEdit records. How were these relation labels constructed? Did this process use any information from the evaluation questions?

  3. I am unclear about what the wrong-target control in Table 2 establishes. The primary scorer checks whether the correct target value or an alias appears in the generated answer, while the wrong-target condition trains the model on a different value. In the released results, the Llama wrong-target states have zero gain even when the corresponding correct-target SDBO states succeed frequently. How often does a wrong-target model actually generate the wrong value that it was trained on? Without this information, how should the correct-vs-wrong contrast be interpreted as evidence of content specificity?

  4. I am also unclear about the false-activation test. In the released episode files, the four same-domain negatives are synthetic prompts such as "The unchanged archive lists item Z2 with value quartz. What value is listed for Z2?", which seem largely unrelated to the edited fact. How does success on these prompts demonstrate that the update remains local? In particular, what happens to other facts about the same subject or other facts involving the same relation?

Reviewer_XY26OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper puts forward SDBO, a novel approach for test-time training through prompt formatting, local gradient updates and local parameter updates. It achieves better performance than raw-support training on llama models. Further experiments reveal local parameter training is not merely a projection of full parameter update.

Critical strengths

The problem setup of test-time training is important.

The method of prompt reformatting is intuitive and interesting.

Critical weaknesses

The main concern is significance issues. The method seems to be very technical and thus the novelty is limited. The generalization across model families is also weak which is acknowledged by the authors in section 6, so I'm concerned whether the prompt format is model-specific and requires careful tuning for each model.

The representation of the paper also needs to be improved. All figures contain huge amount of information without careful arranging them and the captions are not intuitive, leaving the results difficult to read. What's more the complicated formulas in section 3 are defined but doesn't seem to be important in the analysis.

Questions for authors
  1. Can the authors provide a fair comparison result over different methods(e.g. raw content vs formatted content, full vs local parameter SFT , and other test-time training methods), different datasets or domains, different model families, all under best hyperparameters and same data budget, so we can have a broad picture of the significance and limitations of this method?

  2. The observations in section 7 is interesting but lacks analysis or explanation. Can the authors explain why restriction is different from projection, or explain on which settings should restriction be better than projection and on which settings it is the other way around?

1 comment

Thank you for your follow-up experiments. I believe that your method is interesting as it is intuitive and does help training in some setups.

However, my core concern remains that the significance is limited as you did not provide enough evidence or intuition that under what conditions should this method hold. Of course it may not work in every scenerio, but I expect a claim like "SDBO has advantage on some certain domain, this is stable across models, data format and some other unimportant factors". In this case the method can be seen as generally usable. However, in current scenerio we can only conclude "SDBO may be useful on some data and model", which greatly limits the potential application of this method. The gain over vanilla SFT is not so surprising to me as SFT on small, unorganized data automatically causes huge performance degrade, especially given your prompt is carefully tuned but the other baselines are not. If you cannot find a method to deal with this problem, others may easily conclude that your core contribution is a prompt tuning experiment on specific model which is not very interesting.

Concerning the problem on restricting and projection, I can see their mathematical difference, but I wonder if there is a intuitive and general understanding on this, for example what model behaviors do restricting scope during training and full finetuning affect respectively.

Due to above concerns my score remains unchanged.

Reviewer_zjErOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how training objectives, parameter scope, and optimization paths affect whether information learned through test-time updates remains usable after the support is removed. It proposes a Support-Derived Behavioral Objective (SDBO), which constructs a single question–answer target from the support and updates selected MLP parameters with four SGD steps. On 32 independent QAEdit confirmation episodes, SDBO improves joint behavioral and preservation success over raw-support training for Llama-3.1-8B-Instruct, with validation through content controls, held-out query formulations, and fixed-policy preservation tests. Projection, norm-matching, and update-path experiments further suggest that learning-rate-dependent optimization paths contribute to behavioral differences beyond final parameter scope and update magnitude. The complete gain-and-preservation pattern is not reproduced on Qwen3-8B-Base.

Critical strengths
  1. The paper evaluates free generation after support removal and checks preservation. Independent confirmation, content controls, and held-out queries strengthen confidence in the reported Llama gains.

  2. Projection and norm-matching experiments investigate why local and full-model updates behave differently. Learning-rate comparisons further examine the optimization path. These analyses provide explanatory value beyond the performance gains.

Critical weaknesses
  1. The main effectiveness result rests on 32 confirmation episodes. The full gain-and-preservation pattern is not reproduced on Qwen3-8B-Base. This leaves generalization across model settings weakly supported.

  2. The four-view reference achieves higher joint success than SDBO in the main Llama setting. The paper does not quantify an efficiency advantage for SDBO. Its practical benefit over this alternative therefore remains unclear.

Questions for authors
  1. Could the Llama–Qwen difference reflect instruction tuning or prompt format? Is there existing evidence that separates these factors from model-family effects?

  2. How do SDBO and the four-view reference compare in training cost and latency? Does SDBO offer a measurable efficiency benefit that offsets its lower joint success?

Reviewer_7ZqsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether a model can use newly acquired information after that information has been removed from its context. SDBO converts the support into a question–answer target and performs four local parameter updates. Through comparisons involving raw-support training, incorrect targets, different update scopes, and rephrased questions, the authors observe behavioral gains in the Llama configuration and test relative preservation under fixed update policies. Projection, norm-matching, and trajectory experiments further show that matching the final parameter support and update norm does not reproduce the effects of independently trained local updates. The full positive result is not reproduced in the Qwen base-model configuration.

Critical strengths

The paper’s strongest aspect is its careful distinction between fitting information and being able to use it. Evaluating free generation after removing the support, together with incorrect-target controls, rephrased questions, and independently constructed queries, gives the observed improvements a clear behavioral interpretation.

I also appreciate the care taken in organizing the experiments. The authors separate discovery from independent confirmation, use fresh examples for mechanism tests, and distinguish performance attainable over a configuration set from the performance of a fixed policy. Numerical reconstruction checks and explicit reporting of failure boundaries further help establish which interpretations are supported.

Critical weaknesses

The positive results are concentrated in a relatively narrow model and task setting. The main confirmation set contains 32 QAEdit episodes, while the Llama–Qwen comparison changes model family, instruction-tuning status, and prompting format simultaneously. The authors appropriately describe this as a model-setting boundary, but it remains unclear which conditions are necessary for the method to work. This limits my confidence in how broadly the findings transfer.

The practical operating range of the fixed policies also needs further characterization. The relative non-inferiority result against raw-support training is clearly stated, but absolute locality holds for only about two-thirds of episodes, and some episodes exceed the KL budget. The authors disclose these outcomes. My remaining concern is whether the resulting trade-off between behavioral gains and preservation meets the requirements of a plausible deployment setting.

Questions for authors

What motivated the choice of model versions and prompting interfaces for Llama and Qwen? Are there existing controls that help separate the effects of model family, instruction tuning, and prompting format, and thereby clarify the conditions under which SDBO works?

For each already-prespecified fixed policy, how many episodes show behavioral gains while satisfying both absolute locality and the per-episode KL constraint? Please keep this analysis separate from post-hoc configuration selection, as it would directly affect my assessment of the method’s practical usefulness.

Reviewer_15D6OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies whether few train steps on test time support can change models’ generation behavior after the support is removed. For this, they propose SDBO, a framework that converts the support into a question-answer target and trains only the down_proj matrices. Testing on QAEdit episodes, SDBO shows 37.5 percent points improvement compared to raw-support training.

Critical strengths

This paper use fair experimental setup. For example, the set the budget with KL which makes more sense than using learning rate.

Critical weaknesses

The paper is extremely hard to understand and does not clearly state the practical implication of its finding.

To summarize their finding,

  1. When we input information in test-time learinig, do not use raw text.
  2. Alter some of MLP not the entire model parameter.
  3. We can’t assess the success of learning only with loss.
  4. Do not use part of layers it after FFT.

For the finding 1-3 I think these are already covered in the previous work [1,2,3,4]. In addition, in practice, it’s quite rare to use part of the layers.

[1] Self-Supervised Test-Time Learning for Reading Comprehension https://arxiv.org/abs/2103.11263

[2]Mass-Editing Memory in a Transformer

https://arxiv.org/abs/2210.07229

[3]The Mirage of Model Editing: Revisiting Evaluation in the Wild https://arxiv.org/abs/2502.11177

[4]Locating and Editing Factual Associations in GPT https://arxiv.org/abs/2202.05262

Questions for authors

[Q1] The paper that introduced QAEdit reports results of existing knowledge-editing methods such as ROME and MEMIT under the WILD evaluation framework. Can you compare SDBO with these editing methods on the same episodes?

[Q2] The effectiveness is shown in only single model Llama-3.1-8B-Instruct. While they also use Qwen3-8B-Base model, the results is not reproduced with that model.

Reviewer_ZSssOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether information provided at test time can be incorporated into an LLM's parameters and remain accessible after the original support is removed. The authors propose a Support-Derived Behavioral Objective (SDBO), which converts support information into a single question-answer target and performs four SGD steps on selected model parameters. Experiments compare different training objectives and parameter scopes on Llama-3.1-8B-Instruct and Qwen3-8B-Base. SDBO substantially improves behavioral success over raw-support training on Llama while maintaining relative preservation. Additional projection, radius-matching, and optimization-path experiments suggest that the way an update is formed matters beyond its final parameter support and magnitude. However, the main improvements do not generalize to Qwen3-8B-Base.

Critical strengths
  1. The paper investigates an important distinction between fitting test-time information and actually making it accessible through subsequent generation. The experiments go beyond training loss, using support-removed evaluation, held-out query formulations, and content-specific controls. The independent confirmation experiments also make the reported improvements more convincing.
  2. The comparison between independently trained local updates and projected full-model updates is interesting. The additional radius-matching and optimization-path experiments provide useful evidence that the final parameter support and update magnitude alone cannot explain behavioral outcomes.
Critical weaknesses
  1. The main positive results are obtained on only 32 confirmation episodes using structured single-fact updates and Llama-3.1-8B-Instruct. The same improvements do not appear on Qwen3-8B-Base. Since these models also differ in instruction tuning and prompting, it is difficult to understand what causes this difference. More importantly, it remains unclear whether the proposed approach would work with more natural, unstructured, or complex support information.
  2. The advantages of the proposed objective and the mechanistic interpretation are not fully established. The four-view reference objective (O1) actually achieves higher joint success than SDBO in several settings (e.g., 62.5% vs. 50.0% at KL budget 0.10), raising questions about the practical advantage of the single-target objective. Moreover, local and full-model training use substantially different learning rates, and the same-rate comparisons suggest that much of the observed difference is driven by the learning-rate choice. The radius-matching experiments also leave actual KL unmatched, making it difficult to fully isolate the role of update direction.
Questions for authors
  1. The four-view objective O1 achieves better joint success than SDBO in several experiments. What is the main advantage of using a single support-derived target? Could the authors provide a comparison of computational cost and preservation trade-offs?
  2. Could the authors further investigate why SDBO fails to produce similar improvements on Qwen3-8B-Base? In particular, is the difference related to instruction tuning, prompting format, or model architecture?
  3. In the direction-radius experiments, the projected updates still have substantially lower actual KL after radius matching. Would the conclusions remain the same if the updates were compared at similar actual KL levels rather than only matching their parameter-space norms?
Reviewer_538uOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper turns a fact update into one QA target and takes 4 SGD steps on a few MLP layers. This shows a better accuracy than training on the raw text (+37.5 pp). Training the full model does not work. Projecting or rescaling the full update does not help either.

Critical strengths
  • Careful statistics. Separate discovery and confirmation sets.
  • Measures real behavior after the support is removed.
Critical weaknesses
  • Learning rates are set to match KL. So the full model gets about 3e-6 and only 4 SGD steps. Its update is about 30x smaller. It may simply not learn the fact. The path and rescaling results can be explained the same way.
  • SDBO is close to standard QA fine-tuning (FT-M, LIFT). There is no comparison with ROME, MEMIT, AlphaEdit or LoRA editing, even though QAEdit is an editing benchmark.
  • The effect is gone on Qwen3-8B.
Questions for authors
  1. Does the full model fit the SDBO target after 4 steps?
  2. Does the scope effect hold with Adam or more steps?
  3. How does SDBO compare with editing methods?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.