acceptodds
Under review as a conference paper at ICLR 2027

Back to the Weights: Training-Free Model Enhancement via Training Trajectory Extrapolation

Abstract

Fine-tuning large language models with cosine learning-rate decay drives the step size to near zero long before the model reaches a true optimum. The training direction —an aggregate of thousands of gradient steps—still points toward lower loss, yet the vanishing learning rate prevents the model from reaching it. This gap is not an artifact of a particular run but a structural consequence of the schedule, leaving exploitable structure in the weight space beyond the training endpoint. We present TrajectoryForge, a training-free framework that closes this gap: given only two checkpoints ( and ), it extrapolates along with per-matrix stochastic scaling factors, producing an improved single model without any gradient computation or inference overhead. Across 7 model pairs (1.5B–14B parameters), directed perturbation matches or exceeds the fine-tuning endpoint for 6 of 7 models, with gains up to +7.1% on HumanEval. Hessian spectral analysis reveals that the training direction occupies a geometrically privileged “sharp valley,” enabling the success rate of isotropic random search. Cross-task evaluation further confirms a direction-task alignment principle: improvement on aligned tasks with no degradation on orthogonal ones, and large-scale held-out validation (529 candidates on MBPP) confirms that models selected without target benchmark information still exceed the fine-tuning endpoint.

Primary area:
foundation or frontier models, including LLMs
Keywords:
training trajectory extrapolation, post-hoc model optimization, weight-space optimization, large language models, training-free model enhancement, model merging, loss landscape geometry

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LKWSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TrajectoryForge, a gradient-free method for improving a released model using two compatible checkpoints. It computes their displacement, decomposes it into weight-matrix components, and samples candidate models by extrapolating each component with an independent nonnegative half-normal coefficient. Candidates are ranked using benchmark performance, and the selected candidate is deployed as a single model without additional inference-time overhead.

Critical strengths
  1. The method addresses a useful and realistic deployment setting.
  2. The paper provides evidence beyond reporting a single favorable maximum. The positive candidate means for Qwen-Coder-1.5B and DeepSeek-Coder-6.7B are more informative than best-of-NN improvements alone.
Critical weaknesses
  1. The comparison does not clearly establish the benefit of the training direction. Using the same perturbation scale does not necessarily give directed and isotropic search comparable perturbation magnitudes. The methods also explore different search spaces, making it difficult to identify what causes the reported advantage. Furthermore, the scale-dependent angles in Table 12 appear inconsistent with the stated sampling procedure, weakening the explanation of an angular sweet spot.
  2. The generalization evidence is limited, and some reported statistics are inconsistent. Most improvements rely on selecting the best candidate using the evaluation benchmark itself. Only two model pairs improve on average, and cross-benchmark selection is demonstrated on one pair. Additionally, the confidence intervals for the mathematical models in Table 11 do not agree with the reported baselines and means. These inconsistencies need clarification before the empirical conclusions can be assessed reliably.
Questions for authors

Please see the critical weaknesses.

Reviewer_UdaMOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TrajectoryForge, a gradient-free method for improving a fine-tuned model. It extrapolates beyond the fine-tuned checkpoint along the displacement from an earlier checkpoint, samples separate nonnegative scaling coefficients for different parameter matrices, and selects a candidate based on benchmark performance. Experiments cover seven model pairs and include distributional results, cross-benchmark evaluation, and geometric analyses. Two model pairs improve at the mean level, while most headline results concern the best candidate found through search.

Critical strengths

S1. A simple and practical gradient-free approach. The method requires only two compatible checkpoints, avoids backpropagation, and introduces no additional inference overhead after candidate selection. This makes it an accessible approach to post-training model improvement.

S2. Some evidence goes beyond selecting a single favorable result. Two model pairs improve at the mean level. A candidate selected using only MBPP also improves HumanEval by 2.4 percentage points. These results suggest that the approach can produce useful improvements in some settings.

Critical weaknesses

W1. Most headline gains lack independent final evaluation. The main best-of-N results use the same benchmark to select a candidate and report its performance. These estimates are therefore affected by selection bias, especially when many candidates are evaluated. The authors acknowledge this limitation and provide one cross-benchmark example, but the evidence remains insufficient to establish generalization across the evaluated models. This is my main concern because it directly affects the interpretation of the reported improvements.

W2. The efficiency of the proposed search strategy is not sufficiently established. Avoiding gradients does not necessarily make the complete search inexpensive, since many candidate models must still be evaluated. Related coefficient-selection approaches, including gradient-based learning in AdaMerging https://arxiv.org/abs/2310.02575 and Bayesian optimization in ReForge https://arxiv.org/abs/2605.12843 , motivate a clearer justification for random sampling here, although these methods would require adaptation to this setting. A matched-budget comparison with adaptive search, such as Bayesian optimization, would help establish the value of random coefficient sampling. The released materials also lack the complete candidate records and analysis code needed to independently verify all reported results.

Questions for authors

Q1. Beyond the reported MBPP-selected candidate, do you have results that strictly separate candidate selection from final evaluation across multiple models? Please specify which data were used for selection and which were reserved for final reporting.

Q2. How does random coefficient sampling compare with an adaptive search method under the same parameterization, search space, and evaluation budget? Please also clarify the total search cost, including candidate construction and evaluation.

Q3. Can you provide the candidate-level scores, selection configurations, and missing analysis scripts needed to reproduce the main tables and geometric analyses?

Reviewer_53D4OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates how to further improve the performance of fine-tuned large language models without additional gradient training, relying solely on the parameter differences between checkpoints before and after fine-tuning. A gradient-free model enhancement framework is proposed. The method first computes the weight difference between the initial model and the fine-tuned model, treating it as the parameter change direction formed during training. Subsequently, non-negative scaling coefficients are independently sampled for different weight matrices to extrapolate along the corresponding weight change directions, constructing multiple candidate models. The candidate with the best performance on downstream tasks is ultimately selected.

Critical strengths
  1. Requires only two compatible checkpoints (no original data, optimizer states, or gradients needed) and deploys only a single final model without inference overhead.
  2. Tested across multiple architectures, scales, and tasks, with evidence of cross-benchmark generalization (e.g., selecting on MBPP and improving on HumanEval).
  3. Extends global extrapolation to matrix-wise perturbation search, allowing different Transformer components to have distinct extrapolation magnitudes.
Critical weaknesses
  1. The core idea (leveraging training trajectory structure for weight-space extrapolation) has been widely explored in prior works (e.g., Extra-Merge, Linear Dynamics, AlphaRL, RELEX). The paper lacks a systematic comparison and discussion against these, making the main novelty boil down to the matrix-wise random scaling strategy. The authors need to clarify the specific advantages over deterministic or low-rank trajectory prediction.
  2. Only 2 out of 7 model groups improved in average random candidate performance, meaning gains heavily rely on picking the best-performing candidate from a large pool. For instance, CodeLlama-7B-Python's average performance dropped significantly, showing the method is not universally stable. Evaluating hundreds of candidates on small benchmarks (like HumanEval) also risks selection bias.
Questions for authors
  1. Clarify specific differences and advantages over Extra-Merge, Linear Dynamics, AlphaRL, and RELEX, especially compared to deterministic PCA/low-rank extrapolation. Provide a systematic discussion or empirical comparison under the same checkpoints.

  2. Explain the mathematical definition and calculation of the equivalent angle in Appendix K. Since the half-normal sigma should only scale vector length and not direction, why do the reported angles change with sigma?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.