acceptodds
Under review as a conference paper at ICLR 2027

The Order Effect and the Order Imprint Are Not the Same Thing

Abstract

Post-training a language model with two capability operators in different orders produces measurably different models, and a growing line of work uses Lie-bracket surrogates to predict which order scores better. We show that this validation target is largely the wrong object. We compose two on-policy distillation operators (math and code teachers) from a shared start over six seeds, and find that one intervention—resetting optimizer state at the phase boundary—pulls apart two quantities the literature treats as one. It collapses the benchmark order effect: ΔJ_M falls from −0.0107 to −0.0038±0.0027, inside our preregistered indistinguishable-from-zero band, and uniform weight averaging of the two single-operator models matches the better order in any case. The same intervention leaves the parameter order imprint u = θ_MC − θ_CM within 4% of unchanged on every seed. An audit of that imprint shows that its "recency" share depends on a free parameter no prior work controls, namely where operator displacements are measured: 2–3% of ‖u‖² at initialization, 12–15% at the phase boundary. The rest survives every null we can build: it replicates across seeds (mean φ = 0.44), an SFT pair on the same data leaves a near-orthogonal imprint, random directions of equal norm are inert, and its geometry replicates on a 4B student. Benchmarks thus register mostly optimizer bookkeeping and miss the part of the order effect that replicates. We release the audit protocol as a measurement standard for composition claims. Code is available at https://anonymous.4open.science/r/pdc-2027.

Primary area:
foundation or frontier models, including LLMs
Keywords:
post-training, operator composition, order effects, on-policy distillation, model merging, weight averaging, optimizer state, reproducibility, evaluation methodology, language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_axYGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates how the order of sequential on-policy distillation affects benchmark performance and model weights. The authors find that resetting the optimizer reduces the benchmark order gap while largely preserving the norm of the parameter difference between the two training orders. They further analyze this difference, termed the order imprint, through several geometric and functional experiments.

Critical strengths
  1. I appreciate the attempt to look beyond benchmark scores when studying sequential post-training. The question of how different capability operators interact is relevant, especially with the growing interest in multi-teacher distillation and capability integration.

  2. The authors put considerable effort into experimental controls, including multiple seeds, null comparisons, and transparent reporting of unsuccessful experiments.

Critical weaknesses
  1. I am not fully convinced of the significance of the main finding. Benchmark performance and weight-space differences are inherently different quantities, so I would not expect them to correspond directly in the first place. Related work such as Neural Thickets [1] has already shown that task-dependent behaviors can vary considerably within a model's weight-space neighborhood. What remains unclear to me is what we actually learn from the order imprint beyond knowing that different training trajectories produce different weights. Does it tell us anything about capability interference or future adaptation? I would have liked to see a clearer motivation for why this imprint matters.

  2. Some conclusions seem stronger than what the experiments support. In Sec. 4, the authors claim that the imprint is largely preserved after optimizer reset. However, while its norm barely changes, the cosine similarity is only around 0.55, suggesting a considerable directional change. The reduction in benchmark order effect is also relatively weak statistically (around 1.5 standard errors for the paired reduction). I am also not entirely convinced by the interpretation of the unexplained imprint in Secs. 5–6, since the explained proportion depends heavily on the chosen reference directions.

  3. The connection to practical capability integration is limited, and the presentation could be clearer. Recent work such as MOPD [2] investigates how different teacher capabilities can be integrated into one student. I understand that this paper focuses on analysis rather than improving training, but I find it difficult to see what the measured imprint tells us about capability composition in practice. The paper was also difficult to follow despite careful reading. In particular, the motivation for moving from benchmark effects to parameter-space analysis is not clearly established, and several interpretations are introduced before their significance becomes clear.

[1] Gan et al. "Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights", ICML 2026.
[2] Ma et al. "MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training".
Questions for authors

Q1. What does the order imprint tell us beyond the fact that different training orders lead to different weights? Why is this distinction important in practice?

Q2. Have you considered training both order endpoints on a third task to see whether their different imprints affect future learning or capability retention?

Q3. If the imprint is largely preserved after optimizer reset, how should we interpret the relatively low cosine similarity (~0.55)? Could the unexplained component simply reflect differences between the two training trajectories?

Reviewer_5Pr6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the effect of training order when tuning models on different tasks (math and code in this paper) and finds that resetting the optimizer state at the phase boundary largely removes the benchmark order effect, while the parameter-space imprint remains nearly unchanged. Besides, uniformly averaging the two single-task models performs comparably to the better training order. The results suggest that benchmark scores alone may not capture the persistent differences introduced by training order.

Critical strengths
  1. It points out the distinction between benchmark effects and parameter-space imprints.
  2. The explored question is interesting.
Critical weaknesses
  1. The study covers only one task pair, so generalization is uncertain.
  2. It does not analyze whether task updates conflict or are nearly orthogonal, which could help explain the weight-averaging result.
  3. Weight averaging is evaluated at limited training budgets, and its effectiveness with longer training is unclear.
  4. The authors could not measure the contribution of on-policy occupancy to the imprint.
Questions for authors
  1. Can the main findings be tested on additional task pairs?
  2. Can the authors analyze how task-update similarity relates to the success of weight averaging?
  3. Does weight averaging remain competitive at longer training budgets?

If these questions are well addressed or explained, I will consider raising my score.

Reviewer_g151OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the distinction between benchmark-level training-order effects and parameter-space differences induced by sequential post-training. Through experiments on mathematics and code tasks, the authors show that resetting optimizer states substantially reduces benchmark-level order effects while leaving the norm of parameter differences nearly unchanged. Further analyses examine the structure, reproducibility, and functional consequences of these differences. The paper argues that benchmark performance alone may be insufficient to characterize the effects of training order.

Critical strengths

The paper addresses an interesting and relevant research question. The distinction between performance changes and parameter-space differences is meaningful, and the experimental design includes several useful controls, including optimizer-state resetting, cross-seed analyses, and parameter perturbation experiments. The authors also transparently report unsuccessful analyses and acknowledge important limitations.

Critical weaknesses
  1. Limited generalizability. The main experiments focus on a single task pair, mathematics and code, under one OPD setup. It remains unclear whether the observed dissociation generalizes to other tasks, training paradigms, optimizers, or model families. This limits the strength of the paper’s broader claims.

  2. The interpretation of imprint preservation requires greater care. The parameter-norm ratio of 1.007 indicates that the original and reset imprints have similar magnitudes, but the reported directional cosine of 0.53–0.57 suggests only partial alignment. Therefore, the claim that the imprint survives optimizer resetting should distinguish preservation of magnitude from preservation of direction and functional structure.

  3. Statistical evidence for the main conclusion is not fully convincing. Although the benchmark-level order effect decreases after optimizer-state resetting, the authors acknowledge that the paired difference is not statistically significant under the reported analysis. Consequently, the claim that optimizer-state carryover accounts for approximately two-thirds of the original effect should be interpreted cautiously.

  4. The mechanism remains insufficiently explained. The paper identifies a reproducible parameter-space difference, but the origin of its substantial unexplained component remains unclear. In particular, the occupancy-distribution analysis does not provide a reliable quantitative estimate. The results therefore establish an interesting empirical phenomenon more convincingly than they explain its underlying mechanism.

Questions for authors
  1. Interpretation of imprint preservation: The norm ratio between the original and reset imprints is close to 1.0, but their directional cosine is only approximately 0.53–0.57. What evidence supports the conclusion that the parameter-space imprint is preserved beyond its magnitude? Do the two imprints exhibit similar functional effects?
  2. Statistical robustness: The paper reports that the benchmark-level order effect decreases after optimizer-state resetting, but the paired difference is not statistically significant with five seeds. How robust is the conclusion that optimizer-state carryover explains approximately two-thirds of the original order effect? Can the authors clarify the uncertainty associated with this estimate?
  3. Scope of the main claim: Since the primary experiments focus on one model and one task pair, what evidence supports extending the distinction between benchmark-level order effects and parameter-space imprints beyond this particular setup? Is the exploratory 4B replication sufficient to support any broader claims?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.