acceptodds
Under review as a conference paper at ICLR 2027

The Delta Is Not the Skill: Zero-Prior Elicitation Transfers LoRA Behavior Across Model Families

Abstract

Reusing low-rank adapters (LoRA) across model sizes, generations, and architectural families is increasingly essential for deploying specialized capabilities. However, native retraining requires inaccessible training data, while existing synthetic-data transfers rely on task-specific priors (task names, seed examples, labels, or auxiliary discriminators). Weight-space transfer avoids these priors, but we uncover a sharp dichotomy: within a shared parameter basis, zero-edit adapter transplantation is near-lossless (recovering – of native gains), whereas across distinct families, weight-space routes collapse () because low-rank deltas are base-relative. What survives across model boundaries is behavior, not weights. We introduce *zero-prior skill elicitation*, a framework that extracts adapter capabilities without task identities, seed data, labels, discriminators, or white-box access. By identifying inputs where adapter and base generations diverge, evolving these probes to broaden coverage, and distilling the adapter-equipped teacher's completions into students, we transfer complex skills purely through text. Distilling from a single 397B MoE teacher LoRA matches or exceeds native supervised fine-tuning across four student models in two architectural families, reducing target training compute by –. Gains are pronounced on multi-step reasoning (elevating 9B MATH-500 by points; ), competition mathematics, and out-of-distribution instruction following, with zero verbatim -gram benchmark leakage and successful generalization to Granite models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Low-Rank Adaptation, Cross-Model Transfer, Zero-Prior Elicitation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UwiiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates transferring LoRA capabilities across different model families. The authors first show an interesting observation that direct LoRA weight transfer works well between models sharing a parameter basis but largely fails across different families. Motivated by this finding, they propose a zero-prior skill elicitation framework that identifies behavioral differences between a base model and its LoRA-adapted version, evolves informative probes, and distills the resulting responses into student models. Experiments show promising improvements over native SFT and existing transfer baselines. While I find the analysis of LoRA transferability interesting, I am less convinced by the practical motivation and usefulness of the proposed framework.

Critical strengths

The analysis of LoRA transferability across model families is interesting. In particular, the contrasting results between shared-basis and cross-family weight transfer provide useful empirical insights and motivate exploring behavior-based alternatives.

Critical weaknesses

W1. Unclear practical significance. While the paper discusses scenarios where original training data is unavailable, I remain unconvinced about the broader practical significance of cross-family LoRA transfer. In particular, it is unclear how frequently such scenarios arise and when the proposed framework would be preferable to conventional distillation, given its reliance on a powerful teacher model.

W2. Limited guarantees on skill discovery. The method relies on an initial pool of generic probes, followed by evolution of high-divergence examples. However, it is unclear how the framework can discover skills that are poorly represented or entirely absent in the initial probe distribution. The sensitivity to initial probe selection and the completeness of discovered skills are not sufficiently investigated.

W3. Unclear source of improvements over native SFT. Although the paper includes several control experiments, it remains unclear to what extent the improvements come from divergence-guided skill elicitation rather than the quality of synthetic reasoning trajectories generated by a much stronger teacher. The current analysis does not fully disentangle these factors.

Questions for authors
  1. What are the main practical scenarios where cross-family LoRA transfer is preferable to standard fine-tuning or distillation?

  2. How sensitive is skill discovery to the initial generic probe distribution, particularly for skills not represented in the initial pool?

  3. Can the authors further isolate the contribution of divergence-guided elicitation from the benefits of strong-teacher synthetic data?

2 comments

Thank you for the review and for the questions on practical use and on the limits of skill discovery. The new control against ordinary distillation from the same strong teacher is shared with the other reviewers, so its tables are in the General Response (Tables G1–G3). Below we answer each point.

W1 / Q1. When is cross-family LoRA transfer preferable to conventional fine-tuning or distillation?

The main case we target is base-model turnover. A platform that maintains many adapters, for example one per customer or per product line, has to move all of them when the base model is replaced or retired. In that setting:

  • the training data often cannot be kept, because of customer contracts or retention policies;
  • nobody can write a task description by hand for every adapter.

Migration then has to be automatic and independent of task information, which is what the zero-prior design provides. The source model stays queryable until it is retired. Elicitation is paid once per adapter, and its curriculum is reused for every target. Table 3 already includes the one-time teacher cost: for three targets the total is about 62.7 B200 GPU·h for our pipeline against about 116 for Native training, 53.3 less.

A second case is an adapter whose weights cannot be moved. One example is a model fine-tuned through a hosted service, where the owner can sample from both the fine-tuned model and its base but cannot download weights. Our method needs only sampling access to those two models, with no weights or gradients (§5; the revised Ethics statement states this requirement).

When the labeled data are still available, Native SFT remains an option. In the cases above they are not, so the practical alternative is ordinary distillation from the same strong teacher. We tested that comparison directly (General Response, Table G2):

  • Distillation from the 397B teacher with no adapter (R+B) leaves the 4B student at its untrained Base (76.49 vs. 76.45).
  • The full method is +4.78 [4.06, 5.50] above it at 4B and +4.84 [3.92, 5.76] at 9B.
  • With a smaller 27B teacher, the adapter-free arm is again at Base (−0.39) (Table G3).

The advantage of the method is therefore not the strong teacher. It is what the adapter adds.

Following Reviewer Rt78, all Qwen students are also re-scored at a longer generation budget, because the 2,048-token limit in Table 2 truncated many Base generations. Against the untrained Base, the three-seed students gain +0.93 / +5.79 / +2.92 on the five-benchmark mean at 27B / 9B / 4B (Table G1). The revision reports them in §6 and Table 18.

W2 / Q2. How sensitive is discovery to the initial generic probe pool, especially for skills that are poorly represented or absent in it?

We agree that discovery has no completeness guarantee. We have not run a dedicated sensitivity sweep over the initial pool, so we answer with the evidence the paper already contains and state the limit explicitly.

  • The skill structure does not have to be in the pool; it can be recovered from the contrast. Appendix C reports a zero-prior audit at small scale. It removes every manually specified mathematics prior from the pipeline: the <<>> regex patterns, arithmetic character sets, few-shot exemplars, and domain tag filters. From the base-versus-adapted trajectory contrast alone, the pipeline re-derives the same motif specification. It reaches 27.0% with 289 probes, against 27.5% under the manually specified constraints.
  • Evolution writes questions that the initial pool does not contain (§5.2). Parents are mutated by composition, constraint injection, and format perturbation, and every new question is rescored with the adapter/base contrast. Questions outside the pool's original coverage can therefore enter the curriculum. The static pool saturates at 1,294 usable probes, whereas evolution keeps producing new ones.
  • Poorly represented modalities need explicit coverage pressure (Table 10). Without scarcest-modality refill, the elicited set contains 1 code example and 0 multiple-choice examples, and about 1,100 of its trajectories are math. With refill it contains 355 code and 112 multiple-choice examples. Divergence alone concentrates on the dominant mode, which is why the coverage constraint is part of the method.
  • The operators matter. In the small-scale dose-response sweep (Table 9), student accuracy is 18.5% with no multi-step mutation and 34.5% at a mutation fraction of 0.5. What discovery can reach depends on which operators are available.

Taken together, these results show that discovery can reach structure the pool does not specify. They do not show that it finds every skill. A behavior that the mutation operators cannot approach from a generic pool would be missed. The revised Limitations paragraph (§9) now states this.

"Zero-prior" refers to the five priors of Table 1. The discovery algorithm receives no task identity, original data or seeds, labels, discriminator, or white-box access.

W3 / Q3. Can divergence-guided elicitation be isolated from the benefit of synthetic trajectories from a much stronger teacher?

Yes. The 2×2 in General Response Table G2 separates the two:

  • Adapter-selected questions. With the adapted teacher answering in both arms, discovered questions add +1.94 [1.31, 2.57] at 4B and +3.79 [2.68, 4.90] at 9B (E+A − R+A).
  • Adapted answers. With discovered questions in both arms, adapted answers add +4.21 [3.40, 5.02] at 4B and +4.45 at 9B (E+A − E+B; the 9B value is a difference of arm means).
  • Strong-teacher synthesis alone. It adds nothing measurable: R+B is at Base, and the full pipeline is +4.78 above it at 4B.

Discovery matters much less when the base teacher answers: E+B − R+B is +0.57 [−0.34, 1.48] at 4B and +0.39 [−0.62, 1.40] at 9B. Both intervals include zero, so the discovered questions pay off when they are paired with adapted answers. The 27B-teacher replication in Table G3 shows the same effect of discovery under adapted answers (+2.28 on the 4B student, +4.83 on Llama-3.1-8B).

Reviewer_Rt78OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper asks how to reuse a LoRA adapter across models of different sizes, generations and families without the original training data or task-specific priors. It first measures a dichotomy. When two models share a parameter basis (Qwen3.5-27B to Qwen3.6-27B), copying the adapter with zero edits recovers 92–116% of the native fine-tuning gain, and an explicit Procrustes alignment adds nothing. Across families (Qwen2.5-3B to Llama-3.2-3B), weight-space method tested fails (TRR ≤ 0.08). The authors attribute this to LoRA deltas being meaningful only relative to their base model. They then propose "zero-prior skill elicitation", which uses no task names, seed data, labels, discriminator or white-box access. The teacher generates probes from neutral seeds. Probes where outputs with and without the adapter diverge most are kept and evolved for coverage. The adapter-equipped teacher answers them, and students are fine-tuned on these text trajectories. With a Qwen3.5-397B MoE teacher (and a GLM-5.3-Flash teacher), students include Qwen 27B/9B/4B, Llama-3.1-8B and Granite-3.3-8B. They beat native supervised fine-tuning by 6.0–13.9 points, with large gains on MATH-500, IFBench and AIME, at 2.6–3.0× lower target-side compute. Validation includes comparisons with Cross-LoRA and TiTok, a contamination audit, a generic-SFT (Alpaca) control, a count of probes the base teacher fails, teacher-scale and component ablations, and three-seed replicas of the ablations.

Critical strengths
  1. The paper has well-supported rule for reusing adapters: If two models share a parameter basis, copy the LoRA weights with zero edits; if they belong to different families, transfer behavior instead. Transplant within the same parameter-basis recovers 92–116% of the native-training gain.

  2. The approach is a practical, text-only pipeline that needs no task names, seed data, labels, discriminator or white-box access. One elicitation run improves five students across three model families, and the students beat native fine-tuning on the reported mean at every Qwen size.

Critical weaknesses
  1. The experiments do not separate adapter-skill transfer from ordinary distillation off a much larger teacher. The adapter both selects the probes and answers them. The paper does not train students on the base 397B teacher's answers to its own unselected probes, the only arm with no adapter anywhere. So the adapter's contribution is not properly measured. The "unadapted-teacher control" in Section 7 only counts how often the base teacher fails the probes. Removing divergence selection hurts the 9B student (MATH-500 falls from 68.2 to 39.8) but barely changes the 27B student.

  2. Several base scores look like answer-format failures rather than real ability:

    • Qwen3.5-27B scores 3.03% on four-choice GPQA-Diamond (Table 2), far below 25% chance, yet 34–35% in Table 5.
    • The 4B student scores 20.62% on GSM8K but 43.40% on the harder MATH-500 (Table 2).
    • Llama-3.1-8B scores 12.13% on GSM8K and 9.43% on IFEval (Table 2).
    • Qwen3.6-27B scores 15.2% on MATH-500 (Table 4), which also inflates the transplant TRR.

    The elicited trajectories carry GSM-style #### answer formatting (Sec. 5.3). So part of each gain over base, and part of each TRR, may be format learning.

Questions for authors
  1. Native 27B scores differ between Table 2 and Table 4 (GSM8K 91.36 vs 93.03). Which run is the baseline, how was it tuned, and how many seeds were used?
  2. How are answers extracted? The same base model scores 3.03% on GPQA-D in Table 2 and 34–35% in Table 5. What fraction of base failures in Table 2 are format errors?
  3. Are there results for training on training students on base 397B answers to the unselected and the selected probes?
5 comments

Thank you for the detailed review. You are right on both main points. Under Table 2's setting (thinking enabled, 2,048-token limit) several Base cells are truncations, not measurements of ability. The submitted experiments also did not separate the adapter from ordinary distillation off a large teacher. We have revised the paper accordingly; the General Response lists each change with its location in the revised PDF. In short:

  • §6 and the Table 2 caption now state the generation setting, and the truncation-dominated Base cells are marked.
  • All Qwen students are re-scored with thinking disabled and an 8,192-token limit (Table 18).
  • The main result is now stated against the untrained Base: +0.93 / +5.79 / +2.92 on the five-benchmark mean at 27B / 9B / 4B (General Response, Table G1).
  • "Near-lossless" and TRR = 3.39 are removed, and 92–116% is labeled as the 2,048-token result.
  • A new 2×2 experiment (§7, Table 20) gives the adapter-free control you asked for.

W1 / Q3. Train students on base-397B answers to the unselected and the selected probes.

These are the R+B and E+B arms of the 2×2 in General Response Table G2. We also ran R+A and E+A so that the question policy and the answers can be read separately. The 397B teacher and its source adapter are the original ones. Students are Qwen3.5-4B and 9B, three training seeds per arm, all evaluated at the budget-controlled setting, where truncation no longer dominates any arm's errors (Table 17).

  • The arm with no adapter anywhere is at Base. R+B scores 76.49 ± 0.21 against the untrained 4B Base at 76.45.
  • E+A is above it at both sizes: +4.78 [4.06, 5.50] at 4B and +4.84 [3.92, 5.76] at 9B. E+A is the best arm on all six benchmarks at both sizes.
  • E+B is no better than R+B (+0.57 [−0.34, 1.48] at 4B; +0.39 [−0.62, 1.40] at 9B). Selected probes answered by the base teacher do not reproduce the gain.
  • Both ingredients are needed. The adapted answers add +4.21 [3.40, 5.02] at 4B and +4.45 at 9B (E+A − E+B; the 9B value is a difference of arm means, positive on all six benchmarks). The adapter-selected questions add +1.94 at 4B and +3.79 at 9B (E+A − R+A).
  • Data size. R+B is 2,286 base-teacher answers; E+A is the 2,158 default trajectories (2,041 if truncated targets are dropped).
  • GPQA-D. It gives the largest single-task gain, +13.30, which is (13.30/6)/4.78 = 46.4% of the 4B macro gain. Without GPQA-D the gain is +3.07; weighting tasks by item count gives +3.23 (86.69 → 89.92). GPQA-D's seed SDs are 2.41 to 3.72, as expected for 198 items.

With a smaller 27B teacher and a reconstructed adapter (Table G3), R+B is again at Base (−0.39), and E+A is 4.74 above Base.

What is transferred, given that 4B Native is below Base? Native and the method share the source task but not the training signal:

  • Native fine-tunes the student on the original label set (5,689 examples, mean sequence length 271 tokens).
  • The method fine-tunes it on the adapter-equipped teacher's answers (2,158 trajectories, mean 447 tokens).

The 2×2 measures the adapter's effect inside the method. In E+A − E+B the questions, teacher backbone, data size, and recipe are identical, and the only difference is whether the source adapter is attached when the teacher answers. That difference is +4.21 at 4B and +4.45 at 9B. The base 397B teacher also fails outright on 342 of the 2,071 probes in the w/o-coverage set (16.5%, §7), so the selected probes cover behavior that the adapter adds to the teacher. What reaches the student is therefore the adapter's behavior as expressed by the 397B teacher, not the label set. We have not isolated why direct SFT on the labels hurts the 4B at the longer budget. The shorter targets are one visible difference, but we did not test them as the cause, and the paper does not claim it.

On the 27B student: you noted that removing divergence selection barely changes it in the submitted ablation. We agree that the value of elicitation depends on the student:

  • The submitted ablations are reported per student (Tables 11 and 13). The 9B MATH-500 drop from 68.2 (Table 2) to 39.8 (Table 13) is a 2,048-token result and is not combined with the new numbers.
  • The budget-controlled 2×2 was run on 4B and 9B only. We do not claim a budget-controlled effect at 27B, where the overall gain over Base is under one point (Table G1).

W2 / Q2. How are answers extracted, and what fraction of the low Base scores are format errors?

Extraction uses Math-Verify for GSM8K and MATH-500, option-letter accuracy for ARC-C and GPQA-D, and prompt-level strict accuracy for IFEval and IFBench. On every audited cell, the extractor missed 0 correct answers.

For Qwen3.5-27B Base under the Table 2 setting:

  • all 192 GPQA-D errors are generations truncated at 2,048 tokens;
  • all 278 MATH-500 errors are truncated;
  • 150 of 155 GSM8K errors are truncated.

The low cells are therefore unfinished generations, not extraction failures. The GSM-style #### lines in the elicited trajectories (§5.3) do not explain them either: none of the 192 GPQA-D errors is a complete answer that the scorer failed to read.

Table R1 varies thinking and budget separately (Table 16 in the revision). Entries are accuracy (%) with the fraction of items truncated in brackets. The thinking-on, 2,048-token row reproduces Table 2.

ThinkingBudget27B GSM8K27B MATH-5004B GSM8K4B MATH-500
off2,04886.42 [41.2%]38.20 [63.8%]62.71 [74.6%]21.40 [81.3%]
off8,19297.61 [1.1%]94.60 [2.4%]94.31 [0.9%]91.80 [1.8%]
off16,38497.79 [0.2%]94.72 [0.6%]94.55 [0.1%]92.10 [0.4%]
on2,04888.25 [96.8%]44.40 [100%]20.62 [99.2%]43.40 [97.6%]
on8,19297.20 [2.9%]93.40 [4.1%]93.88 [2.2%]90.60 [3.4%]
on16,38497.44 [1.0%]93.88 [1.7%]94.02 [0.8%]90.94 [1.3%]

Budget is the large factor:

  • From 2,048 to 8,192 tokens with thinking off, the four cells rise by 11.19, 56.40, 31.60, and 70.40.
  • From 8,192 to 16,384, all eight cells rise by at most 0.48.
  • At 8,192 tokens and above, thinking changes accuracy by 0.35 to 1.20 points.

The 4B inversion in Table 2 (GSM8K 20.62 below MATH-500 43.40) sits in the short-budget cell, where 99.2% and 97.6% of items are truncated.

The 2,048-token limit truncates Native and Ours as well, so Table R2 audits all three arms (Table 17). "Truncated / errors" uses the error count as denominator.

SettingArmTaskAcc.ErrorsTruncated / errorsExtraction misses
on, 2,048BaseGPQA-D3.03192/198192/192 (100%)0
NativeGPQA-D48.48102/19841/102 (40.2%)0
OursGPQA-D39.90119/19888/119 (73.9%)0
on, 2,048BaseMATH-50044.40278/500278/278 (100%)0
NativeMATH-50060.80196/500137/196 (69.9%)0
OursMATH-50068.20159/50096/159 (60.4%)0
on, 2,048BaseGSM8K88.25155/1,319150/155 (96.8%)0
NativeGSM8K91.36114/1,31971/114 (62.3%)0
OursGSM8K92.5798/1,31955/98 (56.1%)0
off, 8,192BaseGPQA-D72.2255/1980%0
NativeGPQA-D68.1863/1983.2%0
OursGPQA-D74.7550/1982.0%0
off, 8,192BaseMATH-50094.6027/5003.7%0
NativeMATH-50090.8046/5006.5%0
OursMATH-50094.2029/5006.9%0
off, 8,192BaseGSM8K97.6132/1,3190%0
NativeGSM8K97.9827/1,3190%0
OursGSM8K98.9414/1,3190%0

At 2,048 tokens the truncation share does not favor one arm consistently. It is higher for Ours than for Native on GPQA-D (73.9% vs. 40.2%), but lower on MATH-500 (60.4% vs. 69.9%) and on GSM8K (56.1% vs. 62.3%). The submitted 27B GPQA-D gap between Native and Base (48.48 vs. 3.03) is the gap between a finished run and a run whose errors are all cut off. At 8,192 tokens the order on GPQA-D becomes Ours 74.75, Base 72.22, Native 68.18, so the revision drops the claim that Native "retains an advantage on GPQA-Diamond" as a general statement. Table 2 and Table 5 list 3.03 and 34.3–34.9 for the same Base model because the generation settings differ. The Table 2 caption now states its setting, and the Table 5 caption notes that the two are not comparable.

Llama-3.1-8B is a different case:

  • The evaluated checkpoint is a pretrained base model whose outputs often repeat the question. Its task scores (GSM8K 12.13, IFEval 9.43, five-task mean 24.80) are unchanged at 8,192 tokens, so this is not truncation.
  • An instruct-style prompt raises its six-task macro from 25.63 to 32.82 (+7.19) and its GSM8K to 28.20. Part of the Llama GSM8K gain is therefore prompt adaptation, and §6 now says so.

The same length cap affects the shared-basis transplant: the Qwen3.6-27B MATH-500 base of 15.20 is a short-budget score. Table R3 re-scores the transplant at 8,192 tokens. Brackets are 95% item-level intervals.

PairTaskBaseNativeTransplantTransplant − NativeTRR
3.5-27B → 3.6-27BGSM8K97.0499.1798.26−0.91 [−1.71, −0.11]0.573 [0.38, 1.16]
MATH-50098.0099.4099.80+0.40 [−0.44, 1.24]1.286 [0.62, 4.10]
3.6-27B → 3.6-4BGSM8K95.8297.1096.22−0.88 [−1.64, −0.12]0.312 [0.04, 0.94]
MATH-50093.4095.6095.34−0.26 [−1.16, 0.64]0.882 [0.42, 2.15]

At 8,192 tokens the native gain shrinks to 2.13 points (28 of 1,319 items) and 1.40 points (7 of 500), so the ratios are unstable (MATH-500 interval [0.62, 4.10]). We therefore read the absolute scores:

  • Both pairs stay above Base on the point estimates.
  • Transplant is within 0.91 points of Native.

The revised §4.1 and abstract now state this result next to the 2,048-token 92–116%, label each with its setting, and no longer use "near-lossless". IFEval, ARC-C, and MBPP were not re-run at this budget, and §4.1 says so.

Q1. Why do the Native 27B GSM8K scores differ between Tables 2 and 4, how was the baseline tuned, and how many seeds?

  • The two entries are seeds of one configuration. 91.36 and 93.03 are two training seeds of the same Qwen3.5-27B Native configuration, under the Table 2 setting. A third seed scores 92.12, so the mean is 92.17 ± 0.84. The Table 4 caption now identifies both entries as single seeds and gives this mean.
  • Training. The configuration is trained on the paper's 5,689-example label set. It uses the same LoRA recipe as the distillation arms, with LoRA on the attention and MLP projections. The reported checkpoint is the final one after 3 epochs; no validation or test score enters the selection.

Because the 4B Native falls below Base at the budget-controlled setting, we tuned it on 200 held-out examples drawn by hash from its own label set. Table R4 (Table 19 in the revision) shows the runs; each row is one seed, and best-dev selects on this split only. Row 0 is the paper recipe.

#Learning rateEpochsCheckpointDevGSM8KMATH-500GPQA-DMacroΔ Base
01e−4, cosine3final55.187.0449.6034.3460.37−16.08
15e−51final62.992.8768.2444.0267.66−8.79
25e−52best-dev64.293.2474.6148.1370.09−6.36
31e−41best-dev63.693.6671.3546.2469.24−7.21
42e−41best-dev63.193.0263.8041.5766.62−9.83
51e−43best-dev, dropout 0.158.390.4458.9237.4563.75−12.70

The best run recovers 9.72 points and stays 6.36 below Base; most of the recovery is MATH-500 (+25.01) and GPQA-D (+13.79). For this reason the revision states the 4B effect against Base and against R+B, not against Native.

Seed counts.

  • Each arm in Tables G2 and G3 has three training seeds. For the 27B-teacher 4B E+A arm we report the three-seed mean (81.19) with per-task SDs of 0.23 to 1.00, not a macro SD.
  • Table R4 is one seed per row, and Base evaluations are single deterministic runs.
  • The three-seed replicas in Table 15 cover only the ablation arms listed there.
  • The 4B checkpoint in Table 2 is a separate single run. At the budget-controlled setting it scores 79.54 on the five-benchmark mean (+0.43 over Base). Appendix H reports it separately from the three-seed result and does not combine the two.
Reviewer_xdD3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the challenge of transferring Low-Rank Adaptation (LoRA) capabilities across different language model families without access to original training data or task labels. Because direct weight transfer fails across incompatible model architectures, this work transfers adapter-induced behavior instead by identifying and evolving prompts where the base and LoRA-adapted teacher models diverge most, then distilling the adapted teacher's responses into student models via LoRA fine-tuning.

Critical strengths
  1. Meaningful problem formulation and it is practically relevant
  2. It shows some promising experimental results acriss ultiple student models
Critical weaknesses
  1. Limited methodological novelty and insufficient expeirmental evidence of isolating adapter-specific skill transfer. This method combines several already established methods. In particular titok already utilizes the behavioral differences between a bas emodel and its lora adapted finetuned mdoel to identify informative training contents and mag[ie demonstrates instruction data synthesis w/o seed questions.
  2. Overgeneralized conclusion about weight-space transfer and also insufficient experimental consistency. The paper's central motivation relies on the claim that direct adapter transplantation is approximately lossless within a shared parameter basis, whereas weight-space transfer collapses across model families. But the main cross-family failure analysis is primarily demonstrated using a single Qwen2.5-3B to Llama-3.2-3B transfer setting on GSM8K (Appendix B). This is insufficient to establish a general limitation of weight-space approaches across different architectures, tasks, model capacities, and transfer directions.
Questions for authors
  1. How to demonstrate that the transferred improvements originate specifically from the source LoRA adapter?
  2. What is the independent contribution of each component relative to existing synthetic-data methods?
  3. What role does the LoRA adapter play when alternative answering teachers are used?
2 comments

Thank you for the review. The main addition is a controlled experiment with the original 397B teacher that separates what the adapter contributes through question discovery from what it contributes through its answers. Its tables are shared with the other reviewers and appear in the General Response (Tables G1–G3).

W1 / Q1. How can the improvements be attributed to the source LoRA adapter and not to ordinary strong-teacher distillation?

We trained Qwen3.5-4B and 9B students in a 2×2 design (General Response, Table G2), with three training seeds per arm and a budget-controlled evaluation (thinking off, 8,192 tokens):

  • Questions are either the raw pool the teacher generates without the adapter (R) or the set discovered with it (E).
  • Answers come from either the base teacher (B) or the adapter-equipped teacher (A).

R+B is plain large-teacher distillation with no adapter anywhere.

  • Plain distillation does not help. R+B leaves the 4B student at its untrained Base (76.49 vs. 76.45). The large teacher alone does not produce the gain.
  • The full method does. E+A is +4.78 [4.06, 5.50] above R+B at 4B and +4.84 [3.92, 5.76] at 9B, and it is the best arm on all six benchmarks at both sizes.
  • The gain comes from the adapter, through both channels.
    • Adapted answers on discovered questions add +4.21 at 4B and +4.45 at 9B (E+A − E+B; the 9B value is a difference of arm means).
    • Discovered questions with adapted answers add +1.94 at 4B and +3.79 at 9B (E+A − R+A).
    • Discovered questions answered by the base teacher add nothing measurable (E+B − R+B, 4B: +0.57 [−0.34, 1.48]; 9B: +0.39 [−0.62, 1.40]).
  • GPQA-D supplies (13.30/6)/4.78 = 46.4% of the 4B gain. Without it the gain is +3.07.
  • Replication. With a smaller teacher (Qwen3.5-27B, reconstructed adapter), the pattern repeats on Qwen3.5-4B and on Llama-3.1-8B (Table G3). R+B is again at Base (−0.39) on the 4B student, and E+A is 12.46 above R+B on Llama-3.1-8B.

The revised §7 and Table 20 report this experiment. The revised §6 reports the main comparison against the untrained Base at the same evaluation budget (Table G1).

W1 / Q2. What does the method add beyond TiTok and seed-free synthesis, and what does each component contribute?

We agree that both ingredients have precedents, and we claim neither as new:

  • TiTok already uses the contrast between a base model and its LoRA-adapted version.
  • Magpie already synthesizes instructions without seed questions.

The difference is what the contrast is used for. In TiTok it ranks data that has already been generated. In our method it is the objective of an iterative search over questions.

Table R6. Use of the base/adapter contrast in TiTok and in our method.

TiTokOurs
Source of questionsWritten by the adapted source model from a prompt with five examples of the original training dataProposed by the base teacher from neutral seeds, then evolved; no task name or examples
Contrast signalPer-token log-probability excess of the adapted model over the base modelDifference between the two models' decoded answers to the same question
Role of the contrastFilters a generated pool once, keeping the top-scoring samples and tokensSelects parent questions, which are mutated and rescored in a loop under coverage constraints
Training signalLoss on the selected tokens onlyStandard SFT on full plain-text answers

The first and third rows are what remove the task prior:

  • TiTok's filter needs a pool to rank, and that pool is generated from examples of the task.
  • Our score needs only two decoded answers per question, so it applies to any question. The evolution loop uses it to move from a generic pool toward the questions on which the adapter changes the teacher's behavior.
  • Evolution also goes beyond what a fixed pool contains: the pre-evolution pool saturates at 1,294 usable probes, while evolution keeps producing new ones (§5.2).

Seed-free synthesis without the adapter corresponds to our R questions. Table G2 measures what the contrast-guided search adds on top of it: E+A − R+A = +1.94 at 4B and +3.79 at 9B.

For the individual components, the submitted ablations are measured at 2,048 tokens:

  • Divergence scoring and evolution (ablated jointly). On the 9B student, combined AIME accuracy falls from 20.0% to 8.3% without them (Table 11). On the 27B student the difference is a single problem.
  • Coverage. This step mainly changes the composition of the data, raising code examples from 1 to 355 and multiple-choice examples from 0 to 112 (Table 10).

The budget-controlled evidence for the question policy is the E+A − R+A contrast above. We keep these results separate.

W2. The weight-space analysis does not establish a general limitation across families, tasks, capacities, and transfer directions.

We agree that the controlled cross-family analysis uses one small pair: Qwen2.5-3B → Llama-3.2-3B on GSM8K, with the methods we tested. The revised Limitations paragraph (§9) states this scope. The paper has one more weight-space data point: Cross-LoRA, a data-free projection baseline for heterogeneous LLMs, applied to every student.

  • In Table 2 it falls below Base on GSM8K at every student, including Llama-3.1-8B (3.87 vs. 12.13).
  • In the budget-controlled re-scoring of the 27B student its six-task mean is 61.17 against Base 81.84 (Table 18).

This is a second method and more students, not a proof that every weight-space route fails, and we phrase it that way.

The shared-basis side has been revised as well:

  • "Near-lossless" is removed. The 92–116% recovery is kept, labeled as the 2,048-token result.
  • At 8,192 tokens, Qwen3.5-27B → Qwen3.6-27B scores Base / Native / transplant 97.04 / 99.17 / 98.26 on GSM8K and 98.00 / 99.40 / 99.80 on MATH-500 (§4.1, Appendix I). The transplant is above Base on both and within 0.91 points of Native (GSM8K −0.91 [−1.71, −0.11]; MATH-500 +0.40 [−0.44, 1.24]).
  • At this budget the native gain is only 2.13 and 1.40 points, so the ratios (0.573 and 1.286, the latter with interval [0.62, 4.10]) are unstable. We read the absolute differences instead.
  • A second shared-basis pair, Qwen3.6-27B → Qwen3.6-4B, also stays above Base, with ratios 0.312 and 0.882.

The claim we keep is that shared-basis transplant transfers positively and stays close to native training, not that it is lossless.

Q3. What role does the source adapter play when an alternative teacher supplies the answers?

In the GLM-5.3-Flash rows, GLM-5.3-Flash is an answering teacher only. The probes are the default set selected with the source adapter on Qwen3.5-397B, and GLM-5.3-Flash answers them without that adapter. The revised §6 states this. The adapter's role there is therefore question selection, i.e. adapter-guided discovery with external supervision.

Two caveats:

  • The GLM rows were measured only at the 2,048-token setting of Table 2 (§9).
  • In the budget-controlled 2×2, discovered questions answered without the source adapter (E+B) are not distinguishable from plain distillation. The questions pay off when paired with adapted answers (E+A − R+A = +1.94 at 4B).

We therefore present the GLM rows as evidence that the elicited probes can be answered by a different model. We do not present them as a measure of the adapter's skill transfer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.