acceptodds
Withdrawn from ICLR 2027

Beyond Valid Outputs: Auditing Success Preservation in Test-Time LLM Fusion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Combining specialist language models without further training does not guarantee preservation of their existing successes. We study this question through an empirical comparison of frozen expert groups, pairing native task accuracy with retained, lost, and additional successes. WordGame provides deterministic text-state-machine tasks and a public task-to-expert mapping. On its 768-input development panel, syntax constraints increase parseable outputs from 256 to 754, but final successes only from one to two. A separate, frozen 96-input confirmation with the same specialist bank reproduces a large weighting-policy gap: fixed task-aware weighting solves 93 inputs and dynamic weighting 46, despite all responses being parseable and uncapped. A four-policy contrast recovers 93 successful cases when the main weight is fixed at 0.75; equalizing auxiliary weights while retaining the dynamic main weight recovers 73 cases. These are effects of complete generation policies, including their fallback behavior, rather than universal evidence against dynamic fusion. With common-parent math, coding, and instruction specialists, EMFuse, CoRE, and Mean solve 5–15 code problems outside the saved specialist union yet underperform the coding specialist by 3.75–6.75 percentage points. Together, these results show that admissible outputs and additional coverage can coexist with poor preservation. The study identifies concrete policy-dependent tradeoffs without proposing a new fusion algorithm or imposing a universal method ranking.

Primary area:
reinforcement learning
Keywords:
Model Fusion, Decision Making, Test Time Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_d8EjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper is an empirical study of training-free test-time fusion of frozen specialist models. The main focus is whether successes that each specialist already had are preserved after combination. The authors do not propose new algorithm, and count retained / harmed / rescued cases instead of only reporting accuracy.

Critical strengths
  • Firstly. the motivation is practical. Aggregate accuracy cannot show which successes are kept or lost after combination, and retained / harm / rescue counting is a useful reporting practive for fusion research.

  • Second wordgame seperates evaluation axes cleanly. the deterministic simulator lets them check format, final state, and reported transitions separately. It shows clearly the case of right format wrong contents.

  • Finally, the confirmation panel is frozen separately. Fixing 96 inputs before seeing development results and reproducing the main contrast on them is good practice.

Critical weaknesses
  • The main wordgame results are almost trivial from the setup. Specialists have completely disjoint skills, so any fusion that does not put most weight on the home expert must fail. It is expected that equal-weight fusion and parameter merging are near 0%, and FixFuse equals HOME because it is basically HOME routing. This does not represent realistic case where experts abilities partially overlap.

  • The fixed vs dynamic contrast may be about fallback, not weighting. In D75, only 1 step out of about 98k tokens used normal pooling, and the rest was token-string-prefix fallback or main-EOS bypass. So FixFuse and D75 are basically main-expert-only decoding, which is why they equal HOME. Bit fallback rates for FixFist, DynFuse, and CoRE themselves are not reported. Whithout this I cannot accept the interpretation of main-weight protection vs auxiliary allocation, and the difference may just be when pooling happens.

  • Why fallback happens so often is not explained. It seems to come from interaction of top-10 support and token-string prefix rule, which depends heavily on implementation details. If the key result depends on an implementation artifact, it is hard to draw general conclusions.

  • DynFuse is not very representative. DynFuse is the authors’ ablated version of CoRE without token-level factors, while full CoRE works well (697). So DynFuse failure may be a problem of this specific simplification, not of “dynamic weighting” in general. There is no analysis of what exactly the CoRE token factors prevent.

  • The key contrast is not tested in the external setting. FixFuse and DynFuse are only run on WordGame, and external setting only evaluates EMFuse, CoRE, Mean, TIES, and PackLLM. External results also use matched subsets of 96 (AIME only 30), so most CIs overlap and no clear pattern appears. The only meaningful comparison is basically the 400-problem LCB.

  • Novelty and takeaways are weak. That right format is not correctness, that same accuracy can come from different solved problems, and that fused model can lose to the best expert are already known observations. The negative flip concept comes from prior work (Yan et al., 2021), and authors also say it is not a new metric. Because every claim is heavily hedged, there is almost no practical recommendation left.

Questions for authors

Basically, please refer to the critical weaknesses first.

  • How much do FixFuse outputs match HOME outputs at the token level? Is it basically the same as HOME decoding?

  • Under what exact condition does token-string-prefix fallback trigger? Why does fallback happen on almost every token?

  • The only difference between CoRE and DynFuse is the token-level factors. How exactly do these factors prevent DynFuse’s failure? Did you analyze the token position where the failure first appears?

  • Can you also run FixFuse and DynFuse in the external (Qwen3-4B) setting?

  • Do the results hold on a WordGame variant where specialists’ abilities partially overlap (for example, off-family accuracy is not zero)?

  • If specialists are retrained with multiple seeds and equal training budgets, is the fixed vs dynamic gap reproduced?

Reviewer_mERBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary
  • Empirical study of combining frozen specialists' outputs at inference time.
  • Compares per-question correctness against a reference.
  • New synthetic suite, WordGame (3 tasks, 3 LoRA experts).
    • Fixed weight 0.75 on home expert: 697/768. Dynamic weight: 341/768.
    • Syntax constraint: parseable 256 -> 754, correct 1 -> 2.
  • Math/code/IF experts: fusion solves some new code problems but loses more.
Critical strengths
  • Item-level accounting of retained/lost successes, with a separate confirmation set. A useful practice.
Critical weaknesses
  • Expected results from an artificial setting.
    • WordGame experts are disjoint by design (0% off-family).
    • High weight (0.75) on the home expert preserves successes. Others destroy them.
    • Parseable != correct, equal accuracy != same successes: known cautions, not findings.
    • 0.75 is not justified.
    • No analysis on why fusion fails. Just counts.
  • Weak link to real tasks.
    • Math/code/IF experts barely differ (Math = Parent on GSM8K).
    • FixFuse/DynFuse/HOME not run there.
    • Small panels (30--96 inputs).
Questions for authors
  • How was 0.75 chosen? Does the FixFuse/DynFuse gap hold on math/code tasks?
Reviewer_ZQhMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies whether combining specialist LLMs at test time preserves the problems each specialist can solve individually. It introduces a way to track which solutions are retained, lost, or newly gained after model combination. The authors evaluate their approach on WordGame, a synthetic benchmark with three tasks and a separate LoRA specialist for each task. They also test math, coding, and instruction-following specialists on external benchmarks. Their experiments compare fixed and dynamic weighting methods, examine the relationship between output format and answer correctness, and measure how many solutions fusion retains or adds relative to the individual specialists.

Critical strengths

The paper examines whether problems solved by individual specialist models are still solved after combining the models. WordGame allows the authors to evaluate output format, final-state correctness, and reported steps separately. The authors also repeat their evaluation on a separate set of 96 inputs.

Critical weaknesses
  • The main result in the abstract (93 vs. 46, Lines 17-19 and Table 19) comes from DynFuse, which uses several design choices, including top-10 support, epsilon, beta, a main-weight clamp, and fallback behavior (Appendix C). The authors do not test these choices separately. Section 5.2 also acknowledges that this result does not show that dynamic fusion is not effective. Therefore, it is unclear whether the performance gap comes from dynamic weighting itself or from the specific design of DynFuse.

  • Section 5.2 and the Conclusion claim that the four-policy test separates the effects of main-weight protection and auxiliary allocation. However, Table 20 shows that normal pooling occurs on only 1 of 98,138 tokens for D75 and 51 of 98,208 tokens for DEQ. This means that almost all tokens are generated by the fallback or main-EOS branch. As a result, the comparison does not clearly show how the two weighting choices affect the results. The paper also provides no token-level examples explaining why DynFuse fails, even though the main expert generates almost every token on its own.

  • In WordGame, each specialist scores 0% on tasks outside its assigned family, and a public label identifies the correct expert (Tables 2, 3, and 14). Because of this, HOME routing already achieves 696/768, while FixFuse adds only one input (Table 19). Since the correct expert is already known, it is unclear what benefit fusion provides over simply routing each input to the right expert. This setup also does not reflect realistic specialist models that may have overlapping skills.

  • The 5 to 15 new coding solutions and the net losses reported in Table 6 come from single greedy runs. Without repeated sampling, it is difficult to know whether the new solutions are consistent. The abstract and conclusion highlight the LiveCodeBench result as the external evidence for net loss, while the other external benchmarks (GSM8K, IFEval, AIME) show no clear pattern (Tables 4, 5, and 16).

Questions for authors
  • Given that normal pooling rarely occurs and the main expert generates nearly all tokens through the fallback or main-EOS branch, how can the loss of 47 successes in DynFuse be attributed to dynamic weighting rather than poorly tuned fallback or EOS logic?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.