acceptodds
Under review as a conference paper at ICLR 2027

Conflict-Aware Model Merging for Data-Free Long-to-Short Reasoning

Abstract

Chain-of-thought (CoT) reasoning improves performance on challenging problems by generating structured reasoning steps, but it often produces unnecessarily long responses to easy ones. While typical long-to-short methods require additional training, calibration data, or input-dependent inference, model merging provides a promising way by combining long-CoT model with short model in parameter space. However, existing merging approaches either apply uniform coefficients across layers or rely on auxiliary data or activation, lacking systematic analysis about conflicts between long-CoT and short models. Importantly, strong CoT update is not necessarily beneficial when it conflicts with corresponding short-model update, motivating an explicit conflict-aware allocation of reasoning strength across layers. In this paper, we identify two patterns that guide long-to-short merging: (1) strength of CoT update, which reflects the potential reasoning benefit, and (2) conflict with short model, which indicates the risk of disrupting short-answer behavior at certain layer. Based on this observation, we propose BCL-Merge, a data-free method that design corresponding signals to select layers and assign proper coefficients into short model under given reasoning strength budget. Under explicit assumptions, we show that the resulting layer selection and strength allocation optimize an entropy-regularized proxy objective that balances reasoning benefit against short-answer disruption. Extensive experiments across reasoning benchmarks show that our method reaches 46.3% accuracy on Qwen2.5-1.5B, outperforming existing merging method by 2.9% while reducing response length by 81.5%. Notably, it also exceeds performance of long-CoT model by 5.3%. Further analysis reveals that our method shortens responses on easy examples and improves accuracy on hard examples, validating its utility.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
model merging, large language models, chain-of-thought reasoning, efficient reasoning, data-free adaptation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_KmvcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies data-free model merging for long-to-short reasoning. The main motivation is to transfer useful reasoning ability from a long-CoT model to a short-response model without requiring additional training data, activation statistics, or an inference-time controller. The proposed method, BCL-Merge, constructs two layer-wise parameter-space scores. The first is a strength score based on the relative magnitude of the long-CoT task vector, intended to measure the amount of transferable reasoning signal. The second is a conflict score based on the fraction of parameter coordinates for which the short and long task vectors have opposite signs, intended to measure the risk of disrupting short-model behavior. The two scores are combined as sl=Rl(1−Cl)s_l = R_l(1-C_l). Layers are ranked using this score, a top-KK subset is selected according to a reasoning budget, and layer-wise merging coefficients are assigned through an entropy-regularized allocation objective. The method is evaluated mainly on mathematical reasoning benchmarks across several model sizes and backbones. The reported results show competitive accuracy-length trade-offs and, in several settings, outperform existing merging baselines while substantially reducing response length relative to the long-CoT model.

Critical strengths
  1. Practical and simple data-free formulation. The paper targets a relevant problem: reducing unnecessary long-CoT reasoning while retaining useful reasoning capability. BCL-Merge does not require additional training data, activation statistics, or an input-dependent inference controller, which makes the method simple and potentially useful in practice.

  2. Competitive empirical accuracy-length trade-off. The reported empirical results are consistently competitive across multiple model sizes and backbones. In particular, the method often improves over existing merging baselines while substantially reducing response length relative to the long-CoT model. This suggests that the proposed merging strategy is empirically useful even though I have concerns about its interpretation.

Critical weaknesses

1. The semantic interpretation of the two proposed scores is not sufficiently validated.

The central conceptual claim of the paper is that the relative magnitude of the long task vector represents transferable "reasoning strength", while sign disagreement between the short and long task vectors represents harmful "conflict". However, the current experiments mainly show that using these two scores leads to a strong final merged model; they do not directly establish that the scores actually measure the quantities claimed by the paper.

For example, if the strength score truly reflects transferable reasoning capability in a given layer, a more direct validation would be to intervene on or merge individual layers and test whether the strength score correlates with the actual improvement in hard-task reasoning accuracy. Similarly, if the conflict score is intended to measure the risk of disrupting short-model behavior, the paper should test whether it predicts the actual behavioral degradation caused by injecting the corresponding long-model layer into the short model.

Without such direct evidence, it is difficult to determine whether the proposed scores reveal a genuine parameter-space mechanism or simply constitute an empirically effective layer-ranking heuristic.

This concern also affects my assessment of the theoretical contribution. The appendix introduces assumptions that connect long-update magnitude to a latent reasoning component, but these assumptions partially encode the very relationship that should be validated. Likewise, once sls_l is defined in advance, adding entropy regularization naturally leads to a top-KK/softmax-style allocation. Therefore, I view the current theory more as an interpretation of the proposed rule than as independent evidence supporting the claimed reasoning mechanism.

2. The current evaluation does not sufficiently establish that the proposed method is better than simpler alternative heuristics, nor does it fully support the paper's broad framing.

Although the paper evaluates multiple model scales and several mathematical reasoning benchmarks, most experiments repeatedly measure mathematical reasoning accuracy and output length. This is reasonable for evaluating final performance, but it is relatively narrow compared with the paper's broader claims about general long-to-short reasoning and parameter-space "patterns".

More importantly, several simple alternatives appear insufficiently explored. Under exactly the same budget and coefficient-allocation setting, I would like to see systematic comparisons against:

  • depth-only layer selection;
  • raw long-update norm ranking;
  • cosine similarity / negative inner product;
  • magnitude-weighted sign disagreement;
  • and other simple ways of combining the strength and conflict signals.

The current ablations show that the strength and conflict components are individually useful, but they do not establish that the specific Rl(1−Cl)R_l(1-C_l) formulation is necessary, nor that the emphasized parameter-space interpretation is unique or essential.

In addition, the visualization suggests that the selected layers are concentrated more heavily in the middle and deeper layers. This makes a simple depth-based baseline particularly important. If a depth-only heuristic can achieve similar results, the contribution of the proposed parameter-space scoring mechanism would be substantially weakened.

Questions for authors

1. Can the authors directly validate the semantic meaning of the proposed strength and conflict scores?

In particular, could you report whether the layer-wise strength score correlates with the actual reasoning gain obtained by intervening on or merging an individual layer, and whether the conflict score correlates with the actual degradation of short-model behavior? Such evidence would materially affect my assessment because it would help distinguish a genuine parameter-space mechanism from an empirically effective heuristic.

2. How does BCL-Merge compare with simpler layer-selection criteria under exactly the same coefficient-allocation and reasoning-budget setting?

I am particularly interested in comparisons with depth-only selection, raw long-update norm, cosine similarity / negative inner product, and magnitude-weighted sign conflict. If the proposed score provides a clear advantage over these alternatives, it would substantially strengthen both the novelty of the method and the credibility of its interpretation.

3. To what extent does the proposed layer ranking generalize beyond mathematical reasoning?

Most of the current evaluation is conducted on math-oriented benchmarks. If the same parameter-space criterion and layer ranking remain effective on qualitatively different reasoning domains, this would make the paper's broader framing of long-to-short reasoning considerably more convincing.

5 comments
AuthorsOpenReview ↗

Response to Reviewer Kmvc (1/2)

Dear Reviewer Kmvc,

Thank you for your effort in reviewing our paper with constructive suggestions! We will answer each question respectively.

W1&Q1: Direct validation of strength and conflict scores.

Thank you for this insightful comment! Proposition 1 theoretically derives the optimal selection and allocation formulation in a general way by constructing a proxy objective, and we totally agree that further empirical validation is necessary to directly validate whether the proposed strength and conflict scores reflect actual model behavior, particularly reasoning benefits on hard tasks and potential degradation on tasks that can be solved with short answers, respectively.

To investigate these relationships, we conduct controlled layer-group intervention experiments following your suggestion. Specifically, we compare Top-K and Bottom-K layer groups according to three criteria: strength score RlR_l, conflict score ClC_l, and their combination Rl(1−Cl)R_l(1-C_l). Only the identities of the selected layers are changed, while other experimental settings remain fixed, allowing us to isolate the effects of different layer-selection signals.

Since the effectiveness can be reflected by different types of questions, we divide the performance evaluation into Easy and Hard subsets, which are defined according to the outcomes of the short outcomes (Tab.6): Short models answers every Easy example correctly and every Hard example incorrectly. This partition allows us to examine whether layer interventions improve reasoning on challenging examples while preserving the original model's ability to handle examples that do not require extensive reasoning. Results are summarized below.

CriterionSelectionEasy Acc.Easy Len.Hard Acc.Hard Len.Acc.
Strength RlR_lTop-KK96.8158571.0288587.8
Bottom-KK96.870054.4230084.1
Conflict ClC_lTop-KK95.7201565.0354883.9
Bottom-KK96.9104068.5259086.8
Rl(1−Cl)R_l(1-C_l)Top-KK97.343274.0252588.8
Bottom-KK96.1130058.2280084.8

We can conclude that:

(1) Strength scores correlate with reasoning improvements. Comparing Top-K and Bottom-K selection under RlR_l (row 1&2), selecting high-strength layers improves Hard accuracy from 54.4% to 71.0%, a substantial gain of 16.6%. Meanwhile, Easy accuracy remains unchanged with longer response. This suggests that layers with higher strength scores contribute more to difficult reasoning tasks, supporting our interpretation of RlR_l as a proxy for potential reasoning benefits.

(2) Conflict scores reflect short-answer disruption. Under the conflict criterion ClC_l, selecting Top-K layers (row 3&4) results in lower Easy accuracy (95.7% vs. 96.9%) and substantially longer Easy responses (2015 vs. 1040) compared with Bottom-K selection. This indicates that interventions on high-conflict layers are more likely to disrupt the model's ability to produce concise answers on tasks that the Short model can already solve. These observations reveal the interpretation of ClC_l as a proxy for potential short-answer disruption.

(3) Combining the two signals provides a better balance. The joint criterion Rl(1−Cl)R_l(1-C_l) explicitly favors layers with high reasoning strength and low conflict (row 5&6). Its Top-K selection achieves the best accuracy, while producing shorter responses, suggesting that the proposed two signals effectively balance reasoning improvements with the preservation of concise responses.

AuthorsOpenReview ↗

Response to Reviewer Kmvc (2/2)

W2&Q2: Comparison with simpler layer-selection criteria.

Thank you for raising this important concern. Demonstrating the individual effectiveness of strength and conflict scores is not sufficient to establish the necessity of our specific formulation Rl(1−Cl)R_l(1-C_l) for layer selection. To address these concerns, we conduct an additional systematic comparison with several alternative layer-selection criteria, including depth-only selection, raw long-update norm ranking, cosine similarity, negative inner product, magnitude-weighted sign disagreement, and alternative combinations of the strength and conflict signals. All methods use exactly the same coefficient-allocation and reasoning-budget setting. Thus, the performance differences can be attributed to the layer-selection mechanism and detailed results are presented in table below.

Selection criterionAvg Acc.Avg Len.
depth-only layer selection(Deepest KK layers)55.981833.3
Raw long-update norm ranking54.972370.0
Cosine similarity55.601968.3
Negative inner product55.552088.3
Magnitude-weighted sign disagreement57.131906.7
Minimum score min⁡(Rl,1−Cl)\min(R_l,1-C_l)56.631852.5
Ours57.801603.3

It can be summarized that:

(1) Depth alone is insufficient to explain the effectiveness of our layer selection. The depth-only baseline exhibits substantially inferior performance compared with our method, which indicates that the effectiveness of layer selection depends on the parameter-space characteristics captured by our proposed scoring mechanism, rather than the positional information of layers alone.

(2) Simpler layer-selection criteria like parameter-space similarity and magnitude-based criteria cannot replace our proposed scoring mechanism. These criteria do not explicitly capture the complementary roles of reasoning strength and short-answer conflict in the same manner as our formulation. Their inferior performance suggests that simply measuring update magnitude, directional similarity, or parameter disagreement is insufficient to identify the most beneficial layers. By contrast, our method provides a more informative criterion to identify layers by jointly considering the reasoning strength and short-answer conflict.

W2&Q3: Generalization beyond mathematical reasoning.

Thanks for your insightful advice! We totally agree that the ability beyond mathematical reasoning to different reasoning domains is crucial for the generalization ability. To this end, we choose HumanEval, MBPP and GLUE to evaluate the ability of code-generating and language-understanding. The performance are shown below.

MethodHumanEval pass@1HumanEval Len.MBPP pass@1MBPP Len.GLUEGLUE Len.
Short model63.4144062.4038080.5936
CoT model78.66560070.20380078.96165
TIES71.34220068.40156079.7978
DARE-TIES70.73205067.40150078.9172
Ours77.44190071.00120081.5257

It can be seen that compared to existing methods, our mtehod obtains substantial improvements (+6.1%, +2.6% and +1.73%, respectively) with shorter responses, and even outperforms CoT model in MBPP and GLUE, which strongly showcase the effectiveness and generalization ability of the proposed method.

We have updated the manuscript in blue based on the valuable advice. Specifically, we add direct validation of strength and conflict scores in Tab.13, comparison with simpler layer-selection criteria in Tab.14, and generalization beyond mathematical reasoning in Tab.11. If you have any further questions, do not hesitate to contact us and we will try our best to solve them. Thank you again for your time and effort!

AuthorsOpenReview ↗

Looking Forward to Your Reply

Dear Reviewer Kmvc,

We truly appreciate your time and effort in reviewing our work. Following your insightful advices, we provide direct validation of strength and conflict scores, comparison with simpler layer-selection criteria, and generalization beyond mathematical reasoning to comprehensively demonsrtate the effectievess of our proposed method, which we believe have adequately adressed your concerns.

Could you please spend several minutes looking through our response and revised manuscript to see if there are any questions left about the work? If you have any remaining problems, respond to us at your convenience and we will try our best to address them.

Best wishes,

Submission 2713 Authors.

Reviewer_K4AhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on long-to-short reasoning and identifies two patterns: the strength of CoT updates and their conflict with the short model. The authors propose BCL-Merge, a data-free method that uses these signals to select layers and assign merging coefficients. Experiments demonstrate the effectiveness of the proposed method.

Critical strengths
  • The paper is easy to follow, with a clear introduction to the problem and motivation for the method.
  • The proposed method is simple and practical, requiring only model weights without additional training.
Critical weaknesses
  1. TIES-Merging already addresses sign conflicts between task vectors. This work mainly adds strength–conflict scoring and budgeted allocation. Table 5 supports using both signals, but does not show why the proposed combination and allocation are better than simpler alternatives.
  2. Table 5 should also report Easy-Acc and Hard-Len for full comparisons.
  3. Comparisons with recent long-to-short reasoning baselines should be added.
  4. Code is not provided during the review process. Given the competitive nature of the venue, reproducibility is one of my evaluation criteria, and I consider providing at least the basic code structure a minimum requirement.
Questions for authors

Please refer to Critical Weaknesses

7 comments
AuthorsOpenReview ↗

Response to Reviewer K4Ah (1/2)

Dear Reviewer K4Ah,

We are truly grateful for your time in reviewing our paper with constructive suggestions! We will answer each question respectively.

Q1: Explanation about the proposed combination and allocation.

Thank you for the constructive comment! We fully agree that Tab.5 (now Tab.6 after revising) supports using both strength and conflict signals is effective, and further validating the specific form of combination and allocation are both important. We will answer them respectively.

Effectiveness of combination. Further empirical validation is necessary to establish the specific formulation Rl(1−Cl)R_l(1-C_l) for layer selection. To address these concerns, we conduct an additional systematic comparison with several alternative criteria, including raw long-update norm ranking, cosine similarity, negative inner product, magnitude-weighted sign disagreement, and alternative combinations of the strength and conflict signals. All methods use exactly the same coefficient-allocation and reasoning-budget setting. Thus, the performance differences can be attributed to the layer-selection mechanism and detailed results are presented in table below.

Selection criterionAvg Acc.Avg Len.
Raw long-update norm ranking54.972370.0
Cosine similarity55.601968.3
Negative inner product55.552088.3
Magnitude-weighted sign disagreement57.131906.7
Minimum score min⁡(Rl,1−Cl)\min(R_l,1-C_l)56.631852.5
Ours57.801603.3

It can be summarized that simpler layer-selection criteria like parameter-space similarity and magnitude-based criteria cannot replace our proposed combination mechanism. These criteria do not explicitly capture the complementary roles of reasoning strength and short-answer conflict in the same manner as our formulation. Their inferior performance suggests that simply measuring update magnitude, directional similarity, or parameter disagreement is insufficient to identify the most beneficial layers. By contrast, our method provides a more informative criterion to identify layers by jointly considering the reasoning strength and short-answer conflict.

Effectiveness of coefficient allocation. Proposition 1 theoretically derives the optimal selection and allocation formulation in a general way by constructing a proxy objective. (1) Intuitively, it focus on crucial layers by allocating larger coefficients; (2) Empiriacally, we conduct an additional ablation study with three configurations to disentangle the contributions. Specifically, we compare random layer selection with uniform allocation, our selected layers with uniform allocation, and our selected layers with the proposed nonuniform coefficient allocation. The results are shown below.

Layer selectionCoefficient allocationAcc.Len.
random KUniform54.752038.3
selected KUniform57.381692.5
selected Knonuniform(Ours)57.801603.3

It is revealed that: (1) Effectiveness of layer selection. Comparing the first and second results, replacing random layer selection with our selected layers, while retaining uniform coefficient allocation, improves accuracy from 54.75 to 57.38 (+2.63) and reduces the average output length by −16.97%. This demonstrates that identifying the appropriate layers is crucial to the overall performance improvement.

(2) Effectiveness of nonuniform coefficient allocation. Comparing the second and third results, both of which use the same selected layers, shows the additional benefit of nonuniform coefficient allocation. Specifically, our allocation strategy further improves accuracy from 57.38 to 57.80 (+0.42), while reducing the average output length from 1692.5 to 1603.3 (−5.27%). This indicates that allocating coefficients nonuniformly across the selected layers enables more effective utilization of the coefficient budget.

Q2: Full comparisons of Table 5.

Following your suggestions, we report full comparisons, including Easy Accuracy, Easy Length, Hard Accuracy and Hard Length of Tab.5 (now Tab.6 after revising), where easy and hard sets are separated by the ability of short model. Results are shown below.

VariantEasy Acc.Easy Len.Hard Acc.Hard Len.
Short model100.001180.00805
Random selection96.50146866.002741
High-conflict95.71201565.003548
w/o conflict score96.79158571.002885
w/o strength score96.89104068.502590
Ours97.3043274.002525

It can be concluded that our method has higher score and shorter responses than other merging variants on both easy and hard sets, validating the effectiveness of the proposed method.

AuthorsOpenReview ↗

Response to Reviewer K4Ah (2/2)

Q3: Comparisons with recent long-to-short reasoning baselines.

To provide a more comprehensive evaluation of our approach, we supplement comparisons against broder recent training-free long-to-short reasoning baselines, including model merging (QA-Merging [2]), early-exit reasoning (DEER [1]), and estimation-and-prompting-based compression (TALE-EP [3]). The average results are presented in table below and detailed results of each dataset are shown in Tab.3 and Tab.12 of the revised manuscript.

MethodAvg Acc.Avg Len.
Short model32.4859.7
CoT model54.53979.3
TIES54.81745.5
DARE-Ties52.41848.7
ACM-Ties55.81646.3
QA-Merging[2]56.01705.0
DEER[1]56.81730.0
TALE-EP[3]55.51810.0
Ours57.81603.3

It can be seen that when comparing with recent long-to-short baselines, our method still achieves the best performance with the shortest response length, which strongly validates the effectiveness of the method. If you think some recent training-free long-to-short baselines are missing, feel free to point it out!

[1] Dynamic Early Exit in Reasoning Models. ICLR 2026.

[2] QA-Merging: Query-Adaptive Reasoning via Layer Selective Model Merging. CIKM 2026.

[3] Token-Budget-Aware LLM Reasoning. ACL 2025.

Q4: Code reproducibility.

The core code of the merging algorithm is presented below. Speicifically, our method takes base model, short model and CoT model as inputs, computes the score for each module, obtains selected layers and coefficients, and finally merges models. The whole process aligns with both methods and theoretical analysis in Sec. 3.2 and 3.3. It involves three models without any additional training. We promise to release the full code upon acceptance.

def merge(base_model, model1, model2, B, rho, T, tau):
    scores = []
    # 1. compute the score
    for x0, x1, x2 in zip(base_model, model1, model2):
        a, b, c = [x.detach().flatten() for x in (x0, x1, x2)]
        d1, d2 = b - a, c - a
        n1, n2 = d1.norm().item(), d2.norm().item()
        R = n2 / (n1 + n2) if n1 + n2 else 0.0
        valid = (d1.abs() > tau) & (d2.abs() > tau)
        opposite = (d1 < 0) != (d2 < 0)
        C = (opposite[valid].mean().item() if valid.any() else 0.0)
        scores.append(R * (1 - C))
    
    scores = torch.tensor(scores)
    
    # 2. obtain selected layers
    K = math.ceil(B * len(model1))
    selected = torch.argsort(scores, descending=True, stable=True)[:K]
    beta = torch.zeros_like(scores)

    # 3. obtain coefficients
    active = selected.clone()
    mass = rho * K
    while active.numel() and mass > 0:
        z = scores[active]
        p = mass * torch.softmax((z - z.max()) / T, dim=0)
        capped = p > 1
        if not capped.any():
            beta[active] = p
            break
        beta[active[capped]] = 1
        mass -= capped.sum().item()
        active = active[~capped]
    
    # 4. get merged models 
    merged = []
    for i, (x1, x2) in enumerate(zip(model1, model2)):
        w = beta[i].item()
        x = (1 - w) * x1 + w * x2
        merged.append(x)
    return merged

We have updated the manuscript in blue based on the valuable advice. Specifically, we add explanation about the proposed combination and allocation in Tab.5 and Tab.14, full comparison of Tab.5 (now Tab.6 after revision), and comparisons with recent long-to-short reasoning baselines in Tab.3 and Tab.12. We hope the additional experiments and explanation can help you better understand our paper. If you have any further questions, feel free to respond to us at your convenience and we will do our best to address them. Thank you again for your time and effort!

AuthorsOpenReview ↗

Looking Forward to Your Reply

Dear Reviewer K4Ah,

We respectfully appreciate your time and effort in reviewing our work. We truly agree with your constructive suggestions, and we provide detailed explanation about the proposed combination and allocation, compare with more recent baselines and conduct additional experiments to validate the effectiveness of the proposed method, which we believe has adequately solved your problems.

Could you please spend several minutes looking through our response and revised manuscript to see if there are any questions left about the work? We are standing by and happy to address any remaining questions you might have.

Best wishes,

Submission 2713 Authors.

Reviewer_FVwQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents BCL-Merge, a data-free model merging method that combines the reasoning capabilities of a long-CoT model with the concise response behavior of a short model. It uses layer-wise update strength and sign-conflict scores to select layers and allocate merging coefficients under a prescribed budget, achieving favorable accuracy–response length trade-offs across several reasoning benchmarks and model families.

Critical strengths
  1. The method is intuitive and well motivated: jointly considering update strength and conflict provides a principled heuristic for deciding where and how much long-CoT influence to introduce. Its reliance solely on model parameters makes it practical, requiring neither calibration data nor additional training.
  2. The paper is clearly written, with a coherent progression from motivation to algorithm design and experimental evaluation. Experiments across multiple model families, together with component ablations and difficulty-aware analyses, provide encouraging evidence of the method’s effectiveness.
Critical weaknesses

The strength and conflict scores are indirect proxies for reasoning benefit and short-model disruption. While the theoretical analysis motivates these choices, its optimality guarantee applies to the constructed proxy objective, and the connection to actual model behavior would benefit from more direct empirical validation.

The evaluation focuses primarily on mathematical reasoning and relatively small model scales. It therefore remains unclear how well the proposed scoring and allocation strategy generalizes to larger models and tasks such as code generation or broader language understanding.

Questions for authors
  1. Could the authors examine whether layer-wise strength and conflict scores correlate with measured reasoning improvements and short-answer degradation under controlled layer interventions? This would further substantiate the interpretation of the two signals.
  2. How much of the improvement comes from selecting the appropriate layers versus allocating nonuniform coefficients? A comparison with uniform allocation over the same selected layers, while keeping the total coefficient budget fixed, would help separate these contributions.
  3. How should practitioners choose the budget BB and capacity parameter ρ\rho without calibration data? Additional guidance on their relationship to output length and accuracy would strengthen the method’s practical usability.
5 comments
AuthorsOpenReview ↗

Response to Reviewer FVwQ (1/3)

Dear Reviewer FVwQ,

We would like to express our great gratitude for your effort in reviewing our paper with constructive suggestions! We will answer each question respectively.

W1&Q1: Interpretation of two signals.

Thank you for this insightful comment! Proposition 1 theoretically derives the optimal selection criterion by constructing a proxy objective, and further empirical validation is necessary to establish whether the proposed strength and conflict scores reflect actual model behavior, particularly reasoning benefits on hard tasks and potential degradation on tasks that can be solved with short answers.

To investigate these relationships, we conduct controlled layer-group intervention experiments following your suggestion. Specifically, we compare Top-K and Bottom-K layer groups according to three criteria: strength score RlR_l, conflict score ClC_l, and their combination Rl(1−Cl)R_l(1-C_l). Only the identities of the selected layers are changed, while other experimental settings remain fixed, allowing us to isolate the effects of different layer-selection signals.

Since the effectiveness can be reflected by different types of questions, we divide the performance evaluation into Easy and Hard subsets, which are defined according to the outcomes of the short outcomes (Tab.6): Short models answers every Easy example correctly and every Hard example incorrectly. This partition allows us to examine whether layer interventions improve reasoning on challenging examples while preserving the original model's ability to handle examples that do not require extensive reasoning. Results are summarized below.

CriterionSelectionEasy Acc.Easy Len.Hard Acc.Hard Len.Acc.
Strength RlR_lTop-KK96.8158571.0288587.8
Bottom-KK96.870054.4230084.1
Conflict ClC_lTop-KK95.7201565.0354883.9
Bottom-KK96.9104068.5259086.8
Rl(1−Cl)R_l(1-C_l)Top-KK97.343274.0252588.8
Bottom-KK96.1130058.2280084.8

We can conclude that:

(1) Strength scores correlate with reasoning improvements. Comparing Top-K and Bottom-K selection under RlR_l (row 1&2), selecting high-strength layers improves Hard accuracy from 54.4% to 71.0%, a substantial gain of 16.6%. Meanwhile, Easy accuracy remains unchanged with longer response. This suggests that layers with higher strength scores contribute more to difficult reasoning tasks, supporting our interpretation of RlR_l as a proxy for potential reasoning benefits.

(2) Conflict scores reflect short-answer disruption. Under the conflict criterion ClC_l, selecting Top-K layers (row 3&4) results in lower Easy accuracy (95.7% vs. 96.9%) and substantially longer Easy responses (2015 vs. 1040) compared with Bottom-K selection. This indicates that interventions on high-conflict layers are more likely to disrupt the model's ability to produce concise answers on tasks that the Short model can already solve. These observations support the interpretation of ClC_l as a proxy for potential short-answer disruption.

(3) Combining the two signals provides a better balance. The joint criterion Rl(1−Cl)R_l(1-C_l) explicitly favors layers with high reasoning strength and low conflict (row 5&6). Its Top-K selection achieves the best accuracy, while producing shorter responses, suggesting that the proposed two signals effectively balance reasoning improvements with the preservation of concise responses.

W2: Extension to larger model.

Thanks for the insightful advice! We totally agree that extension to larger models is important for the scalability of the proposed method. Follow your advice, we conduct experiment on Qwen2.5-32B. Results are shown below.

MethodGSM8K Acc.GSM8K Len.MATH500 Acc.MATH500 Len.AIME24 Acc.AIME24 Len.
Short model95.726081.085016.71350
CoT model96.1130094.2410063.37600
DARE-TIES95.390091.0285053.35500
Ours96.266093.6215063.34500

It shows that (1) larger model gets higher absolute accuracy; (2) our method obtains the highest accuracy and shortest output on all three benchmarks. Relative to CoT model, our method reduces output lengths by 49.23%, 47.56%, and 40.79% with comparable accuracy. The table illustrates a consistent promotion across different model scales, further showing the effectievess. We provide full results in Tab.10 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer FVwQ (2/3)

W3: Generalizability to broader tasks.

Thanks for your insightful advice! We totally agree that the ability beyond mathematical reasoning to different reasoning domains is crucial for the generalization ability. To this end, we choose HumanEval, MBPP and GLUE to evaluate the ability of code-generating and language-understanding. The performance are shown below.

MethodHumanEval pass@1HumanEval Len.MBPP pass@1MBPP Len.GLUEGLUE Len.
Short model63.4144062.4038080.5936
CoT model78.66560070.20380078.96165
TIES71.34220068.40156079.7978
DARE-TIES70.73205067.40150078.9172
Ours77.44190071.00120081.5257

It can be seen that compared to existing methods, our mtehod obtains substantial improvements (+6.1%, +2.6% and +1.73%, respectively) with shorter responses, and even outperforms CoT model in MBPP and GLUE, which strongly showcase the effectiveness and generalization ability of the proposed method.

Q2: Contribution of each component.

Great appreciate for your valuable suggestion! It is important to separate the effectiveness of layer selection and nonuniform coefficient allocation. To disentangle the contributions, we conduct an additional ablation study with three configurations. Specifically, we compare random layer selection with uniform allocation, our selected layers with uniform allocation, and our selected layers with the proposed nonuniform coefficient allocation. The results are presented below.

Layer selectionCoefficient allocationAcc.Len.
random KUniform54.752038.3
selected KUniform57.381692.5
selected Knonuniform(Ours)57.801603.3

It is revealed that: (1) Effectiveness of layer selection. Comparing the first and second results, replacing random layer selection with our selected layers, while retaining uniform coefficient allocation, improves accuracy from 54.75 to 57.38 (+2.63) and reduces the average output length by −16.97%. This demonstrates that identifying the appropriate layers is crucial to the overall performance improvement.

(2) Effectiveness of nonuniform coefficient allocation. Comparing the second and third results, both of which use the same selected layers, shows the additional benefit of nonuniform coefficient allocation. Specifically, our allocation strategy further improves accuracy from 57.38 to 57.80 (+0.42), while reducing the average output length from 1692.5 to 1603.3 (−5.27%). This indicates that allocating coefficients nonuniformly across the selected layers enables more effective utilization of the coefficient budget.

Overall, these results demonstrate that layer selection provides the primary performance gain, while nonuniform coefficient allocation offers additional improvements in both accuracy and output efficiency. The two components are complementary, and their combination leads to the best overall performance.

AuthorsOpenReview ↗

Response to Reviewer FVwQ (3/3)

Q3: Selection of hyperparameters.

We thank the reviewer for this insightful suggestion. In practice, the selection of the budget parameter B and the capacity parameter ρ\rho have different roles in controlling the method's behavior.

Selection of the budget parameter B. The parameter B primarily controls the accuracy–length trade-off. As illustrated by the curves in Fig.3, varying B allows practitioners to adjust the output length and the corresponding accuracy. Therefore, it provides a direct and interpretable way to balance computational cost and task performance according to the requirements of the target deployment scenario. For latency-sensitive applications, a smaller budget can be used to reduce output length and inference latency, while applications prioritizing accuracy can adopt a larger budget to allow more extensive reasoning.

Selection of the capacity parameter ρ\rho. Unlike B, ρ\rho is primarily a performance-related hyperparameter. To investigate its sensitivity, we evaluate different values on Qwen2.5-7B and Qwen2.5-32B. The results are shown below and we provide it in Tab.8 of the paper.

Modelρ\rhoGSM8K Acc.MATH500 Acc.Avg Len.Avg Acc.
Qwen2.5-7b0.391.883.9121056.9
0.592.585.0142057.8
0.792.685.2168058.0
0.992.785.3195058.1
Qwen2.5-32b0.396.093.02200.082.99
0.596.293.62436.784.38
0.796.194.02883.384.49
0.996.194.03400.084.49

It can be concluded that both models exhibit consistent trends: increasing ρ\rho generally improves accuracy, and the marginal gains become progressively smaller. In particular, accuracy largely stabilizes when ρ\rho increases from 0.5 to 0.9, while the average output length continues to increase. Consequently, we adopt ρ\rho=0.5 as the default setting in our experiments, as it provides a favorable balance between accuracy, output length, and performance stability.

We have updated the manuscript in blue based on the valuable advice. Specifically, we add interpretation of two signals in Tab.13, full results of scaling to Qwen2.5-32B in Tab.10, generalizability to broader tasks in Tab.11, and contribution of each component in Tab.5. Feel free to respond to us if you have any further concerns. Thank you again for your time and effort!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.