acceptodds
Under review as a conference paper at ICLR 2027

From Complementarity to Integration Value: A Supply–Conversion Framework for Multi-Model Reasoning

Abstract

Recent advances in large language models have made multi-model integration an important approach to enhancing reasoning. Additional models can contribute through two broad modes: online integration, exemplified by inference-time collaboration, and offline transfer, exemplified by teacher-guided supervised fine-tuning (SFT) followed by standalone deployment. Existing methods improve partner complementarity or teacher–student compatibility within each mode. Beyond these within-mode advances, there remains limited understanding of how the same candidate models create different integration value across roles and what governs these differences. To this end, we introduce a supply–conversion framework that evaluates shared candidate pools across online and offline modes against their corresponding single-model alternatives. For online collaboration, we quantify complementary coverage and selection loss to derive an exact condition for outperforming the stronger candidate. For offline SFT, we measure realized student improvement and the additional value of combining teachers relative to the better single-teacher alternative. Across diverse reasoning tasks, we find that collaboration gains depend jointly on complementarity and selection quality, teacher value follows a different structure from inference-partner value, and the better single teacher provides a strong reference for uniform teacher mixtures. Building on this analysis, we combine task-local probes with cross-pair judge calibration to select between single-candidate inference and collaboration, achieving lower selection regret than the evaluated score-, size-, and diversity-based baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Model Complementarity, Inference-Time Collaboration, Knowledge Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DN6SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether the same models provide different value as inference-time partners and as training teachers. It introduces a “supply–conversion” framework to separate the useful capabilities that additional models offer from the gains a system actually obtains. For collaboration, where two models generate answers and a judge selects one, the paper derives an exact decomposition: the gain over the stronger model equals complementary coverage minus selection loss. For distillation, it measures student improvements from single teachers and uniform two-teacher mixtures. The experiments cover four task domains, 18 teachers, and three students, with nine shared model pairs compared across roles. The results suggest that disagreement alone does not guarantee useful collaboration, a good partner is not necessarily a good teacher, and uniform teacher mixing under a fixed budget usually falls short of the better single teacher. Finally, the authors combine small task-specific probes with cross-pair judge calibration to choose between single-model inference and collaboration, achieving lower selection regret than the evaluated baselines.

Critical strengths

The paper presents a broad and relatively systematic empirical study with appropriately demanding reference points. The main study covers 18 teachers, three students, and 54 single-teacher training configurations across mathematics, science, coding, and logical reasoning. It also evaluates nine shared candidate pairs in both inference-partner and training-teacher roles, expands the collaboration analysis to 23 retained pair configurations, and includes additional sensitivity studies on supervision construction, training budget, optimization, and data filtering. Importantly, collaboration is compared against the stronger single candidate, while teacher mixtures are compared against the better single-teacher student, rather than only against a weaker target model. This experimental coverage, the controlled cross-role setup, and the reporting of negative results make the empirical evidence more informative than a small collection of selected case studies.

The collaboration analysis is simple but interpretable and diagnostically useful. The paper decomposes the gain of fixed-answer collaboration into complementary coverage and judge selection loss, providing a clear account of why a model pair succeeds or fails. The authors also evaluate cross-pair judge calibration, held-out-pair prediction, and decision performance under different probe budgets. These experiments provide some evidence that the analysis may help decide when collaboration is worthwhile, rather than serving only as a post-hoc description of observed results.

Critical weaknesses

The main findings feel somewhat intuitive, and the scientific novelty is limited. The paper shows that model disagreement does not always mean useful complementarity, that complementary answers help only when the judge can select them correctly, that a good inference partner is not necessarily a good teacher, and that uniformly mixing two teachers may be worse than using the better teacher alone. These results are carefully supported by a substantial amount of experimentation, but most of them are not particularly surprising given that inference-time collaboration and offline distillation use different mechanisms and evaluation baselines. Several of these observations are also closely related to recent work on model ensembles and teacher–student compatibility. Therefore, I am not fully convinced that the paper provides a substantially new scientific insight beyond carefully formalizing and validating reasonable intuitions.

The connection between the online and offline settings feels mostly conceptual, and it leads to limited methodological progress. The supply–conversion framing gives the two settings a shared vocabulary, but the actual analyses remain separate. The online setting has an exact accounting identity for selecting between two fixed answers, while teacher value is mainly measured after SFT and does not have a comparable pre-training characterization. It is therefore unclear what we gain from studying the two roles together rather than analyzing collaboration and teacher selection separately. The paper does introduce an online decision rule using task-specific probes and cross-pair judge calibration, but the final decision mainly comes from estimating the terms in the derived identity and plugging them into the formula. The teacher-selection rule is also simple and does not outperform the parameter-count baseline in the reported results. Overall, despite the large experimental effort, the framework does not yet lead to a new collaboration mechanism, distillation objective, or competitive teacher-selection method.

Questions for authors

What do we actually gain by studying inference-time collaboration and training-time teachers under one framework? The online part mainly analyzes selection between two fixed answers, while the offline part measures student performance after SFT with different teachers. It seems that most of the conclusions could also be reached by studying these two problems separately. Could the authors clarify which key insight or decision capability specifically comes from the cross-role comparison?

What is the main new finding or practical capability of this paper compared with prior work? Previous studies have already shown that model disagreement or diversity does not always lead to ensemble gains, and that stronger models are not always better teachers. Recent work has also proposed more direct methods for teacher selection and multi-teacher routing. Beyond measuring these effects more systematically, what new understanding or actionable capability does this paper provide?

What practical conclusion should we take from the teacher-selection results? The current rule mainly selects the teacher with higher accuracy on a task-specific probe, but its reported regret does not outperform the parameter-count baseline. Do the authors view this as a practically useful teacher-selection method, or mainly as evidence that teacher value is still difficult to predict before training? Please clarify how this result fits into the paper’s overall contribution.

4 comments
AuthorsOpenReview ↗

Response to Reviewer DN6S - Part 1

1. What actionable contribution goes beyond existing intuitions?

Thank you for this comment. Our framework provides three testable operations: predicting collaboration gains under different judges, deciding whether collaboration should be used, and intervening in offline supervision allocation. To this end, we have added some experimental evidence.

Online prediction: judge intervention with fixed candidate answers. For fixed candidate answers, Proposition 1 gives

GJ=(ηJ−ρ)D,ρ=12+Δ2D.G_J=(\eta_J-\rho)D,\qquad \rho=\frac12+\frac{\Delta}{2D}.

We evaluated Qwen3.5-9B and gemma-4-31B-it in addition to the original Qwen3.5-4B judge, keeping the candidate-answer cache fixed.

Table R3-A. Judge interventions with fixed candidate answers.

PairCandidatesThreshold ρQwen9B ηQwen9B gainGemma31B ηGemma31B gain
1Qwen4B + GemmaE4B60.87%75.00%+2.9384.00%+4.79
2Qwen4B + Ministral3B57.21%77.00%+3.9886.00%+5.79
3GemmaE4B + Ministral3B59.02%75.00%+3.2884.00%+5.12
4Qwen9B + Gemma12B69.65%75.00%+1.0883.00%+2.68
5Qwen9B + Ministral14B63.44%74.00%+1.9683.00%+3.64
6Gemma12B + Ministral14B58.64%76.00%+2.8184.00%+4.11
7Gemma31B + Qwen3.8-27B59.05%73.00%+1.4782.00%+2.41
8Gemma31B + OLMo32B80.10%77.00%−0.6284.00%+0.78
9Qwen3.8-27B + OLMo32B82.04%76.00%−1.2483.00%+0.20

The same candidate supply yields different gains under different judges: Qwen9B does not exceed the threshold on pairs 8–9, whereas Gemma31B exceeds it on all nine pairs. This intervention validates the distinction between complementary supply and conversion quality.

Online decision: cross-pair calibration versus direct probing. T1 estimates the judge's conditional selection accuracy from other pairs and predicts the gain for the target pair; judge outcomes from the target pair are excluded from calibration.

wj,d=nj,dD^j,d.w_{j,d}=n_{j,d}\hat{D}_{j,d}.

N=∑j≠kwj,dη^j,dN=\sum_{j\neq k}w_{j,d}\hat{\eta}_{j,d}

Z=∑j≠kwj,dZ=\sum_{j\neq k}w_{j,d}

η^−k,d=N/Z\hat{\eta}_{-k,d}=N/Z

The rule selects Team when G^k,d>0\widehat G_{k,d}>0, and otherwise selects the single candidate with higher probe accuracy.

Table R3-B. Selection regret with 125 probes per domain.

Decision ruleQwen9BGemma31B
Generic-score0.920.85
Parameter count1.241.10
Always Single2.904.00
Always Team1.020.58
Direct-Judge-Probe0.4300.370
Ours: cross-pair calibration0.3580.309

Our regret is lower than Direct-Judge-Probe by 16.7% and 16.5%, respectively. The paired improvements and 95% CIs are +0.072 [0.012, 0.132] pp and +0.061 [0.014, 0.108] pp.

Table R3-C. Probe-budget ablation with Gemma31B as judge.

Probes per domainDirect regretOurs regretImprovement95% CI
101.3201.100+0.220[0.105, 0.335]
250.9400.810+0.130[0.050, 0.210]
500.6500.610+0.040[0.008, 0.072]
1000.4300.392+0.038[0.006, 0.070]
1250.3700.309+0.061[0.014, 0.108]

Our method achieves lower regret at all five budgets, with positive lower confidence bounds. Initial calibration costs 4,000 calls. With b probes per domain, direct probing requires 4b calls per new pair. The approaches break even after eight new pairs at b=125 and 100 new pairs at b=10.

AuthorsOpenReview ↗

Response to Reviewer DN6S - Part 2

2. What is gained by studying online collaboration and offline teaching under one framework?

The shared candidate pool separates available capability supply from the gains realized by different mechanisms. Online integration selects among fixed answers, whereas offline SFT changes student behavior through training. The common event decomposition provides a shared measurement basis:

-q_{10}(1-u_{10})-q_{11}(1-u_{11}).$$ The four terms represent recovery of teacher-only coverage, successes when both models were initially wrong, loss of student-only successes, and loss of shared successes. Fixed-answer selection satisfies $u_{00}=0,u_{11}=1$, whereas SFT is not subject to these constraints. Cross-pair calibration tests online conversion, while quality-guided supervision allocation tests offline conversion. **Offline intervention: quality-guided supervision allocation.** The main setting shares 4,000 prompts and 12,000 presentations; a separate control matches 12M supervised tokens. Quality-Single samples two answers from teacher B, whereas Quality-Mix samples one answer from each teacher, using the same training-side selection rule. **Table R3-D. Supervision-allocation ablation for Qwen4B.** | Training scheme | Pair 1 | Pair 5 | Pair 9 | | ----------------------- | --------: | --------: | --------: | | Teacher A only | 60.80 | 66.00 | 70.50 | | Teacher B only | 62.10 | 70.10 | 72.90 | | Fixed A:B=75:25 | 61.50 | 66.40 | 70.90 | | Fixed A:B=50:50 | 62.70 | 66.70 | 71.60 | | Fixed A:B=25:75 | 62.60 | 68.90 | 72.40 | | Epoch-wise reassignment | 62.90 | 69.50 | 72.60 | | Quality-Single B×2 | 62.90 | 70.80 | 73.10 | | Quality-Mix A+B | **63.30** | **71.90** | **73.60** | For Pair 5, Quality-Mix improves over fixed mixing by 5.20 pp, ordinary Single by 1.80 pp, and quality-controlled Single by 1.10 pp. **Table R3-E. Token-matched control for Pair 5.** | Scheme | Pair 5 | | ----------------------- | --------: | | Fixed 50:50 | 66.80 | | Epoch-wise reassignment | 69.30 | | Quality-Single B×2 | 70.70 | | Quality-Mix A+B | **71.50** | | Mix−Quality Single | **+0.80** | The Quality-Mix gain over Quality-Single is +0.80 pp, with a 95% CI of [0.43, 1.17]; its gain over fixed mixing is +4.70 pp, with a 95% CI of [4.10, 5.30]. Thus, the additional Pair-5 gain is not explained solely by the number of supervised tokens or the two-sample opportunity. **Table R3-F. Supervision allocation for the second student, GemmaE4B.** | Training scheme | Pair 1 | Pair 5 | Pair 9 | | ----------------------- | --------: | --------: | --------: | | Better ordinary Single | 64.60 | 73.50 | 77.10 | | Fixed 50:50 | 63.70 | 70.40 | 75.80 | | Epoch-wise reassignment | 64.10 | 72.70 | 76.70 | | Quality-Single | 64.90 | 73.90 | 77.50 | | Quality-Mix | **65.40** | **74.00** | **77.60** | | Mix−Quality Single | +0.50 | +0.10 | +0.10 | For the second student, quality-guided mixing improves over fixed mixing by 1.70, 3.60, and 1.80 pp. The two +0.10 pp differences over quality-controlled Single are not presented as significant gains.
AuthorsOpenReview ↗

Response to Reviewer DN6S - Part 3

3. What is the practical implication of teacher selection?

The original T2 selects teachers using pre-SFT task accuracy. We further compare Student NLL, CAR, GRACES, and the compatibility-enhanced selection variant to evaluate role-dependent teacher value.

Table R3-G. Teacher-selection regret in restricted candidate pools.

Teacher-selection ruleRegret
Generic-score / Capability profile2.160
Original T2: target-task accuracy1.110
Parameter count0.840
Student NLL0.980
CAR0.630
GRACES0.570
Compatibility-enhanced selection variant0.540
Oracle0.000

The compatibility-enhanced variant reduces regret by 5.3%, 14.3%, 35.7%, and 51.4% relative to GRACES, CAR, parameter count, and original T2, respectively. Its paired improvement over GRACES is +0.030 pp, with a 95% CI of [0.008, 0.052].

Table R3-H. Component ablation of the compatibility-enhanced variant.

VariantRegret
Full method0.540
w/o core compatibility signal0.620
w/o calibration0.680
w/o task-local information0.750

Table R3-I. Leave-One-Pair-Out generalization.

MethodMean regret over nine held-out pairs
Generic-score1.800
Parameter count0.840
CAR0.630
GRACES0.570
Compatibility-enhanced selection variant0.540

For each fold, post-training outcomes from the target pair are excluded from fitting and hyperparameter selection. The nine-fold mean regret is 0.540 pp.

Table R3-J. Complete-pool and teacher-removal robustness.

Selection methodFull poolRemove strongestRemove top threeRandomly remove 40%
Fixed strongest0.000N/AN/AN/A
Generic-score0.0000.1500.4500.550
CAR0.1800.2500.5500.650
GRACES0.1200.2000.4500.580
Compatibility-enhanced selection variant0.0000.1000.3000.380

In the complete pool, the compatibility-enhanced variant matches the strongest fixed teacher at zero regret. When the strongest teacher is unavailable, it outperforms GRACES in all three restricted-pool settings. The corresponding improvements are +0.100 [0.025, 0.175], +0.150 [0.060, 0.240], and +0.200 [0.090, 0.310] pp.

Together, these experiments establish three actionable capabilities of the cross-role framework: predicting collaboration gains under judge changes, selecting teachers under changing candidate availability, and improving student transfer through supervision-allocation interventions.

Dear Reviewer DN6S,

Thank you for your thoughtful feedback. We have provided additional analyses and clarifications addressing your concerns, particularly regarding the cross-role comparison, supervision allocation, and practical implications of teacher selection.

As the discussion period approaches its end, we would appreciate your feedback on whether our responses adequately address your concerns or if any points require further clarification. We would be happy to provide additional explanations.

Kind regards,
Authors

Reviewer_mprWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how multiple LLMs can be integrated to improve reasoning, focusing on two settings: inference-time collaboration and teacher-guided supervised fine-tuning (SFT). The authors propose a supply–conversion framework that distinguishes the complementary capabilities available from different models from how effectively these capabilities translate into performance gains.

Critical strengths
  1. The paper has one identifiable research hypothesis that the value of the same model in the roles of inference collaboration and SFT teacher may differ, and this difference can be understood through supply and conversion. And it carries out 45 groups of SFT and shared-pool comparisons.
Critical weaknesses
  1. The central cross-role comparison covers only nine candidate pairs, using a single model (Qwen4B) as the judge and student. Online collaboration is restricted to selecting between two fixed answers, while offline integration is restricted to uniform 50/50 teacher mixing. These narrow settings limit the generality of the conclusions
  2. The evaluation benchmarks are relatively easy and outdated.
  3. Minor issue: The central collaboration result (Proposition 1) follows directly from elementary joint-correctness accounting. While useful for interpretation, it does not establish a new theoretical understanding of model collaboration.
  4. The proposed supply–conversion framework does not provide a substantive unified characterization of online collaboration and offline distillation.
Questions for authors
  1. What substantive theoretical connection does the supply–conversion framework establish between online collaboration and offline distillation?

  2. Are the negative results of Offline overly dependent on the experimental design? Uniform teacher mixing is inferior to the better single teacher in 8/9 model pairs. But what the paper actually tested was randomly dividing the training prompts, with each teacher responsible for 50% of the data.

3 comments
AuthorsOpenReview ↗

Response to Reviewer mprW - Part 1

1. Are the offline negative results caused by fixed 50/50 allocation?

Thank you for the suggestion. We added ratio ablations, epoch-wise reassignment, quality-guided allocation, and a quality-controlled single-teacher baseline. The main setting shares 4,000 prompts and 12,000 presentations; a separate control matches 12M supervised tokens. Quality-Single samples two answers from teacher B, whereas Quality-Mix samples one answer from each of teachers A and B, followed by the same training-side selection rule.

Table R2-A. Supervision-allocation ablation for Qwen4B.

Training schemePair 1Pair 5Pair 9
Teacher A only60.8066.0070.50
Teacher B only62.1070.1072.90
Fixed A:B=75:2561.5066.4070.90
Fixed A:B=50:5062.7066.7071.60
Fixed A:B=25:7562.6068.9072.40
Epoch-wise reassignment62.9069.5072.60
Quality-Single B×262.9070.8073.10
Quality-Mix A+B63.3071.9073.60

For Pair 5, Quality-Mix improves over fixed mixing by 5.20 pp, ordinary Single by 1.80 pp, and quality-controlled Single by 1.10 pp.

Table R2-B. Control with 12M supervised tokens.

SchemePair 1Pair 5Pair 9
Fixed 50:5062.6066.8071.50
Epoch-wise reassignment62.8069.3072.40
Quality-Single B×262.9070.7073.00
Quality-Mix A+B63.2071.5073.40
Mix−Quality Single+0.30+0.80+0.40

For Pair 5, the gain remains 0.80 pp after token matching, with a 95% CI of [0.43, 1.17]. Thus, the additional gain cannot be explained solely by more supervised tokens or a two-sample opportunity.

Table R2-C. Results for the second student, GemmaE4B.

Training schemePair 1Pair 5Pair 9
Better ordinary Single64.6073.5077.10
Fixed 50:5063.7070.4075.80
Epoch-wise reassignment64.1072.7076.70
Quality-Single64.9073.9077.50
Quality-Mix65.4074.0077.60
Mix−Quality Single+0.50+0.10+0.10

For the second student, quality-guided mixing improves over fixed mixing by 1.70, 3.60, and 1.80 pp. The two +0.10 pp differences over quality-controlled Single are small and are not presented as significant gains. Overall, uniform mixing is not the upper bound of multi-teacher learning: suitable allocation recovers its loss and yields additional gains in settings such as Pair 5.

2. Does the decision benefit extend to harder tasks?

Table R2-D. Selection regret on harder tasks.

Selection ruleMATH-500GPQA DiamondLiveCodeBench
Generic-score1.801.702.30
Parameter count2.101.852.60
Always Single2.801.953.10
Always Team1.902.602.20
Direct-Judge-Probe1.000.941.10
Ours: cross-pair calibration0.7600.8700.880

Our regret is lower by 24.0%, 7.4%, and 20.0%, respectively. Macro-average regret decreases from 1.013 to 0.837 pp, a reduction of 17.4%.

Table R2-E. Transfer accuracy of the Qwen4B checkpoint.

Student configurationMATH-500GPQA DiamondLiveCodeBench
Base student49.6040.9024.80
Ordinary single-teacher SFT55.2045.5031.20
Fixed 50:50 Mix53.4044.4029.40
Quality-Single55.9045.9032.00
Quality-Mix56.4046.0032.40
Mix−Quality Single+0.50+0.10+0.40

These results show that improved supervision allocation recovers the fixed-mixture deficit and retains positive point-estimate gains in harder-task transfer.

AuthorsOpenReview ↗

Response to Reviewer mprW - Part 2

3. What substantively connects the online and offline analyses?

Let qij=P(cS=i,cT=j)q_{ij}=P(c_S=i,c_T=j), and let uiju_{ij} denote the probability that the integrated system is correct within each event. Then

AZ−AS=q01u01+q00u00−q10(1−u10)−q11(1−u11).A_Z-A_S=q_{01}u_{01}+q_{00}u_{00}-q_{10}(1-u_{10})-q_{11}(1-u_{11}).

The four terms represent recovery of teacher-only coverage, successes when both models were initially wrong, loss of student-only successes, and loss of shared successes. Fixed-answer selection satisfies u00=0u_{00}=0 and u11=1u_{11}=1, whereas SFT is not subject to these constraints. Online collaboration is therefore a constrained case of the shared event decomposition, while offline training permits more general behavioral changes.

The event decomposition provides a common measurement basis; judge interventions, supervision-allocation interventions, and held-out decision tests examine how different mechanisms convert candidate supply. Although OLMo has lower standalone accuracy and less teacher-only coverage than Qwen3.8, it achieves a higher SFT recovery rate, showing that supply magnitude and conversion efficiency are distinct.

Dear Reviewer mprW,

Thank you again for your constructive feedback. We have added further supervision-allocation ablations, token-matched controls, and evaluations on harder benchmarks, along with a clarification of the theoretical connection between online collaboration and offline distillation.

As the discussion period approaches its end, we would greatly appreciate your thoughts on whether these additional results and clarifications address your concerns. Please let us know if any points require further discussion.

Kind regards,
Authors

Reviewer_4956OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks how the same pool of candidate LLMs creates different value in two roles: inference partners (answer selection between two fixed responses by a frozen judge) and SFT teachers. It proposes a supply–conversion framing separating available capability from the mechanism that realizes it. For the online side it derives an accounting identity D = 2H + Δ, giving G_col = H − (1−η)D and an exact break-even condition η > ρ = 1/2 + Δ/(2D). For the offline side it reports an 18-teacher × 3-student SFT matrix plus nine pairs measured in both roles. Main findings: disagreement does not identify useful complementarity; teacher rankings are stable but do not track standalone ability (OLMo-32B); uniform two-teacher mixing rarely beats the better single teacher; and a task-local probe plus cross-pair judge calibration lowers selection regret on T1.

Critical strengths
  1. Broad, well-documented experimental coverage. 18 teachers across many families and sizes, 54 SFT configurations, 23 retained pair configurations, four balanced domains, with complete per-cell results in the appendices.

  2. The supply/conversion split gives a simple and actionable decomposition. Separating complementary coverage H from the stronger model's existing advantage Δ makes precise something the diversity literature conflates, and it is empirically sharp: peer pairs reach H ≈ 8.1–8.6 pp while cross-scale pairs reach only 1.3–1.5 pp at higher disagreement (Table 14).

  3. Self-pairs as a resampling reference correctly resist attributing all disagreement to distinct expertise.

Critical weaknesses
  1. The collaboration analysis is bounded by one fixed weak judge, with no ablation. All results use a frozen Qwen3.5-4B judge (Table 5), including cases where a 4B model adjudicates between frontier models. Combining Table 14 with Table 2†: every pair carries 3.7–8.6 pp of recoverable headroom H while at most 1.98 pp is realized — for pair 2, ~77% of available complementarity is lost to selection error, which suggests that the bottleneck is the judge, not the pairings.

  2. Teacher effectiveness is nearly identical across students, leaving the offline half of the framework with little to explain. From Tables 8–10, cross-student teacher-ranking correlation is high. Teacher value is essentially a scalar property of the teacher. Thus, cheap scalar proxies should suffice (e.g. target-task standalone accuracy correlates and parameter count). The claim "standalone strength does not predict teaching ability" holds for generic benchmarks, not for target-task accuracy. Finally, GRACES [1] and CAR [2] are cited as motivation but never compared, leaving the practical teacher-selection question open -- especially as the paper's own T2 rule does not beat parameter count.

  3. No uncertainty is reported anywhere, and most headline effects are not clearly separable from run-to-run noise. With 250 evaluation items per domain, the per-pair collaboration gains and mix-over-best differences in Table 2 are of the same order as the sampling error on those comparisons, and every SFT run uses a single seed with no variance estimate. The regret gaps in Table 3 and Figure 5 are likewise reported without confidence intervals.

Questions for authors
  1. OLMo-32B is 13 pp weaker standalone than Qwen3.8-27B yet a better teacher for all three students in every domain, but this is noted rather than analyzed. Can the signals in GRACES [1] or CAR [2] account for the reversal? If yes, a comparison would strengthen Sec. 6.1 and address T2 losing to parameter count.

  2. Can you rerun the nine pairs with a stronger judge? By your own thresholds η ≈ 0.70 should flip pairs 4–5. If the conclusion does not hold, should T1's advice be "upgrade the judge" rather than "use one candidate"?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 4956 - Part 1

1. Are collaboration gains dependent on judge strength?

Thank you for noting that judge quality may affect online collaboration. For fixed candidate answers, Proposition 1 gives

GJ=(ηJ−ρ)D,ρ=12+Δ2D.G_J=(\eta_J-\rho)D,\qquad \rho=\frac12+\frac{\Delta}{2D}.

The complementary supply D and threshold ρ remain fixed, while changing the judge changes its conditional selection accuracy η. We evaluated Qwen3.5-9B and gemma-4-31B-it in addition to the original frozen Qwen3.5-4B judge, keeping the candidate-answer cache fixed.

Table R1-A. Judge interventions with fixed candidate answers.

PairCandidatesρQwen9B ηQwen9B gainGemma31B ηGemma31B gain
1Qwen4B + GemmaE4B60.87%75.00%+2.9384.00%+4.79
2Qwen4B + Ministral3B57.21%77.00%+3.9886.00%+5.79
3GemmaE4B + Ministral3B59.02%75.00%+3.2884.00%+5.12
4Qwen9B + Gemma12B69.65%75.00%+1.0883.00%+2.68
5Qwen9B + Ministral14B63.44%74.00%+1.9683.00%+3.64
6Gemma12B + Ministral14B58.64%76.00%+2.8184.00%+4.11
7Gemma31B + Qwen3.8-27B59.05%73.00%+1.4782.00%+2.41
8Gemma31B + OLMo32B80.10%77.00%−0.6284.00%+0.78
9Qwen3.8-27B + OLMo32B82.04%76.00%−1.2483.00%+0.20

The same candidate supply can yield different collaboration gains under different judges. Qwen9B does not exceed the break-even threshold on pairs 8–9, whereas Gemma31B exceeds the corresponding threshold on all nine pairs. This is consistent with our framework: complementary supply and conversion quality are distinct factors. The system should decide whether to collaborate based on calibration for the deployed judge rather than always invoking a team.

 

2. Does cross-pair calibration provide practical value over direct probing?

T1 uses task-local probes to estimate candidate accuracy and joint correctness, and estimates conditional judge accuracy from other pairs:

\quad \widehat G_{k,d}=\frac{(2\widehat\eta_{-k,d}-1)\widehat D_{k,d}-|\widehat p_{a,d}-\widehat p_{b,d}|}{2}.$$ The rule selects Team when the predicted gain is positive; otherwise it selects the single candidate with higher probe accuracy. Judge outcomes from the target pair are excluded from calibration. **Table R1-B. Selection regret with 125 probes per domain.** | Decision rule | Qwen9B | Gemma31B | |---|---:|---:| | Generic-score | 0.92 | 0.85 | | Parameter count | 1.24 | 1.10 | | Always Single | 2.90 | 4.00 | | Always Team | 1.02 | 0.58 | | Direct-Judge-Probe | 0.430 | 0.370 | | Ours: cross-pair calibration | **0.358** | **0.309** | Compared with Direct-Judge-Probe, our regret is reduced by 16.7% and 16.5%, respectively. The paired improvements and 95% CIs are +0.072 [0.012, 0.132] pp and +0.061 [0.014, 0.108] pp. **Table R1-C. Probe-budget ablation with Gemma31B as judge.** | Probes per domain | Direct regret | Ours regret | Improvement | 95% CI | |---:|---:|---:|---:|---:| | 10 | 1.320 | 1.100 | +0.220 | [0.105, 0.335] | | 25 | 0.940 | 0.810 | +0.130 | [0.050, 0.210] | | 50 | 0.650 | 0.610 | +0.040 | [0.008, 0.072] | | 100 | 0.430 | 0.392 | +0.038 | [0.006, 0.070] | | 125 | 0.370 | 0.309 | +0.061 | [0.014, 0.108] | Our method achieves lower regret at all five probe budgets, and every improvement interval has a positive lower bound. Initial calibration costs 4,000 calls. With b probes per domain, direct probing requires 4b calls for each new pair. Thus, the two approaches break even after eight new pairs at b=125 and 100 new pairs at b=10.
AuthorsOpenReview ↗

Response to Reviewer 4956 - Part 2

3. Do the teacher-selection comparisons support role-dependent teacher value?

We retain the original T2 definition: select the teacher with higher pre-SFT task accuracy on the target-task probe, without using post-training transfer efficiency τ as a pre-training signal. We further compare this rule with Student NLL, CAR, GRACES, and a compatibility-enhanced selection variant.

Table R1-D. Teacher-selection regret in restricted candidate pools.

Teacher-selection ruleRegret
Generic-score / Capability profile2.160
Original T2: target-task accuracy1.110
Parameter count0.840
Student NLL0.980
CAR0.630
GRACES0.570
Compatibility-enhanced selection variant0.540
Oracle0.000

The enhanced variant reduces regret by 5.3%, 14.3%, 35.7%, and 51.4% relative to GRACES, CAR, parameter count, and original T2, respectively. Its paired improvement over GRACES is +0.030 pp, with a 95% CI of [0.008, 0.052].

Table R1-E. Component ablation of the compatibility-enhanced variant.

VariantRegret
Full method0.540
w/o core compatibility signal0.620
w/o calibration0.680
w/o task-local information0.750

Removing any component increases regret, indicating that task-local information, compatibility signals, and calibration contribute to the enhanced variant.

Definition of the compatibility-enhanced teacher selector. In addition to the original T2 rule, we define a pre-training, student-conditioned teacher ranker. For student SS, candidate teacher tt, and domain dd, it extracts three signals from a fixed task-local probe:

  • Task supply At,dtaskA_{t,d}^{\mathrm{task}}: teacher tt's probe accuracy in domain dd;
  • Student fit NLLS(yt∣x)\mathrm{NLL}_{S}(y_t\mid x): the student's token-level negative log-likelihood on teacher-generated supervision, normalized by the number of valid answer tokens. Lower NLL indicates that the student can more readily fit the teacher's outputs;
  • Role complementarity Ct,S,dC_{t,S,d}: the fraction of disagreement examples on which the teacher is correct and the student is incorrect:
\frac{q_{01}}{q_{01}+q_{10}+\epsilon},$$ where $q_{01}=P(c_S=0,c_t=1)$, $q_{10}=P(c_S=1,c_t=0)$, and $\epsilon$ is a numerical-stability constant. Within each training fold, feature means and standard deviations are computed using historical teacher–student configurations only. We define the three normalized features as follows: $$x_1=z(A_{t,d}^{\mathrm{task}})$$ $$x_2=-z(\mathrm{NLL}_{S}(y_t\mid x))$$ $$x_3=z(C_{t,S,d})$$ The resulting feature vector is $$\mathbf{x}_{t,S,d}=[x_1,x_2,x_3].$$ Observed SFT accuracy gains from historical configurations serve as calibration targets for an L2-regularized linear predictor: $$\widehat g_{t,S,d}=\mathbf w^\top\mathbf x_{t,S,d}+b, \qquad t^*=\arg\max_{t\in\mathcal T} \frac{1}{|\mathcal D|}\sum_{d\in\mathcal D}\widehat g_{t,S,d}.$$ The regularization coefficient is selected within the training fold. In Leave-One-Pair-Out evaluation, the target pair's SFT-gain labels, feature-normalization statistics, and hyperparameters are excluded from fitting. The ranker maps task-local capability supply, student fit to teacher supervision, and student–teacher complementarity to a common estimate of expected transfer gain; the original T2 remains a separate baseline. **Table R1-F. Leave-One-Pair-Out results.** | Method | Mean regret over nine held-out pairs | |---|---:| | Generic-score | 1.800 | | Parameter count | 0.840 | | CAR | 0.630 | | GRACES | 0.570 | | Compatibility-enhanced variant | **0.540** | For each fold, post-training outcomes from the held-out pair are excluded from fitting and hyperparameter selection. The nine-fold mean regret is 0.540 pp, supporting generalization across candidate pairs. **Table R1-G. Complete-pool and teacher-removal robustness.** | Selection method | Full pool | Remove strongest | Remove top three | Randomly remove 40% | |---|---:|---:|---:|---:| | Fixed strongest | 0.000 | N/A | N/A | N/A | | Generic-score | 0.000 | 0.150 | 0.450 | 0.550 | | CAR | 0.180 | 0.250 | 0.550 | 0.650 | | GRACES | 0.120 | 0.200 | 0.450 | 0.580 | | Compatibility-enhanced variant | **0.000** | **0.100** | **0.300** | **0.380** | In the complete pool, the enhanced variant matches the strongest fixed teacher at zero regret. When the strongest teacher is unavailable, it outperforms GRACES in all three restricted-pool settings. The corresponding improvements are +0.100 [0.025, 0.175], +0.150 [0.060, 0.240], and +0.200 [0.090, 0.310] pp.
AuthorsOpenReview ↗

Response to Reviewer 4956 - Part 3

4. Statistical reliability

Online regret uses domain-stratified paired bootstrap. Offline results use paired t intervals over matched seeds, with test-item sampling variability evaluated separately.

Table R1-H. 95% paired intervals for key online improvements.

ComparisonImprovement95% CI
Qwen9B, 125 probes+0.072[0.012, 0.132]
Gemma31B, 10 probes+0.220[0.105, 0.335]
Gemma31B, 25 probes+0.130[0.050, 0.210]
Gemma31B, 50 probes+0.040[0.008, 0.072]
Gemma31B, 100 probes+0.038[0.006, 0.070]
Gemma31B, 125 probes+0.061[0.014, 0.108]
GPQA+0.070[0.012, 0.128]

Table R1-I. Five-seed paired offline results.

Student / PairMix−Quality SingleSeed SD95% seed CI
Qwen4B / P1+0.400.20[0.15, 0.65]
Qwen4B / P5+1.100.30[0.73, 1.47]
Qwen4B / P9+0.500.22[0.23, 0.77]
GemmaE4B / P1+0.500.22[0.23, 0.77]
GemmaE4B / P5+0.100.15[−0.09, 0.29]
GemmaE4B / P9+0.100.15[−0.09, 0.29]

Table R1-J. Pair-5 token-matched control.

ComparisonImprovement95% CI
Quality Mix − Quality Single+0.80[0.43, 1.17]
Quality Mix − Fixed Mix+4.70[4.10, 5.30]

The additional teacher gain for Pair 5 is supported across training seeds and test items.

Dear Reviewer 4956,

Thank you again for your constructive feedback. We have added stronger-judge evaluations, further teacher-selection comparisons with CAR and GRACES, and statistical uncertainty estimates across key results.

As the discussion period approaches its end, we would greatly appreciate your thoughts on whether these additional experiments and clarifications address your concerns or if any points require further discussion.

Kind regards,
Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.