acceptodds
Under review as a conference paper at ICLR 2027

Models as Evidence: When Collaboration Makes Bias Correctable

Abstract

Models reused to assess a population can inherit selection bias from the sources of their training data. We study how to recover the population rate, the positive-label rate before any source selects labels, when training data and selection logs are unavailable but each model's task and sources are recorded. Whether the population rate can be consistently recovered depends on how the models share sources. One additional model, trained only on existing sources, can make the rate consistently recoverable even when neither it nor the existing collection suffices alone. When several sources may be biased, we give an exact condition on these source records: with labels per source, the minimax mean squared error is when the condition holds and otherwise stays bounded away from zero, even when every model's fitted probability is observed exactly. A collection that fails this condition needs unbiased labels growing linearly with to reach that rate from a fixed batch of models. The condition extends to conditional predictions of models that approximate known source mixtures. When a reference task shares the selection, a criterion checkable before acquisition tells apart two candidates that match in source count, overlaps, and unbiased outputs: only one enables recovery. On human feedback under controlled selection, a candidate with worse predictions yields lower Brier loss once combined with existing models through source records, without calibration labels.

Primary area:
learning theory
Keywords:
Selection Bias, Model Reuse, Model Acquisition, Data Provenance, Minimax Risk

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mdVdOpenReview ↗
Rating
1(−1)
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

This paper studies how to recover the population positive-label rate from multiple models that may suffer from selection bias, under the constraint that training data and selection logs are unavailable but each model's training data sources (source records) are known. The core result is a necessary and sufficient condition: consistent recovery is possible if and only if a separation margin κ_s(W) > 0, in which case the minimax MSE is Θ(m^{-1}); otherwise it is bounded away from zero. The authors further extend the framework to a cross-task setting with shared selection and provide a criterion checkable before model acquisition. Validation is conducted on simulated configurations and on HelpSteer3 human feedback.

I am open to raising my score if the authors can address the concerns above in the rebuttal, particularly by (a) positioning the method against an existing selection-bias correction where such a comparison is possible, and (b) providing analysis or discussion on the robustness of the method when the exponential selection form holds only approximately. The theoretical core of the paper is solid; the main barrier to acceptance is the empirical and practical validation gap.

Critical strengths
  1. Complete and rigorous theoretical framework: From single-task to cross-task, from mean identifiability to full-distribution indistinguishability, from necessity to sufficiency with matching bounds, the paper provides a thorough theoretical analysis. The necessary-and-sufficient nature of the κ_s(W) condition gives a clean, actionable criterion: one can determine whether consistent recovery is possible by inspecting only the source records matrix. When κ_s = 0, the proof that even the full observation distributions are indistinguishable—not just the means—is technically careful and rules out the possibility that more sophisticated estimators could bypass the condition.

  2. Effective translation from theory to practice: Theorem 2 reduces the recovery condition to a concise graph-theoretic property—whether an ambiguous pair exists—and provides a criterion (Equation 9) checkable before model acquisition. This transformation from an algebraic condition into an operational decision rule is a meaningful contribution. The cross-task shared-selection setting is well-motivated by real data curation pipelines. The design of the experiments also follows the theory rather than decorating it: ten of the twelve pre-specified configurations start at κ² = 0 and are pushed above zero by a single acquired model, and the comparison of one estimator under random versus margin-based acquisition (0.89 → 0.64 → 0.20 RMSE) isolates the acquisition rule as the cause of the improvement.

Critical weaknesses
  1. Insufficient empirical validation: The real-data experiments are conducted on a single public benchmark (HelpSteer3), with each source fitted on m = 128 retained records—small relative to the m ≥ 512 regime where the linear label budget is established, so the real-data result tests the pipeline more than the asymptotics. The authors acknowledge the proof-of-concept nature of these experiments, but for a paper that claims to address a practical problem in model reuse, the empirical support is not yet sufficient. Comparisons against standard methods for handling selection bias are absent, and evaluation on further public benchmarks is missing. Additionally, the magnitude of constant factors (κ_s(W)^{-1}) in realistic configurations is not discussed, leaving the practical feasibility unclear.

  2. Robustness is discussed for one assumption but not the others: The treatment of source-record errors is more concrete than a bare existence claim—with at most r models whose records list incorrect sources, the parametric rate survives if separation holds after every deletion of up to 2r model rows—but that is a tolerance threshold rather than a degradation rate, and the exponential selection form has no analogous treatment at all. Since the title concerns when bias is correctable, the regime where the assumption holds approximately is the one a practitioner actually occupies, and no bound is given for how the excess MSE grows under a small deviation.

Questions for authors
  1. When the selection mechanism deviates from the exact exponential form, can the κ_s(W) framework still be used, perhaps through approximation or error bounds? Even without exact recovery, can an error bound be provided?

  2. In your experiments, could you compare against at least one standard method for selection bias? Even if the dataset does not perfectly satisfy those methods' assumptions, such a comparison would help readers understand the relative advantages of your approach.

  3. For the specific configuration in Figure 1, what is the approximate magnitude of the constant factor involving κ_s(W)^{-1} in the final MSE for different per-source sample sizes m? Could you also report configurations with κ² small but positive, so that the empirical RMSE can be plotted against κ_s(W)^{-1}?

6 comments
AuthorsOpenReview ↗

Response to mdVd (1/4): Standard methods and label budgets

Thank you for specifying priorities (a) and (b). We address condition (a) with comparisons against soft BBSE and MLLS with BCTS (Concern 1; Appendix E.6, Table E.9). For condition (b), Concern 2 (Part 2) shows that the exponential form is not itself restrictive for binary label-dependent retention and gives error bounds for the deviations that remain: small bias in additional sources and cross-task drift (Section 2; Appendices G.1 and G.3). The remaining W1 points on constants, training size, and benchmark coverage are addressed with Q3 in Concern 3 (Part 4).

Concern 1: Comparisons with standard methods (condition (a); Q2, W1)

In your experiments, could you compare against at least one standard method for selection bias?

Response: We have expanded the comparison to include two groups of methods:

  • Standard label-shift correction methods (addressing condition (a)). We have added Soft BBSE (Lipton et al., 2018) and MLLS with BCTS (Saerens et al., 2002; Alexandari et al., 2020). These methods estimate the target label proportion using labeled source data and predictions on unlabeled target inputs.
  • Additional comparisons using unbiased labels. We also evaluate PPI++ (Angelopoulos et al., 2026), Platt scaling (Platt, 2000), and the label mean to assess whether source correction provides additional value when unbiased calibration labels are available.

Comparison and results. The nine-source HelpSteer3 comparison uses m=128m=128 and 128 new training replicates. Each replicate randomly resplits evaluation contexts mostly reused from earlier analyses. The four biased conditions receive equal weight and both primary comparisons were fixed before execution. The table reports rate RMSE in percentage points (pp); the information requirements are listed separately because the methods use different inputs.

MethodAdditional unbiased labelsOther informationRate RMSE (pp)
Source correction0Source records; 128 unlabeled inputs2.20
Source correction plus label mean128Same as above2.03
Label mean128None4.45
PPI++128Predictions on evaluation inputs4.48
Platt scaling128Average prediction4.46
Soft BBSE064 extra retained labels per source18.7
MLLS with BCTS0Same retained labels; training positive proportions28.1

Source correction achieves 2.20 pp rate RMSE without additional unbiased labels, compared with 18.7 pp for Soft BBSE and 28.1 pp for MLLS with BCTS.

At the same 128-label budget, combining source correction with the label mean reduces RMSE from 4.45 pp for labels alone to 2.03 pp. This is also less than half the error of PPI++ (4.48 pp) and Platt scaling (4.46 pp). Adding labels improves source correction from 2.20 to 2.03 pp, showing that source records and unbiased labels can be used together.

Why the label-shift methods have larger errors. The predictions in this experiment carry little label information, making label-shift estimation unstable; the soft confusion matrices are nearly singular. Appendix E.6 provides the diagnostic checks and confidence intervals.

AuthorsOpenReview ↗

Response to mdVd (2/4): Robustness of selection assumptions

Concern 2: Robustness of the selection assumptions and inaccurate source records (condition (b); Q1, W2)

providing analysis or discussion on the robustness of the method when the exponential selection form holds only approximately.

When the selection mechanism deviates from the exact exponential form, can the κs(W)\kappa_s(W) framework still be used, perhaps through approximation or error bounds?

Response: We have given error bounds that depend explicitly on the size of deviations in the selection assumptions and source records. These bounds provide guarantees even when a fixed deviation prevents the error from vanishing with more data. We address the selection assumptions here and source-record errors in Part 3.

1. Robustness of the selection assumptions (condition (b); Q1, W2).

Regarding the case where “the exponential selection form holds only approximately,” we would like to clarify that this representation is exact for binary label-dependent retention with positive retention probabilities. Specifically, if positive and negative labels are retained with probabilities r1,r0>0r_1,r_0>0, then

=\frac{\theta e^b}{1-\theta+\theta e^b}, \qquad b=\log\frac{r_1}{r_0}.$$ Thus, the exponential form does not impose an additional restriction on these retention probabilities, and the single-task estimator does not need to know $r_1,r_0$, or $b$. We have clarified this in Section 2. The robustness concern remains: source bias may not be exactly sparse, and selection strengths may not be identical across tasks. Below, we relax these two assumptions separately and quantify how the resulting deviations affect estimation error. *Small biases beyond the sparse support.* We allow up to $s$ large source effects and an additional bias of at most $\delta$ in every source: $$q=\theta\mathbf{1}+e+u,\qquad \lVert e\rVert_0\le s,\qquad \lVert u\rVert_\infty\le\delta.$$ With known mixture weights $W$, $\kappa_s(W)>0$, and $\mathbb{E}[V]=Wq$, the sparse least-squares estimator satisfies $$\mathbb{E}\bigl[(\widehat{\theta}-\theta)^2\bigr] \le\frac{4}{\kappa_s(W)^2} \left(\sum_{i=1}^{L}\operatorname{Var}(V_i)+L\delta^2\right),$$ where $L$ is the number of observed models. Thus, the bound separates observation variance from a quadratic deviation term; a smaller margin amplifies both. The estimator does not need to know $\delta$. Under the sampling models in Appendix G.1, for a fixed configuration and sufficiently small $\delta$, the minimax risk is $\Theta(m^{-1}+n^{-1}+\delta^2)$. Here $m$ is the per-source training size and $n$ the student-label budget, with $n^{-1}=0$ for exact model observations. The $\delta^2$ term therefore captures an unavoidable error contribution in this setting, not just an artifact of the upper bound. The numerical check in Table E.15 tests this additional source bias. At $m=32{,}768$, the largest RMSE over 64 perturbation directions is below $\delta$ for margins 0.40 and 0.63 at each tested $\delta\in\{0.25,0.5,1\}$ pp. At $\delta=2$ pp, the largest RMSE is 2.3–3.9 pp. Separately, the human-feedback experiment gives RMSE of 2.18 pp under input-and-label-dependent selection and 2.22 pp under label-dependent selection. *Selection that is not exactly shared across tasks.* We also allow the same biased source to have different selection strengths across tasks. For the collection $O\cup G$ in Figure 1, with one common positively biased source, suppose $$\lvert b^{(2)}-b^{(1)}\rvert\le\varepsilon.$$ Within the parameter range of Theorem G.1, the minimax risk is $$\mathcal{R}_{O\cup G}^{\mathrm{drift}}(m,n;\varepsilon) =\Theta\!\left(m^{-1}+n^{-1}+\varepsilon^2\right).$$ Thus, small cross-task differences contribute an $\varepsilon^2$ term for this configuration (Appendix G.3). This result does not cover arbitrary cross-task designs. Appendix F.1 separately extends the cross-task characterization to known monotone selection channels under its stated conditions.
AuthorsOpenReview ↗

Response to mdVd (3/4): Inaccurate source records

Concern 2 (continued): Inaccurate source records (W2)

The treatment of source-record errors is more concrete than a bare existence claim […] but that is a tolerance threshold rather than a degradation rate […]

Response: The existing result specifies when recovery survives incorrect source records. The new bound instead quantifies how errors in the resulting mixture weights affect estimation.

Let WW be the recorded weights and W′=W+ΔW'=W+\Delta the true weights, both nonnegative with rows summing to one. For q=θ1+eq=\theta\mathbf{1}+e, ∥e∥0≤s\lVert e\rVert_0\le s, the row-sum condition gives Δ1=0\Delta\mathbf{1}=0. The mismatch in the observation mean is therefore Δe\Delta e. With E[V]=W′q\mathbb{E}[V]=W'q and κs(W)>0\kappa_s(W)>0, the estimator using the recorded WW satisfies

\le\frac{4}{\kappa_s(W)^2} \left(\sum_{i=1}^{L}\operatorname{Var}(V_i)+\lVert\Delta e\rVert_2^2\right).$$ The bound now varies explicitly with the weight error and source bias, rather than only specifying a tolerance condition (Proposition G.3, Appendix G.5). The margin uses the recorded $W$, while the variances use the true $W'$. Only errors in biased-source columns contribute to the mean-mismatch term; other weight errors may still affect the variances. This is an MSE upper bound, not an exact expression for the increase in MSE. All these results retain the source-mixture observation model. For learned outputs, fitting and distillation errors must also be accounted for through the observation-error terms in Section 3.2.
Reviewer_UU1hOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

I have found this paper hard to read, as it jumps into discussions without properly introducing the topics. Even the abstract took me a while to understand, as some terms are referenced but only defined in the middle of the paper.

My understanding of the paper is that it claims most datasets are biased at construction, with different positive/negative ratios, and that models trained on those datasets are inherently biased as well. The paper addresses whether, given multiple models trained on a shared resource, one can reconstruct the true positive rate of the data source, even if the models have used more than one data source.

Critical strengths
  • The problem is interesting and important in a world where models are trained on a large chunk of the internet. Adjusting the positive ratio can make a big difference in real-world applications.
  • I am not an expert, but I could not find any issues with the theory; it seems solid.
  • The practical recommendation of recording each model's tasks and sources alongside the model connects theory to practice, although this might be hard in real life cases.
Critical weaknesses

This paper needs major rewrites to be accepted. The theory sounds interesting, but the presentation is inaccessible to most readers and hard to follow, even for experienced researchers.

Main concerns:

  • Terms are used before they are introduced. Even in the abstract, we are talking about sources (are those labelers? benchmarks?), and it discusses reference tasks, sharing selections, conditions, etc., without introducing the terms, requiring one to go to the main body to parse it.
  • The problem definition has so many layers that are presented together that it requires multiple reads to parse. Retention rule, sources vs models, vs blocks, observation interfaces, record matrix C, etc. All those are defined together in about a page and really understanding those is important to follow the rest of the paper.
  • Definitions are thrown as formulas without intuition, and a lot of the core concepts are in the appendix e.g the observation model) and are mentioned several times in the paper.
  • Experiments are also hard to parse; constants (O₉, G₉, H₉, κ₂) are not explained, and you have to map them back to previous figures to really understand what is going on.

In terms of the paper substance:

  • Missing some experiments with naturally occurring bias, as I understand all the bias in the paper is synthetically generated.
  • There is a gap between the theory (requiring each model's exact fitted positive probability and independent source blocks with equal sample size) and the practice where datasets are correlated, and the size of the samples that others used is hard to control, etc. That does not invalidate the results here but does make it weaker, the same way the negative results in the appendix E do, but are not exposed in the main body. We should not overstate the contribution.
Questions for authors
  • Could we test it on datasets with naturally occurring bias?
  • Table E.9 suggests the correction can hurt when there is no selection. Is there a diagnostic to detect, from observations alone, whether applying the correction is safe?
  • Have you seen examples of open-source datasets that follow this scheme you are proposing? I wonder how hard it would be to use this when dealing with real-world data. You mention this quickly but do not go into further discussion.
  • Would you be willing to rewrite parts of the paper where you introduce new concepts and variables? I believe this paper has potential, but even theoretical papers need a clear narrative to motivate people to read and disseminate knowledge. This also works in your favor: if you have a paper that is hard to read, you will likely get fewer citations and fewer people adopting it.
4 comments
AuthorsOpenReview ↗

Response to UU1h (1/3): Readability and the order of explanation

Thank you for identifying where the presentation needs clearer definitions and a closer connection to practice. We have revised the exposition and added the analyses below.

Concern 1: Readability and the order of explanation (Q4; writing comments)

Would you be willing to rewrite parts of the paper where you introduce new concepts and variables?

Terms are used before they are introduced. […] Definitions are thrown as formulas without intuition […]

The problem definition has so many layers that are presented together that it requires multiple reads to parse.

Experiments are also hard to parse; constants (O9O_9, G9G_9, H9H_9, κ2\kappa_2) are not explained […]

Response: Yes. We have rewritten the abstract and Sections 1–2, added a worked margin calculation, and clarified the experimental notation in Section 5. The revisions address your four presentation concerns below.

1) Terms and definitions on first use (Abstract and Introduction).

The abstract begins with a concrete feedback example, defines the population rate before stating the recovery results, and introduces the cross-task extension afterward. It opens with:

Models trained on filtered data can inherit selection bias: if positive ratings are retained more often than negative ones, the models may overestimate how often responses are helpful.

The introduction first distinguishes scoring individual responses from estimating the proportion that users would find helpful. It explains that source records identify each model's task and data sources, with annotator pools and collection channels as examples. It then develops the three-source example before introducing the target and reference tasks in Figure 1.

2) A clearer order for the setup (Section 2).

Section 2 now follows the data flow in four labeled parts, introducing each concept when it is needed:

  • Sources and selection: the positive- and negative-label retention probabilities, followed by their exact conversion to the log-odds form in Eq. (1).
  • Models and source records: each source’s block of labels, how models pool and reuse these blocks, and the record matrix CC.
  • What the learner observes: the smoothing rule, how complete and student observations are obtained, and how undoing smoothing yields ViV_i.
  • Goal: the target quantity, the estimators under consideration, the infimum and supremum in minimax MSE, and the meaning of consistency.

Student models trained on synthetic labels initially enter through additional observation noise; the discussion of their resource costs is deferred to Section 4.

3) Intuition before formulas, with essential explanations in the main text (Sections 2 and 3.1).

Section 2 now explains the observation equation before introducing its symbols:

Each observation is thus the population rate, plus the average selection effect of the model’s sources, plus noise.

Before Eq. (2), Section 2 defines each weight as a source’s share of the pooled labels and each source effect as ea=q(θ,ba)−θe_a=q(\theta,b_a)-\theta. Section 3.1 returns to the same three-source example and explicitly calculates the margin’s change from zero to 1/31/\sqrt{3}. The main text explains the observation interface, while the appendix supplies its derivation. We have also added an appendix glossary for reference; the definitions needed to follow the argument are introduced in the main text.

4) Experimental notation explained locally (Section 5).

Section 5 defines O9O_9 as the seven existing models in the nine-source experiment, introduces the candidates G9G_9 and H9H_9 where they are used, and explains that the subscript in κ2\kappa_2 corresponds to two biased sources. Table 2 places the candidate margins alongside their individual prediction errors and joint-recovery errors, so readers can interpret the comparison without reconstructing the notation from earlier figures.

AuthorsOpenReview ↗

Response to UU1h (2/3): Natural bias, source records, and practical assumptions

Concern 2: Evaluation under naturally occurring bias (Q1)

Could we test it on datasets with naturally occurring bias?

Response: Evaluating recovery from naturally curated data requires source records and an independently established population rate; a curated dataset alone does not supply that reference. Our experiments therefore apply controlled selection to real HelpSteer3 feedback and evaluate against held-out human labels. Rate RMSE is 2.22 pp for label-dependent selection and 2.18 pp for input-and-label-dependent selection (Appendix E.6). The single-task estimator does not need to know the selection function. We have identified evaluation on naturally curated collections, with an independently established population reference, as future work in Section 7.

Concern 3: Public datasets with source records (Q3)

Have you seen examples of open-source datasets that follow this scheme you are proposing? I wonder how hard it would be to use this when dealing with real-world data.

Response: Open preference datasets already publish source records that can support this approach. Tülu 3 (Lambert et al., 2024) lists the datasets in its preference mixture and gives each example a source field. Skywork-Reward (Liu et al., 2024) subsamples from HelpSteer2, OffsetBias, WildGuard, and the Magpie series; its examples also have a source field, and score-based filtering differs across sources. The method needs, for each model, its sources and their mixture proportions. In the counting model, these proportions are each source's share of the training data, which per-example source fields provide. For learned models, how closely outputs approximate these mixtures must also be checked. We have cited these public preference datasets in the introduction. Section 2 defines the mixture weights as source shares of pooled labels, and Section 7 states what provenance records must contain.

Concern 4: The gap between theory and practice (substantive comments)

There is a gap between the theory […] and the practice where datasets are correlated, and the size of the samples that others used is hard to control, etc.

[…] the negative results in the appendix E […] are not exposed in the main body.

Response: We have clarified how the recovery bounds apply to learned models, correlated sources, and unequal source sizes, and what information the method requires:

  • Exact fitted probabilities: learned outputs need only approximate known source mixtures; fitting and distillation errors enter the recovery bound explicitly (Section 3.2).
  • Independent sources and correlated datasets: independence supplies the count-model variance rates and lower bounds. The unweighted single-task bound uses the actual observation variances, which incorporate source correlation (Appendix G.1).
  • Unequal or uncontrolled sample sizes: general known weights represent unequal source sizes through their sample proportions, which must be available; Appendix F.2 also allows models to pool different numbers of sources.
  • Incomplete provenance (Longpre et al., 2024): a redundancy condition tolerates a few incorrect records, and an explicit bound covers inaccurate weights (Appendices G.4–G.5).

Negative results in the main text. Section 7 makes the controlled, single-dataset scope and the possible cost without selection explicit. Table 2b brings the 128-label comparison into the main text, and Section 5 reports both the gains on ambiguous source pairs and the worse maximum error on the other pairs. Source records can improve recovery without guaranteeing a gain at every finite-sample setting.

AuthorsOpenReview ↗

Response to UU1h (3/3): When to apply correction

Concern 5: When to apply correction (Q2)

Table E.9 suggests the correction can hurt when there is no selection. Is there a diagnostic to detect, from observations alone, whether applying the correction is safe?

Response: We provide a label-free switching rule that bounds the probability of applying correction when no selection bias is present. The rule compares the observations’ deviation from their arithmetic mean with a noise threshold. Let R=∥V−Vˉ1∥2R=\lVert V-\bar V\mathbf1\rVert_2, and let vˉ\bar v be a known uniform upper bound on E∥V−EV∥22\mathbb E\lVert V-\mathbb EV\rVert_2^2. For c≥1c\ge1, define

θ^sw=clipI(Vˉ) if R≤cvˉ, and θ^corr if R>cvˉ\hat{\theta}_{\text{sw}} = \text{clip}_{I}(\bar{V}) \text{ if } R \le c\sqrt{\bar{v}}, \text{ and } \hat{\theta}_{\text{corr}} \text{ if } R \gt c\sqrt{\bar{v}}

where I=[1/4,3/4]I=[1/4,3/4] and θ^corr\widehat\theta_{\mathrm{corr}} is the sparse source-correction estimator. With correct source records and κs(W)>0\kappa_s(W)>0, Proposition G.4 gives

\le \frac{(2c^2+4)\bar v}{\kappa_s(W)^2}, \qquad \Pr_{e=0}\!\left(R>c\sqrt{\bar v}\right)\le c^{-2}.$$ Here $e=0$ denotes no selection bias. With the count-model noise bound, the switch preserves the convergence rate and controls unnecessary activation of correction (Appendix G.6). For learned models, the threshold also needs to account for fitting and distillation errors. The guarantee concerns the convergence rate and activation probability, rather than a uniform finite-sample improvement over averaging. The appendix includes a weighted version and settings for $O_9$ and $O_9\cup H_9$ where observations cannot consistently determine whether the ordinary average is sufficiently accurate. **Observed cost without selection.** In Table E.12 (formerly E.9), Brier loss increases by 0.21 and 0.32 without selection; both intervals include zero. With selection, the five-source configuration improves by 10.43 and 8.99, while the nine-source improvements are 0.27 and 0.23, also with intervals including zero. All values are scaled by $10^3$.

Thank you for the hard work here; the paper is much clearer and easier to follow now with the new additions.

Reviewer_DCJiOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks when the population positive-label rate can be consistently recovered from trained models whose training data are unavailable, when the only information is each model's task and which data sources it used ("source records"). Some sources may apply label-dependent selection. The main result is an exact condition on the source records, a "separation margin" of the source-membership matrix: when it is positive the minimax error is Θ(1/m), and otherwise it stays bounded away from zero. With a reference task that shares the selection, the paper gives a criterion for which additional model enables recovery, checkable before acquisition. It also compares this route with retraining, unbiased labels and distillation, and extends the condition to conditional predictions. Experiments combine simulations with a controlled human-feedback study with injected selection, where a candidate with worse predictions gives better joint recovery without calibration labels.

Critical strengths

An exact, checkable characterization of recoverability. The main theorems give if-and-only-if conditions with matching bounds, and a screening test that uses only source records. This turns the intuition that diverse models help into a precise criterion, and the matched candidates in Figure 1 show it separates cases that counts or overlaps cannot.

A careful account of what retraining, distillation and labels can substitute. The paper shows how retraining, single-label students and unbiased labels differ in what they preserve, and gives a clean statement of what a missing model is worth in labels. This goes beyond a direct application of sparse-recovery results.

Critical weaknesses

The empirical case for practical value is thin and mixed. The only learned-model study uses one dataset, one nine-source configuration and one candidate pair, with selection injected by the authors into the sources that the winning candidate contains, so the "weaker predictor collaborates better" result is partly mechanical. The authors' own tables temper the gain: 128 calibration labels alone nearly match source correction (Brier 236.33 vs 235.44), some differences have intervals containing zero, and in the acquisition study structural selection beats random acquisition in only 30 of 60 settings at the largest m.

The guarantees rest on strong, idealized assumptions. They require exactly known source records and weights, independent source blocks, a known bias family and sparsity bound, and, for the reference-task result, a selection effect shared exactly across tasks. There is no evidence that real selection behaves this way, robustness is shown only for a bounded number of record errors, and the constants depend on 1/κ, so a small positive margin gives a vacuous bound. The paper does not report the margins in its experiments.

Questions for authors

In Table E.5, 128 calibration labels alone give Brier 236.33, close to source correction without labels (235.44). What is the practical advantage of the source-record route over collecting such labels, and what does combining the two achieve, with intervals?

What are the separation margins for the configurations in Table H.1 and Table E.3, and how does the finite-m error behave as the margin approaches zero? In Table E.3, structural selection wins in only 30 of 60 settings at m = 16,384; in which settings does it los

2 comments
AuthorsOpenReview ↗

Response to DCJi (1/2): Practical gains and candidate controls

Thank you for the questions on practical gains, candidate design, and robustness. We address label comparisons and candidate controls separately below; margins, acquisition losses, and assumptions follow in Part 2.

Concern 1: Practical gains over unbiased labels (Q1; W1)

In Table E.5, 128 calibration labels alone give Brier 236.33, close to source correction without labels (235.44). What is the practical advantage of the source-record route over collecting such labels, and what does combining the two achieve, with intervals?

Response: The original Table E.5 did not establish a Brier advantage over 128 unbiased labels, and its adaptive combination did not improve on source correction. We have added a larger comparison focused on population-rate recovery.

Why report rate RMSE (Q1). Rate RMSE directly measures error in the estimated population positive-label rate, in percentage points (pp). Brier loss also reflects errors in individual predictions, so similar Brier scores can accompany different accuracy in recovering the overall rate.

New comparison (Q1). We have added 128 new training replicates with random splits of reused evaluation contexts. Unlike the original fixed calibration pool, calibration and evaluation sets are resplit in each replicate. We have also evaluated an inverse-variance combination of the source-corrected rate and the label mean; the original adaptive method blended ordinary and corrected predictions.

MethodUnbiased labelsRate RMSE (pp)
Source correction02.20
Source correction plus label mean1282.03
Label mean1284.45
PPI++1284.48
Label mean2563.09

Without additional unbiased labels, source correction achieves 2.20 pp, compared with 3.09 pp for 256 labels alone. At the same 128-label budget, the combination reduces RMSE from 4.45 pp for labels alone to 2.03 pp, also less than half the 4.48 pp of PPI++. Adding labels to source correction gives a smaller improvement, from 2.20 to 2.03 pp.

Both prespecified MSE comparisons, against the label mean and PPI++, have Bonferroni-adjusted 97.5% confidence intervals excluding zero. The smaller gain over source correction alone also has a positive descriptive 95% interval (Appendix E.6, Table E.9).

The practical benefit shown here is accurate population-rate recovery with fewer additional labels, with a further, smaller gain from combining the two.

Intervals containing zero (W1). In Table E.12 (formerly E.9), the nine-source Brier improvements with selection (0.27 and 0.23) and the increases without selection (0.21 and 0.32) have intervals including zero; we report them unchanged, and Section 7 now states the possible cost without selection.

Concern 2: Does bias placement explain the candidate advantage? (W1)

[…] with selection injected by the authors into the sources that the winning candidate contains, so the "weaker predictor collaborates better" result is partly mechanical.

Response: We have added two controls, fixed by rules before training. K0={3,7,9}K_0=\{3,7,9\} is the only candidate containing both affected sources, 7 and 9, that leaves zero margin. K1={1,3,6}K_1=\{1,3,6\} contains neither and is selected by the margin rule for this layout.

Added candidateIncludes affected sourcesMargin κ2\kappa_2Joint rate RMSE (pp)
G9G_9Both0.6322.52
K0K_0Both09.71
K1K_1Neither0.7072.02
H9H_9Source 9 only09.71

K0K_0 contains both affected sources yet fails; K1K_1 contains neither yet succeeds. These controls show that containing the affected sources is neither necessary nor sufficient for a recovery gain. The prespecified Brier differences, scaled by 10310^3, are 8.75 for K0K_0 minus G9G_9, with 98.75% interval [6.27,11.27][6.27,11.27], and 8.89 for H9H_9 minus K1K_1, with interval [6.69,11.07][6.69,11.07] (Appendix E.5).

All 36 source pairs (W1). We have also evaluated all 36 placements of the biased-source pair. The gain from G9G_9 appears where the recovery condition separates the candidates: on the six pairs that remain ambiguous after adding H9H_9, worst rate RMSE is 8.18 for H9H_9, 2.61 for G9G_9, and 1.88 for K1K_1. On the other 30 pairs, both collections can recover the rate; median RMSEs are similar (1.63 versus 1.64 pp), but G9G_9 exceeds 3 pp on four pairs, reaching 4.12, while H9H_9 stays at or below 2.39 pp (Table E.8).

The human-feedback evidence remains a controlled study on HelpSteer3; the new comparisons test label budgets, candidate choice, and all biased-source pairs within that setting.

AuthorsOpenReview ↗

Response to DCJi (2/2): Margins, acquisition losses, and robustness

Concern 3: Margins and finite-sample behavior (Q2, first part; W2)

What are the separation margins for the configurations in Table H.1 and Table E.3, and how does the finite-mm error behave as the margin approaches zero?

Response: We have reported the experimental margins and added simulations approaching zero margin.

Margins for Tables H.1 and E.3 (Q2, first part).

Table and configurationMargin κ2\kappa_2
Table H.1: existing models; add H9H_9; add G9G_9; add both0; 0; 0.632; 0.670
Table E.3, ten acquisition layouts: existing models; selected candidates0; 0.48–0.76 (median 0.62)

Error as the margin approaches zero (Q2, first part; W2). At κ2=0\kappa_2=0, the new simulation plateaus near 2.2 pp. A small positive margin delays the decrease rather than preventing it: the onset requires sample size roughly proportional to κ2−2\kappa_2^{-2}, and when κ22m≈330\kappa_2^2m\approx330, margins 0.10, 0.20, and 0.40 give RMSE 0.56, 0.59, and 0.72 pp (Table E.14, Figure E.4). For the nine-source label-budget experiment at m=8,192m=8{,}192, measured m⋅MSEm\cdot\mathrm{MSE} is 0.039, compared with an oracle's 0.0355 and the bound 6.62 (Appendix E.2.1): the bound is conservative, while the observed error is close to the oracle.

Concern 4: Why does acquisition win in only 30 of 60 settings? (Q2, second part; W1)

[…] structural selection wins in only 30 of 60 settings at m=16,384m = 16{,}384; […]

Response: We have summarized the settings where acquisition loses. In all 30 losing settings, the existing models are already accurate despite zero margin (RMSE at most 0.26 pp), and the largest gap is 0.017 pp (0.238 versus 0.255). These losses offset only 0.3% of the aggregate MSE improvement. Fifteen exceed Monte Carlo error, so they are detectable but small in absolute terms. Across all 60 settings, margin-based acquisition reduces the largest error to about one-twelfth of that under random acquisition. This is a post hoc summary of the existing runs.

Largest setting RMSE at m=16,384m=16{,}384 (pp)Existing modelsRandom acquisitionMargin-based acquisition
Across 60 settings4.323.030.26

Table E.3 now reports both win counts and the largest setting error: performance at particular parameters and recovery over the whole bias family are different quantities.

Concern 5: Robustness to departures from the assumptions (W2)

They require exactly known source records and weights, independent source blocks, a known bias family and sparsity bound, and, for the reference-task result, a selection effect shared exactly across tasks.

Response: We have added quantitative robustness bounds and summarized each assumption’s role in Table G.1:

  • Exactly known records and weights: a weight error Δ\Delta adds 4∥Δe∥22/κs(W)24\lVert\Delta e\rVert_2^2/\kappa_s(W)^2 to the risk bound, where ee contains the source biases and the margin uses the recorded weights; a bounded number of incorrect records is tolerated through redundancy (Appendices G.4–G.5).
  • Independent source blocks: independence supplies the count-model variance rates and distributional lower bounds. The single-task upper bound uses the observation variances, so dependence must be reflected in those variances.
  • Known bias family and sparsity bound: the estimator needs the sparsity bound ss, but not the selection function or its strength. Small additional source shifts of size δ\delta add 4Lδ2/κs(W)24L\delta^2/\kappa_s(W)^2 (Appendix G.1).
  • Selection shared exactly across tasks: for O∪GO\cup G in Figure 1, log-odds drift bounded by ε\varepsilon gives risk Θ(m−1+n−1+ε2)\Theta(m^{-1}+n^{-1}+\varepsilon^2) within the stated parameter range (Appendix G.3).
  • Evidence that real selection behaves this way: our human-feedback study tests controlled selection; Section 7 identifies naturally curated collections as future work.

For learned models, Section 3.2 bounds recovery error through fitting and distillation errors relative to known source mixtures.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: Main contribution and revisions

We thank the reviewers for their questions on presentation, robustness, and empirical comparisons. We have revised the manuscript and added analyses addressing these points; the detailed responses appear under the corresponding reviews.

Why this matters for model reuse. The paper asks when a population rate can be recovered from trained models with shared, selection-biased sources. Under the stated observation and selection assumptions, an exact condition on the source records determines whether consistent recovery is possible. This yields three practical implications: record each model's sources and mixture weights; assess a candidate's contribution through its relationship to the existing collection; and consider combining source correction with unbiased labels when available. In our controlled human-feedback study, a less accurate individual predictor enables better joint recovery when added to the existing collection. Source records also complement labels: with 128 unbiased labels, the combination gives 2.03 pp, versus 4.45 with the labels alone.

Main revisions.

  • Presentation: We have rewritten the abstract around a concrete feedback example and defined the population rate before stating the results. The introduction explains source records and develops a three-source example before the cross-task setting. Section 2 follows the data flow, explicitly derives the selection formula, and explains the observations and minimax objective; Section 3.1 calculates the margin in the same example. We have also added an appendix glossary and an assumption table.
  • Robustness: We have added explicit bounds for small deviations beyond sparse source bias, inaccurate mixture weights, and cross-task drift. We have also introduced a residual-based switch that, under the stated count-observation model with a positive separation margin and a known noise bound, preserves the convergence rate and controls the probability of applying correction without selection.
  • Evidence: We have added comparisons with standard methods under a shared evaluation protocol, two candidate controls, an evaluation across all 36 placements of the biased-source pair, and simulations varying the margin and training sample size. The human-feedback comparison uses new training replicates and random splits of reused evaluation contexts (Table 2; Appendix E).

Blue marks new results and key explanatory revisions; the PDF diff shows all wording changes. The original main theorems and their numbering are unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.