acceptodds
Under review as a conference paper at ICLR 2027

Aligned Tree-State Transfer for One-Message Cross-Silo Adaptation

Abstract

Cross-silo personalization often has to work with one model transfer and few receiver labels. We ask when the internal state a shipped tree ensemble computes for a record adds information beyond the score it already reports. SHIFT (SHipped-model Internal-state Feature Transfer) has the receiver run the immutable teacher on its own records, compute a TreeSHAP vector for each, and train a student on that aligned state with the teacher score and its permitted raw features; nothing is uploaded and the source never participates again. Under a restricted interface, where the student reads a subset of features plus the teacher's declared outputs, aligned state improved AUC over an otherwise identical score-only student by on 18 of 22 tasks after correction, and the gain survived controls that break the pairing between a record and its own state or match that state's width with a random projection. The complete workflow is competitive with 21 published federated and one-shot references, but so is a score-only student built from the same transfer, so that comparison speaks for the one-message transfer rather than the state channel. We also report where the interface stops paying: a label-free leaf reduction of the same state is not separated from it by our tests, extraction costs 20 to 42 times more per record than that reduction, the recalibrated teacher with no student still leads every arm on macro-AUPRC and tuned macro-F1, and adding attribution candidates beyond leaf state leaves a receiver's selected NLL unresolved. The contribution is the interface and its empirical characterization under restricted feature access, not a unique advantage for Shapley attribution.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
one-shot federated learning, source-free adaptation, cross-silo personalization, gradient-boosted decision trees, tabular data, feature attribution, model calibration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_AEbiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates a problem in cross-institutional personalized learning: when the receiver only has access to a model from a single transfer and a limited number of labels, can the internal state of the teacher model provide effective information beyond the predicted score? To address this, the authors propose the SHIFT method: the receiver runs a fixed teacher model locally, extracts a TreeSHAP vector for each sample, and uses this vector, along with the teacher's predicted score and the original features allowed, to train the student model. This process requires no data upload or further involvement from the source side. Under restricted feature access, SHIFT-Q improves mean AUC by 0.0058 over a matched score-only student, with positive gains on 18 of 22 datasets and a Holm-adjusted p-value of 0.033. On seven tasks with semantically defined feature-access restrictions, allowing state-based candidates during validation-based model selection reduces selected test NLL by 0.0128 compared with a scalar-only candidate set that also permits retaining a calibrated teacher.

Critical strengths

1)The paper studies a meaningful and clearly specified question: whether a frozen teacher’s internal state provides predictive value beyond its scores when the receiver’s trainable model has restricted access to raw features. This offers a useful perspective on model reuse under asymmetric feature-access constraints.

2)The experimental design carefully isolates the contribution of aligned state by holding the teacher, student learner, anchor, and blending rule fixed. Alignment-breaking controls, width-matched representations, and comparisons with alternative leaf encodings help distinguish the state contribution from several plausible confounding factors.

3)Extensive experiments demonstrated the effectiveness of the method. On 22 tabular tasks aligned state improves AUC over an otherwise matched score-only student.

Critical weaknesses

1)The method in this paper has limited contribution. There is already a lot of related work on using the leaf nodes or attribution results of the tree model as downstream features, and formula (1) is also a standard conditional entropy relationship. At the same time, the performance improvement of the methods in Tables 2 and 3 is limited. In addition, the average AUC of SHIFT-Q is 0.8443, which does not exceed the 0.8454 of the prior aligned teacher.

2)The reported CPU implementation incurs substantial inference overhead. Table 9 reports end-to-end costs, but SHIFT-Q's batch throughput on Jasmine and MiniBooNE is 1,117 and 312 rows/second, respectively, significantly lower than the score-only baseline of 121,243 and 87,751 rows/second. Combined with limited performance gains, the practicality of this approach requires further explanation.

3)The practical applicability of the assumed access contract remains insufficiently validated. The teacher must retain access to all features at inference, while the student is prohibited from using some raw fields but is permitted to consume derived state that may reveal those fields. Although the paper explicitly acknowledges this distinction, a concrete deployment case demonstrating why such permissions are acceptable and how they are enforced would strengthen the practical motivation.

Questions for authors

See Weaknesses.

2 comments
AuthorsOpenReview ↗

comment 1 out of 2

Thank you for the review. We tested each concern. Per-tree margins, the cheapest state channel, answer the first two together: their batch throughput is 2.6 to 3.1 times lower than score-only, against 108 to 291 times for attribution in the submission's timing run, and they add 6 to 12% to single-record latency (Tables 10 and 24). They are also the best arm of all on two of the three population-shift suites, lowering NLL by .0222 and .0561 relative to the recalibrated teacher on 7/7 tasks and 23/23 forces (Tables 3 and 23).

"the average AUC of SHIFT-Q is 0.8443, which does not exceed the 0.8454 of the prior aligned teacher."

At the fixed blend SHIFT-Raw is .8460, above the teacher's .8454, while SHIFT-Q, the declared primary, is .8443. SHIFT-Raw has the best mean AUC of any arm under the fixed rule and again under a tuned blend with calibration parity, +.0024 AUC over the teacher on 13/22 tasks (raw p .046, Table 21, cited from Section 5.1). At the complete-feature interface it has the highest mean AUC of all 27 arms, above the teacher and all 21 published federated and one-shot systems (.8473 against .8433 for the strongest, Table 2), and it separates from 19 of the 21 after correction.

The larger result is where receiver populations differ. There every state channel has lower mean NLL than the recalibrated teacher on all three suites the revision adds, SHIFT-Q as well: 7/7 tasks, 9 to 10 of 12 countries and 23/23 forces (Tables 3 and 23), with the two silo studies under protocols frozen before their runs.

On Equation 1, we agree that it is a standard identity. Its role in the paper is the condition it sets, which we test in two ways. The access-by-state interaction isolates the effect of the restriction within one protocol (+.0054 AUC at the fixed blend, 19/22, Holm below .001, Table 13), and across 110 units the state gain tracks the direct-access headroom (Spearman .56 and .53 on disjoint seeds, Figure 3 in Appendix M). Leaf and attribution encodings are prior work, and we use them as fixed inputs. What is new is the test that varies only the student's entitlement, and the population-shift results under that interface.

"SHIFT-Q's batch throughput on Jasmine and MiniBooNE is 1,117 and 312 rows/second, respectively, significantly lower than the score-only baseline of 121,243 and 87,751 rows/second."

Section 6 now carries exactly that comparison. The per-tree margin channel, one leaf lookup per tree, removes most of the gap: 40,563 and 29,266 rows per second against 104,696 and 91,735 for score-only, measured in a separate timing run with its own score-only baseline (Table 24). On the 22 tasks it keeps the attribution channels' fixed-blend gain, and it has the lowest held-out probe R² of the state channels. Section 6 states the rule: per-tree margins for batch scoring and for the population-shift gain, SHIFT-Raw where the cost of attribution is acceptable, and width-4 SHIFT-Q or per-tree margins under an exposure cap. Table 24 has the per-tree costs end to end.

AuthorsOpenReview ↗

comment 2 out of 2

"a concrete deployment case demonstrating why such permissions are acceptable and how they are enforced would strengthen the practical motivation."

Section 3.1 now gives both the mechanism and the cases. The asymmetric-access pattern already exists in credit scoring. FICO Score XD uses telecom, utility, pay-TV and public-record information held by external providers and reaches lenders as a score with up to five reason codes. The lender's own underwriting can use that output without directly receiving all the fields behind it. Credit-disclosure rules such as FCRA and Regulation B require specific reasons in relevant circumstances, and cloud systems such as SageMaker Clarify already return per-record attributions. These are different kinds of outputs, but they establish that a score need not be the only information released by a model.

The receiver-executed case is illustrated by hospital early-warning systems. Models including the Epic Deterioration Index and Sepsis Model run inside a health system's production record on identified clinical data and return patient-level risk scores and contributing factors. Their scores have also been used in published clinical research (Singh et al. 2021). The institution's research extracts may have narrower permissions, including restrictions on sensitive records. The model can therefore operate on a broader record than is available to a separate research or prediction workflow. This is an operational analogue, not a claim that the hospitals already use SHIFT.

The question is what the downstream model is allowed to receive. Existing permission to use a score or explanation does not automatically extend to the teacher's SHAP vector or internal state. Appendix B therefore specifies the additional authorization needed for that channel. It describes the two execution topologies, at the sender or inside a cleared receiver-side component, and gives a four-clause deployer's contract with its permissions matrix (Table 5) and audit steps. It also describes an honest-broker arrangement in which a cleared intermediary computes permitted derived outputs from records the downstream researcher cannot access directly.

Where the restriction is based on provenance, because only the shipped component passed the review authorizing its use, a separately approved state output can be permitted. Where the restriction protects the information itself, the state channel must be separately authorized or refused. Enforcement depends on the permissions and audit controls of the actual deployment, not merely on the fact that the output is derived from an approved model.

For a deployer considering a narrower channel, the revision gives the relevant measurements. Width-4 SHIFT-Q keeps 69% of the full-width AUC gain on the 24 population-shifted contexts, at a held-out probe R² of .17 against .05 for the score alone, and still beats the recalibrated teacher on 6/7 tasks (Table 26). Tables 26 and 27 report the utility and measured exposure at different widths. A width cap is not itself a privacy guarantee, but it gives the deployer evidence to assess alongside the access contract.

Reviewer_9bbYOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies local adaptation after a single model transfer: the receiver's frozen teacher can access complete records, while its student can use only a subset of features and the teacher's authorized outputs. Motivated by the possibility that a single prediction score discards information useful for local adaptation, the authors propose providing the student with tree state aligned to each record. They compare raw TreeSHAP, score-residualized TreeSHAP (SHIFT-Q), and a leaf-based representation (Leaf-SVD), and blend the local student's predictions with those of the teacher. Across 22 tabular tasks, SHIFT-Q improves mean AUC from 0.8385 to 0.8443 over a matched student given only the teacher's score in addition to permitted features; when the student can access all features, the gain is only 0.0004. The leaf-based representation achieves similar results, supporting the interpretation that the gains primarily arise from conveying information about restricted features rather than from the explanatory properties unique to SHAP.

Critical strengths
  1. A clear analysis of the problem and mechanism. The paper distinguishes the information provided by teacher scores from that provided by internal state. By varying access permissions within a matched protocol, it tests whether state gains are related to the student's inability to read certain features.
  2. Carefully controlled experiments. Controls include a score-only student, shuffled state, dimension-matched random projections, and direct feature access. Results across multiple random seeds, confidence intervals, and multiple-testing corrections help rule out simple alternative explanations.
Critical weaknesses
  1. The practical value of the research setting is questionable. Both teacher and student run locally at the receiver, which also supplies the complete inputs. Their different feature access therefore relies on an additional permission assumption rather than an inherent constraint of model transfer. If the receiver can freely use these features, it can train the student on them directly; if the restriction is intended to prevent the student from using the associated information, teacher state containing that information may also be disallowed. The paper does not sufficiently explain which real-world needs would justify allowing the teacher to use raw features, prohibiting the student from accessing them directly, yet permitting the student to read rich internal state.
  2. The technical novelty is limited. Using SHAP or leaf encodings as inputs to a downstream model is closely related to existing representation-transfer and stacking approaches. The contribution is primarily an empirical analysis of performance and mechanisms under restricted access, rather than a new representation or learning method.
  3. It remains unclear whether the method works consistently in other settings. In the main experiment, the student cannot access the half of the features ranked most important by the teacher, but can obtain related information through teacher state. This makes the setting particularly favorable for demonstrating the method's benefits. The authors also test other feature restrictions and datasets, but some experiments do not confirm improvements. It is therefore unclear whether the method would consistently help when restricted features are less important or when applied to data from other organizations.
Questions for authors

See "Weakness".

3 comments
AuthorsOpenReview ↗

comment 1 out of 2

Thank you for putting the central question plainly: who would run a model on fields that its own downstream model may not read, and then let that model read the first model's internal state? We now distinguish existing uses of derived model outputs from the additional permission needed to release richer state. Section 3.1 and Appendix B carry the argument, and our second comment reports the new population-shift results.

"The paper does not sufficiently explain which real-world needs would justify allowing the teacher to use raw features, prohibiting the student from accessing them directly, yet permitting the student to read rich internal state."

There are already systems in which a model uses information unavailable to the downstream decision-maker and returns more than a score. In payment fraud detection, Visa A2A Protect returns transaction-risk scores, reason codes and other indicators to banks, with optional network-level intelligence (Visa product documentation: https://www.visa.co.uk/products/visa-protect-a2a-payments.html). In credit scoring, FICO Score XD combines telecom, utility, pay-TV and public-record information held by external providers and reaches lenders as a score with up to five reason codes. These are not SHIFT deployments, but the access pattern is real: the downstream institution can use derived outputs without receiving all the underlying fields.

There is a receiver-executed version in healthcare. Vendor early-warning models, including the Epic Deterioration Index and Sepsis Model, run inside hospital production systems on identified clinical records. Their patient-level scores and contributing factors are available to clinicians, while the institution's research datasets may have narrower access to sensitive records. Section 3.1 discusses these cases, along with existing obligations to provide specific credit-decision explanations and systems such as SageMaker Clarify that return per-record attributions.

None of these examples automatically permits a full SHAP vector to be passed to a downstream student. That is why Appendix B specifies the two execution topologies, a four-clause deployer's contract, a permissions matrix (Table 5) and audit steps. The distinction is between restrictions on which approved component may process a field and restrictions meant to protect the information itself. The former can permit an additional output if it is separately authorized; the latter may prohibit state altogether. Width is a further control, not a privacy guarantee. At width 4, the channel keeps 69% of the full-width AUC gain on the 24 population-shifted contexts, at a held-out probe R² of .17 against .05 for score alone, and still beats the recalibrated teacher on 6/7 tasks (Table 26).

"The contribution is primarily an empirical analysis of performance and mechanisms under restricted access, rather than a new representation or learning method."

Our contribution is the interface and its characterisation. We do not claim that leaf encodings, SHAP features or stacking are new. The central test holds the teacher, student and state channel fixed and varies only the student's feature entitlement. It gives an access-by-state interaction of +.0054 AUC (19/22, Holm below .001), and +.0018 under a random half (Table 13). At complete access, the same state adds only +.0004 for the declared primary. Thus the gain is tied to the restriction, not simply to adding another representation.

The interface also makes a prediction that could have failed: the state gain should track the headroom available from direct access to the withheld block. Across 110 units of six protocols it does (Spearman .56 and .53 on disjoint seeds, Figure 3 in Appendix M). On the IST countries, SHIFT-Raw even has lower NLL than a student that reads the withheld block directly (10/12, Section 5.4). This is consistent with state transferring information the source teacher has already learned about those features, rather than asking a small receiver to learn it again. That access-dependent effect and its limits are what we set out to establish.

AuthorsOpenReview ↗

comment 2 out of 2

"It is therefore unclear whether the method would consistently help when restricted features are less important or when applied to data from other organizations."

The question has two parts.

When the restricted features matter less. A withheld-fraction sweep on the 22 tasks (Table 25) withholds 25%, 50% and 75% of the features, once by the teacher's gain ranking and once at random. The state gain follows the direct-access headroom cell by cell, recovering .71 to 1.10 of it at every fraction. Even in the least favourable cell, a random quarter withheld, it recovers .71 of what there was to recover.

Data from other organizations. We ran two studies on separately governed silos. Each used a tuned blend with calibration parity against a teacher recalibrated on the receiver's own labels, and each protocol was frozen before its run. Table 20 records the predictions and outcomes.

  • Twelve national silos of the International Stroke Trial (Table 3, right panel). The channel the protocol named, SHIFT-Raw, lowers NLL by .0094 relative to the recalibrated teacher on 10/12 countries (Holm .010 over the two named contrasts). The score-only student does not.
  • Twenty-three UK police forces (Table 23). SHIFT-Raw lowers NLL by .0415 relative to the recalibrated teacher on 23/23 forces and by .0279 relative to the score-only student on 23/23. Per-tree margins improve NLL by .0068 over a student that reads the withheld block directly.

The 24 documented receiver contexts of seven tasks give the same answer on 7/7 tasks (Table 3, Section 5.4). Across the three suites, every state channel, including SHIFT-Q, has lower mean NLL than the recalibrated teacher in all 12 channel-by-suite cells (raw p at most .034). Per-tree margins, the cheapest and least exposing of the state channels under our probe, are the best arm on two of the three suites (Tables 3 and 23).

Where it pays. State pays where receiver populations differ and the receiver holds a few hundred labelled rows or more, which is the scale of every new suite above. Section 5.5 now says this explicitly. We do not claim uniform gains where the teacher already fits the receiver well or the withheld block provides little headroom.

Thank you for the response. However, two main concerns remain. First, the real-world examples do not sufficiently establish a practical need for the specific permission setting studied in this paper. Second, while the additional experiments strengthen the empirical contribution, I still find the methodological novelty limited. I therefore maintain my original assessment.

Reviewer_pbNwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This work examines what the internal state of a frozen model adds/leaks beyond its prediction. A source trains a boosted tree ensemble on all features and shares the trained model once with a receiver. The receiver has few labels and a different class balance. It may run the teacher on its records, but its own model may not read a withheld block of features. SHIFT therefore trains a student on the permitted features, the teacher's score, and the teacher's SHAP vector. It then averages the predictions of student and teacher.

On 22 OpenML tasks, the teacher's most important half of the features is withheld. Adding the SHAP vector improves AUC over a student that only sees the teacher's score by ~.006. When the student reads all features, the gain shrinks to .0004. The authors interpret this as evidence that the SHAP vector helps due to the restriction and not because of the extra columns.

I worked on representation learning and certainly may miss appropriate background which I have flagged to (S)ACs. I did my best to assess whether the experiments support the core claims of the authors. From what I can tell, I lean toward rejection, though nvite the authors to rebut my review.

Critical strengths

The paper considers a relevant and valuable research question: What does a frozen teacher's SHAP vector adds beyond its score when the student may not read all features. Eq. 1 formalizes this question cleanly and the author's access experiment tests it within a single setup.

Critical weaknesses

SHIFT is compared against a student that only sees the teacher's score, and both are averaged with the teacher at equal weight. The authors frame this as the comparison of interest. Given the gap in capacity and data between teacher and student, I would not expect the student to beat the teacher. The equal weighting, however, splits essentially half of the prediction to the weaker, smaller model, so the score-only baseline starts below the teacher. Specifically:

  • Averaging the teacher with the score-only student lowers AUC from ~.845 to ~.839 (Table 2). With the SHAP vector, the student merely returns to about the teacher's level.
  • The gain in log loss over the teacher (Table 6) is measured against a teacher that is adjusted to the receiver's class balance but not recalibrated.

In a nutshell, the SHAP vector mostly prevents the student from degrading the teacher's predictions. The missing baselines are thus the teacher recalibrated on the receiver's labels and an average whose weight is tuned on validation, both compared against the teacher alone.

Moreover, the paper argues that the SHAP vector helps under restriction because it carries information about the withheld features. This aligns with established notions of distillation. A linear probe recovers the withheld features far better with the SHAP vector than without it (cf. Table 18). Interestingly, the SHAP vector also appears to be as worth as much as the withheld features themselves (cf. subsection 5.1). However, any proxy of those features should help more once they are withheld. From what I can tell, the experiment thus shows that the withheld features matter, but crucially not what the teacher's SHAP vector adds beyond them, leaving much less added insight than claimed.

The paper frames SHIFT as cross-silo adaptation, but each of the 22 tasks splits a single dataset, so source and receiver only differ in their class balance. In such a setting, the teacher alone already ranks the receiver's records almost as well as any student (Table 3), which leaves little headroom:

  • Receivers are small, all main models are boosted trees, and the gains sit in the third decimal of AUC.
  • Beyond the 22 tasks, the evidence is mixed. A 12-task extension and one sensor dataset (DSADS) confirm the gain over the score-only student, while a 20-dataset suite and a second sensor dataset (GasDrift) do not.

More subjectively, to me, this submission reads in part too much LLM-generated (acknowledging agents are very helpful assistance). The authors "pre-register" low-level choices, e.g., which of two encodings counts as primary (cf. L187 to 189), and document commit timestamps (L512 to 516). Important details about baseline setups are offloaded to the supplement (cf. L236 & 248), which I'm not sure I have seen like that in an ICLR submission.

Questions for authors
  1. Does a low-rank SVD of the withheld features help the student as much as the SHAP vector? It would pass on their information more directly (more analogous to more classic knowledge distillation).
  2. Does SHIFT beat the teacher alone, once the teacher is recalibrated on the receiver's labels?
  3. How does SHIFT perform when source and receiver differ in more than their class balance?
2 comments
AuthorsOpenReview ↗

comment 1 out of 2

Thank you for a precise review. The comparison you identified as missing, a teacher recalibrated on the receiver's own labels under a validation-tuned blend, is now in the paper on the 22 tasks, the 12-task extension and three population-shift suites. Unless we say otherwise, the numbers below are under that tuned, calibrated rule.

"The missing baselines are thus the teacher recalibrated on the receiver's labels and an average whose weight is tuned on validation, both compared against the teacher alone." "Does SHIFT beat the teacher alone, once the teacher is recalibrated on the receiver's labels?"

We added both, and the answer to the second question is yes where receiver populations differ. There every state channel beats the recalibrated teacher on NLL in all three suites, including SHIFT-Q: 7/7 tasks, 9 to 10 of 12 countries and 23/23 forces (Tables 3 and 23, details in our second comment). On the 22 tasks, whose splits change only the class balance, SHIFT-Raw has the highest mean AUC of any arm under both rules. Under the tuned rule it is +.0024 over the teacher, with gains on 13/22 tasks (Tables 1 and 21). At the complete-feature interface it has the highest mean AUC of all 27 arms, above the teacher and each of the 21 published federated and one-shot systems (Table 2). The teacher contrasts are in Table 21, cited from Section 5.1.

"Any proxy of those features should help more once they are withheld. [...] the experiment thus shows that the withheld features matter, but crucially not what the teacher's SHAP vector adds beyond them." "Does a low-rank SVD of the withheld features help the student as much as the SHAP vector?"

The sketch has to read the withheld block, which the restriction forbids, so we ran it as a reference arm. At full rank it lands within .0007 AUC of direct access. SHIFT-Q is within .0005 of it, while SHIFT-Raw's mean AUC is .0024 higher (Table 22). We interpret this as near-parity with a representation of the withheld block, not evidence that SHAP is uniquely necessary. You are right that the claim is better stated as recovery, and the abstract and Section 5.1 now say so: the state recovers withheld-block information through an output the receiver is permitted to consume. Within the 22-task sweep it recovers .71 to 1.10 of the direct-access headroom at every withheld fraction (Table 25).

AuthorsOpenReview ↗

comment 2 out of 2

"How does SHIFT perform when source and receiver differ in more than their class balance?"

We ran three studies, all under the tuned, calibrated rule and against a teacher recalibrated on the receiver's own labels (Table 3 in Section 5.4, and Appendix M).

  • Twenty-four documented receiver contexts of seven tasks. Every state channel beats the recalibrated teacher on NLL on 7/7 tasks. An otherwise identical score-only student does not (Table 3).
  • Twelve national silos of the International Stroke Trial, under a protocol frozen before the run that named SHIFT-Raw (Table 3, right panel). SHIFT-Raw lowers NLL by .0094 relative to the recalibrated teacher on 10/12 countries (Holm-adjusted p .010 over the two contrasts the protocol named).
  • Twenty-three UK police forces, also under a protocol frozen before the run (Table 23). SHIFT-Raw lowers NLL by .0415 relative to the recalibrated teacher on 23/23 forces and by .0279 relative to the score-only student. Per-tree margins improve NLL by .0068 over a student that reads the withheld block directly.

Across the three suites that is 12 channel-by-suite cells, and every channel improves mean NLL over the recalibrated teacher in all of them (raw p at most .034), SHIFT-Q among them. Per-tree margins, the cheapest channel, are the best arm on the contexts and forces (Tables 3 and 23), and SHIFT-Raw is best on the countries (Table 3). Each study's pre-stated predictions and outcomes are in Table 20, Appendix L.

"the teacher alone already ranks the receiver's records almost as well as any student (Table 3), which leaves little headroom [...] the gains sit in the third decimal of AUC."

On the 22 tasks the gains are the size of the headroom, and the revision makes that quantitative. Over 110 units of six protocols the state gain tracks the direct-access gain, Spearman .56 and .53 with headroom and gain measured on disjoint seeds (.79 pooled, Figure 3 in Appendix M). On the 22 tasks direct access to the withheld half buys .0062 AUC, so third-decimal gains are close to all that was available. Under genuine population shift the NLL improvements are larger: per-tree margins lower NLL relative to the recalibrated teacher by .0222 on the contexts and .0561 on the forces (Tables 3 and 23).

"A 12-task extension and one sensor dataset (DSADS) confirm the gain over the score-only student, while a 20-dataset suite and a second sensor dataset (GasDrift) do not."

We retain the mixed findings, including the negative 20-dataset result. GasDrift does have headroom. Direct access to the withheld units improves NLL by .0151 over the score-only student (7/9 batches, p=.020), and Leaf-SVD and SHIFT-Raw recover .0089 and .0086 of it as point estimates, in a run added for this revision.

"The authors 'pre-register' low-level choices [...] document commit timestamps [...] Important details about baseline setups are offloaded to the supplement."

We made these changes. The primary-declaration passage is one sentence, the commit-order passage in the Statements is cut to its conclusion, the baseline access map is now Appendix C, and each new number carries the rule it was computed under. For the two silo studies the protocol freeze is stated in one sentence each, with the outcomes in Table 20.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.