- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper adapts a source predictor to a target population using a correction learned from labeled target data. Cross-fitting allows all labeled observations to contribute to rectification. A joint estimator separately weights the source prediction and the learned correction. The analysis gives fixed-weight unbiasedness, asymptotic distributions, an asymptotic no-harm comparison, and sparse-calibration risk bounds. Experiments evaluate mean and regression targets across several data modalities.
Critical strengths
Separately weighting the source prediction and transfer correction provides a useful safeguard. The oracle comparison follows transparently from the nested estimator classes, and the sparse-calibration bounds quantify the cost of learning the correction.
The empirical study covers several inference targets and modalities and reports coverage and small-sample failures. For example, the Bikeshare results show a relative-MSE reduction from 0.211 for PPI++ to 0.139 for the joint estimator at n=100.
Critical weaknesses
The abstract overstates the guarantees for the feasible jointly tuned estimator. Proposition 3 establishes unbiasedness for fixed weights, while the implementation estimates weights using the same gold responses used for rectification. Cross-fitting the correction does not automatically preserve exact unbiasedness after this additional estimation step. Theorem 3 gives an asymptotic comparison, not finite-sample protection against negative transfer. The body is more careful, but the abstract needs correction.
The comparison with existing Cross-PPI is inaccurate. The paper contrasts it with pretrained-model adaptation, but Zrnic and Candès explicitly allow fine-tuning an off-the-shelf model in Section 3.1. Appendix F.5 evaluates a scratch-trained implementation, which does not resolve this overlap. The contribution should be positioned through joint tuning and the additional risk analysis.
Questions for authors
The abstract attributes exact unbiasedness to the jointly tuned estimator, but Proposition 3 assumes fixed weights. What argument justifies exact unbiasedness when the weights are estimated from the same gold responses used for rectification? If no such argument applies, please qualify the abstract accordingly.
How often did the default [0,1]^2 weight constraints bind in the experiments, and do the main efficiency comparisons change when the weights are unconstrained? This would clarify the empirical relevance of Theorem 3's unrestricted oracle comparison.
3 comments
Thank you for the additional experiments. My main concern about the Cross-PPI comparison remains as it explicitly allows fine-tuning a pretrained model. I maintain my original rating.
Concerns about AI Usage
We'd like to note that this reviewer's original review was rated 100% AI by Pangram (https://www.pangram.com/history/167b3d3c-5843-4d34-87d0-54e33331b842). As this reviewer has continued to provide nothing substantive or constructive and, based on the latest response, seems not to comprehend our original response in any meaningful sense (we already addressed the concern in our initial response), we have nothing further to add to this reviewer thread.
Response to Reviewer
We thank the reviewer for taking the time to review our paper.
We will change the wording in the abstract in the final revision, as this is a minor stylistic concern. As you have noted, we make the distinction clear in the body of the paper. The method is unbiased with oracle fixed λ, while it still remains consistent with estimated λ; both properties are exactly the same as in PPI++.
We have used Cross-PPI as described by the authors of [1]. The paper's theory treats the training algorithm as a black box, requiring only that the fitted models be stable. A fine-tuned model, or a trained-from-scratch model, is just the learned predictor, and there are no guarantees relative to the pretrained model and no consideration of the costs or benefits of adaptation. The cross-PPI paper focuses on training an algorithm using the gold sample data. The methods, theory, applications, and code in [1] all point to that, and we have used the method exactly as the authors intended.
In Figure 8 (Appendix F.5), Cross-PPI and TC-Cross-PPI are compared apples-to-apples since they both start from the source model, use the same gold labels and cross-fitting folds, and neither is power-tuned. Cross-PPI trains an instance of the source model from scratch on the gold labels, while TC-Cross-PPI keeps the source model fixed and learns only a sparse correction from the same gold labels. In this experiment, we sought to isolate the value of starting from the source model. Joint TC-Cross-PPI++, also shown in Figure 8, then keeps the source prediction and the learned correction as separate terms and power-tunes them jointly. We will make this point clearer in a revised version.
A constraint on the weights for general M-estimation targets guarantees convexity (the Bradley-Terry applications). As in the original PPI++ paper [2]: weights outside [0,1] can make the rectified objective nonconvex. For mean targets, unconstrained weights are valid, and removing the box matters only when the source-to-target transfer depends little on the covariates (e.g., an affine relationship between the prediction scores, which rescaling captures). The table below reports mean Classical MSE / mean Method MSE with the [0,1]2 box → without it. Census Income and Capital Bikeshare change the most, while all other settings barely change.
For the Bradley-Terry targets, following [2], we kept each weight in [0,1], but the region can be loosened to 0≤λs+λΔ≤1. PPI++ restricts its single weight to [0,1] so that the rectified GLM loss stays convex [2]; with our two pseudo-label terms in Eq. (61), the same argument depends only on the total weight λs+λΔ, allowing individual weights to be outside of [0,1]. The loss then stays convex, so its minimizer is the root mentioned in Theorem 10. Below we report the percentage of trials in which each constraint type (box vs sum) binds for Joint TC-Cross-PPI++, and report the mean Classical MSE / mean Method MSE for each.
Under the box constraint, binding mostly occurs through λ^Δ, while the sum constraint rarely binds. The metric is essentially unchanged.
References
[1] Zrnic & Candès. Cross-prediction-powered inference. PNAS, 2024.
[2] Angelopoulos, Duchi & Zrnic. PPI++: Efficient prediction-powered inference. 2023.