- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how to choose a regression-target transformation when predictions are evaluated on the original scale. It identifies two competing effects: retransformation bias, which arises when inverting the transformed conditional mean, and resolution, which describes how a transformation allocates the learner’s accuracy across the target range. The authors derive a small-noise risk expansion combining these effects and prove, under an idealized learner model, that the target distribution alone cannot universally determine the optimal transformation. Controlled experiments investigate these mechanisms, while experiments on 41 tabular datasets compare transformations across several learners. The main practical conclusion is to standardize transformed targets and select the transformation using validation performance.
Critical strengths
-
The paper addresses an important and practical problem. Target transformations are widely used, yet their effects on original-scale prediction error are often poorly understood. Explaining why the same transformation helps one regression problem but hurts another is valuable.
-
The theoretical framework provides an interesting synthesis. Combining classical quantization results with retransformation bias offers a coherent explanation of competing effects. The impossibility theorem provides a precise illustration of why information about the relationship between features and targets matters.
-
The controlled experiments are thoughtfully designed. Holding the target distribution fixed while varying predictability helps distinguish marginal-distribution effects from conditional-distribution effects. The bias-correction experiments provide additional evidence that retransformation bias influences the preferred transformation.
Critical weaknesses
-
There remains a substantial gap between the theoretical learner and practical models. Definition 1 assumes a centered learning error independent of the input, with a transform-independent shape. However, Table 1 shows strong error heterogeneity: the median maximum-to-minimum decile error ratio is 11.8 for the standardized MLP and 23.6 for HGB. Even after variance normalization, errors vary substantially across transformations. Proposition 1 accommodates more general errors, but their dependence on the transformation remains unspecified. Consequently, the framework explains some qualitative trends without reliably predicting practical optima.
-
The guidance for selecting target transformations is still unclear. While the paper offers valuable theoretical insights into target transformations, translating these insights into actionable selection guidelines remains challenging. The practical recommendation still largely relies on validation-based selection. It would strengthen the paper to clarify when the proposed diagnostics can simplify this process or offer benefits beyond standard validation-based tuning.
While my expertise in this area is limited, my overall assessment of the paper is positive, and I recommend weak acceptance.
Questions for authors
Please see the weaknesses.
4 comments
Response to Reviewer y8mr (2/2)
W2. The guidance for selecting target transformations is still unclear, and the practical recommendation still largely relies on validation-based selection. Please clarify when the proposed diagnostics can simplify this process or offer benefits beyond standard validation-based tuning.
A2. We agree that validation remains the final selector, and the revised abstract reports only that validation achieved lower aggregate selection regret than the target-only selectors we evaluated. Theorem 3 gives a formal reason within the idealized model: no fixed rule that uses only the target marginal is asymptotically optimal for every joint law. The practical role of is to prioritize candidates using training targets before fitting; validation then selects among the fitted candidates. The analysis provides the following guidance:
- Which candidates to fit when fits are limited (new Appendix Y). At each fit budget, the -based, fixed-CRC, and random shortlists use the same number of model fits, with final selection by validation RMSE. All ratios below are test RMSE relative to Std3, the validation-selected choice among raw targets, log1p and Box–Cox.
- For the MLP, adding the three family members with the smallest to these three standard transforms (six fits) gives 0.986.
- Adding the three CRC variants to the same standard set gives 0.992, while adding three random family members gives 0.999.
- With nine fits, the mean paired ratio to validation within the 63-configuration pool is 1.020.
- The gain is modest, does not appear for HGB (whose stored pool is narrower), and does not consistently reduce the 90th percentile of regret. The p-values are exploratory.
- Which candidates to treat cautiously. The uniform quantile transform is worse than CRC on 38–41 of 41 datasets across the four nonlinear learners tested, and on both new frozen-embedding tasks. The larger penalties of the neural learners coincide with more predictions near the steep upper end of the inverse map (Appendix U).
- How to train. Standardizing the transformed target changes how the learner's error scales with the transform and with it the ranking (Section 5), so every candidate should be standardized the same way.
- How to read . tracks how much transforms that place resolution differently can differ (Spearman 0.76 with the CRC-log/CRC contrast; Appendix S). We did not detect an association between and the gain from selection itself, so we no longer use an threshold to decide whether transformation selection is worthwhile. Instead we recommend always including raw targets. For the MLP, validation selection over the evaluated candidate pool gave a mean per-seed test RMSE ratio over 20 seeds (selected/raw) below 0.98 on 9 of 22 low-leverage datasets and above 1.02 on none.
We thank the reviewer again for the constructive review.
I thank the authors for their detailed response. The revisions strengthen the paper and address most of my concerns. Thus, I maintain my positive score of 3.
Thank You for Your Feedback
We sincerely appreciate the reviewer’s acknowledgment that our revisions have strengthened the paper and addressed most of their concerns, as well as their continued positive evaluation of our work. The reviewer’s insightful feedback prompted us to further investigate the gap between theory and practice and provide clearer practical guidance for selecting target transformations.
Response to Reviewer y8mr (1/2)
We sincerely thank the reviewer for the careful review, positive evaluation, and insightful comments, which helped us further improve the paper. To address the reviewer's concerns, we added two new analyses and made our statements more precise. All changes in the revised PDF are marked in blue. We provide our responses below.
A1. We agree with this characterization, and the revision states it explicitly. In the introduction, the answer to the third research question now reads: "It captures qualitative trends; a controlled synthetic diagnostic shows how error location contributes to discrepancies." To measure that contribution, new Appendix T (summarized in Section 5) reports a controlled synthetic diagnostic.
Design.
Results.
Limits.
In this controlled setting, the diagnostic shows that where a learner makes errors can materially change original-scale risk even when the transformed-space error distribution is held fixed. This identifies error location as one contributor to the gap between scalar idealizations and trained learners. How a learner's error profile depends on the transform remains open, as stated in the Discussion (Section 8).