- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies Deep Spurious Regression (DSR), a setting in which a continuous target is correlated with an observed attribute under the training distribution, while evaluation requires generalization across the full target range and across all target–attribute combinations. The paper argues that classification-oriented treatments of spurious correlation do not transfer directly, because continuous targets carry an ordering and meaningful local neighborhoods, and because certain target–attribute combinations may be sparse or entirely absent during training.
The proposed approach smooths the empirical training distribution along two axes. Along the target axis, attribute-conditional label distribution smoothing (LDS) shares information between nearby target values. Along the attribute axis, an affinity matrix transfers count information across related attributes. L-MDS constructs this affinity from Wasserstein-1 distances between per-attribute target distributions, whereas F-MDS uses distances between per-attribute feature centroids. In both cases, the pairwise distances are embedded into two dimensions via multidimensional scaling (MDS), followed by a row-normalized RBF kernel. The resulting smoothed counts define per-sample weights for weighted regression.
The methods are evaluated on UTKFace, SkyFinder, PovertyMap, and CodeNet, with race, camera ID, country, and programming language serving as the respective attributes. Results are reported in terms of overall MAE, per-attribute error, and error across many-, medium-, few-, and zero-shot target–attribute regions. The best proposed variant improves overall MAE over ERM on all four datasets, with the largest gains on CodeNet and in several sparse regimes. However, the improvements are not consistent across metrics or regions, and a number of claims in the experimental discussion are stronger than the tables support. The paper further includes additional baselines, kernel and loss ablations, reduced-data experiments, and a controlled interpolation/extrapolation study.
Critical strengths
-
The paper identifies an underexplored problem in continuous prediction. Continuous targets introduce ordering, local neighborhoods, interpolation, and missing target–attribute combinations that are not naturally captured by the finite group–label formulation standard in classification. A well-designed benchmark for these phenomena would broaden the study of robustness to shortcut correlations.
-
The proposed procedures are simple to integrate and are evaluated across several domains. L-MDS and F-MDS can be implemented as reweighting schemes without redesigning the underlying predictor. The evaluation spans visual, environmental, socioeconomic, and code-regression tasks, and the improvements on CodeNet are sizable—provided that the data construction and experimental protocol can be verified.
Critical weaknesses
-
The benchmark does not convincingly distinguish spurious regression from conditional target imbalance. The proposed methods smooth and reweight discretized
(y, a)counts toward the balanced or more-uniform evaluation distribution. This addresses joint target–attribute imbalance, but it does not demonstrate that the predictor has ceased to rely on a shortcut. Camera location, country, programming language, and demographic attributes may themselves carry legitimate target information. The paper offers no counterfactual intervention or direct measurement of shortcut use, and all evaluation is under a single balanced test construction, so spurious dependence is never separated from ordinary domain variation. This concern is particularly acute for CodeNet, where the training set is constructed with 1,500 samples per language and per-language target distributions described only as "Gaussian" (Appendix C.5); if these distributions are identical across languages, then the language–runtime correlation that motivates the benchmark does not exist by construction. -
The methodological contribution is incremental and lacks theoretical support. The method is essentially attribute-conditional LDS combined with an MDS/RBF affinity over observed attributes. No analysis is provided for when cross-attribute smoothing is statistically justified, when it reduces shortcut reliance, or when it transfers information between incompatible attributes and induces negative transfer. The choice of a two-dimensional MDS embedding is heuristic, and F-MDS relies on a single global centroid per attribute, which may primarily capture differences in target means rather than attribute-level feature geometry.
-
The approach assumes that the spurious attribute is known and fully observed. Both the training weights and the cross-attribute affinities require explicit attribute annotations. The paper does not address noisy, partially observed, or unknown shortcut attributes (acknowledged only as future work in the Limitations section), even though these are common in the deployment settings used to motivate DSR. This substantially narrows the scope of the formulation.
-
The reported experimental configurations and dataset statistics are internally inconsistent. Appendix D states that the main experiments use Gaussian smoothing with
k = 5, σ = 2, yet none of the main-table rows for the proposed methods is consistent with this configuration. For example, the UTKFace F-MDS row (overall MAE 7.22) coincides exactly with thek = 9, σ = 1entry in Table 20 across all average-based columns; the UTKFace L-MDS row (7.30) matches thek = 9, σ = 1entry rather than the statedk = 5, σ = 2entry (7.28); the SkyFinder L-MDS row (3.54) does not match the stated configuration (3.57) but instead matches entries such ask = 9, σ = 3; and the PovertyMap L-MDS row (0.486) matches thek = 9, σ = 2entry exactly. No validation-only model-selection protocol is described. In addition, the stated 10,222 White training samples out of 17,620 correspond to roughly 58%, not the reported 42.4% (which appears to be computed over the full dataset rather than the training split—the denominator should be clarified), and Table 29 reports a maximum per-attribute count of 1,159 for PovertyMap whereas Appendix C.4 reports 1,521 Tanzanian training examples. Because the proposed weights depend directly on these distributions, these inconsistencies prevent reliable reproduction of the headline results. -
The empirical advantage is inconsistent, and the contribution of MDS itself is not isolated. The paper does not compare MDS against applying the RBF kernel directly to the original distances, nor against alternative embedding dimensions. The raw results for the attribute-conditional LDS control underlying Table 5 are also omitted. Moreover, the "Ours (best)" rows select different variants for different columns. On SkyFinder, RnC attains lower overall MAE than every proposed variant, while all proposed variants degrade worst-case zero-shot MAE relative to ERM; the methods also frequently sacrifice many-shot performance. As a whole, the tables do not show a consistent benefit for any one fixed variant.
Questions for authors
-
Main tables vs. Appendix D. The configuration stated in Appendix D (
k = 5, σ = 2) does not appear to be the one that produced the main-table rows — UTKFace F-MDS (7.22) matches thek = 9, σ = 1entry in Table 20 exactly, and similar mismatches show up for SkyFinder and PovertyMap. Could the authors clarify which setting was actually used for each dataset and method, together with the binning rule andK, the exponentα, weight clipping/normalization, MDS dimension, F-MDS update interval, seeds, and the validation criterion used for selection? Two smaller discrepancies: is the 42.4% figure for White in UTKFace computed over the full dataset or the training split (10,222/17,620 ≈ 58%), and which count is correct for PovertyMap — the 1,159 maximum in Table 29 or the 1,521 Tanzanian examples in Appendix C.4? -
The CodeNet construction is unclear to me. Appendix C.5 says every language has 1,500 training samples and that the target distribution "follows a Gaussian distribution," but it does not say whether these Gaussians differ across languages. What are the per-language means, variances, and supports? If the distribution is the same for all 13 languages, I do not see what induces the language–runtime correlation that the benchmark is built around.
-
What evidence is there that models in these benchmarks actually take shortcuts, as opposed to suffering from joint
(y, a)imbalance? Quantifying how predictive the attribute alone is, or probing whether predictors trained with and without the proposed weighting differ in their reliance on attribute information, would go a long way here. Evaluation under a shift other than the prescribed balanced test set would also help — right now, the claim of spuriousness rests mostly on the framing. -
Is MDS itself contributing anything? It seems the RBF kernel could be applied directly to the Wasserstein or centroid distances without the intermediate embedding. Was this simpler variant tried, and if so, how does it compare? Relatedly, for F-MDS, does a single global centroid per attribute work as well as per-bin or target-conditioned centroids, which might avoid collapsing attributes that differ mainly in target mean?
-
The "Ours (best)" envelope obscures which algorithm one should actually use: F-MDS wins overall on UTKFace, L-MDS wins on PovertyMap and CodeNet, and on SkyFinder no proposed variant beats RnC. Could the authors report, for a single variant chosen on validation data alone, raw means/stds and paired comparisons against the strongest baseline in each regime? I suspect the sparse- and zero-shot claims would need some revision under this fixed-method analysis, but it would make the empirical case much more convincing either way.
10 comments
Response to Reviewer CcXZ (2/8)
Q2: The CodeNet construction is unclear to me…
Thank you for pointing out that Appendix C.5 can have more detailed language-specific parameters. We are happy to address uncertainties in the CodeNet experiment construction. The training set distribution does differ by language. We use Gaussian-shaped sampling weights with language-specific centers over the target range (0, 1000] ms. The different distributions per language induce the language–runtime correlation that our CodeNet benchmark is built on. The empirical runtime statistics for each language are listed below:
| Language | Mean | Standard deviation | Observed range |
|---|---|---|---|
| C | 318.4 | 175.7 | 1–900 |
| C# | 510.6 | 200.1 | 20–1,000 |
| C++ | 173.9 | 132.5 | 1–703 |
| D | 87.3 | 118.4 | 1–910 |
| Go | 389.5 | 244.3 | 1–1,000 |
| Haskell | 243.2 | 259.1 | 1–1,000 |
| Java | 227.0 | 147.7 | 30–835 |
| JavaScript | 203.2 | 153.5 | 20–920 |
| Kotlin | 491.2 | 208.6 | 79–999 |
| PHP | 64.6 | 96.7 | 9–650 |
| Python | 199.7 | 140.1 | 10–777 |
| Ruby | 406.9 | 188.1 | 7–994 |
| Rust | 307.9 | 181.9 | 2–1,000 |
We have clarified the language-dependent sampling procedure and included the empirical means, standard deviations, and observed runtime ranges for all 13 programming languages in Appendix C.5 (Table 45).
W1&Q3: What evidence is there…as opposed to suffering from joint (y, a) imbalance?
We thank the reviewer for raising this important distinction. We agree that attributes such as camera location, country, and programming language may carry legitimate target information, and we do not assume that all attribute-related information is spurious. Our concern is whether models rely on attribute–target associations that are predictive during training but become unreliable under distribution shift. To distinguish this reliance from the effects of joint imbalance, we examine the predictive power of the attribute alone, evaluate robustness beyond the balanced test set, and directly probe shortcut reliance through attribute interventions on both synthetic and real-world datasets.
Predictive power of the attribute alone. We evaluate how predictive the attribute alone is using two measures. First, we fit an attribute-only regression model separately on the training and test splits, using the mean target value of each attribute group to measure how much target variance the attribute explains within each distribution (). Second, we use the attribute-group means estimated from training to predict the test targets without refitting, measuring whether the learned attribute–target associations transfer to the test distribution. We also compare the transferred predictor against the global training-set mean.
| Dataset | Train | Test (within split) | Test (transferred) | Test MAE (attribute vs. global mean) |
|---|---|---|---|---|
| UTKFace | 0.131 | 0.056 | −0.091 | 23.13 vs. 22.25 |
| SkyFinder | 0.382 | 0.230 | 0.186 | 8.63 vs. 9.55 |
| PovertyMap | 0.626 | 0.068 | 0.039 | 0.756 vs. 0.778 |
| CodeNet | 0.366 | 0.005 | −0.117 | 314 vs. 297 |
Across all four datasets, attributes explain substantially more target variance during training than at test time. Notably, attribute-only predictors trained on UTKFace and CodeNet perform worse than the global-mean baseline at test time, while the predictive power of attributes also declines substantially on PovertyMap and SkyFinder. These findings highlight spurious attribute–target correlations that are predictive during training but become unreliable at test time, providing potential shortcuts for models to exploit.
Response to Reviewer CcXZ (3/8)
W1&Q3: What evidence is there…as opposed to suffering from joint (y, a) imbalance? (Continued)
We next examine how model performance changes when these correlations shift.
Robustness beyond the balanced test set. We agree that evaluating additional test distributions can provide a more comprehensive assessment of robustness to spurious correlations. Nevertheless, evaluating on a balanced test distribution is a common practice in prior work on spurious correlations. For example, Group DRO [1] uses a 95% bird–background correlation during training on Waterbirds and a balanced test distribution. Similarly, IRM [12] evaluates ColoredMNIST under a shifted color–label correlation (10% at test time versus 80–90% during training). While these evaluation protocols are widely adopted, we further extend our evaluation to address the reviewer's concern. Specifically, we importance-reweight the saved test predictions [11], without retraining, to evaluate three attribute–target dependencies: training-aligned, balanced, and reversed. The reversed setting emphasizes attribute–target combinations that were underrepresented during training, allowing us to assess model robustness when the spurious correlations are reversed.
| Dataset | Method | Train-aligned | Balanced | Reversed | Reversed − Train |
|---|---|---|---|---|---|
| UTKFace | ERM | 6.75 ± 0.03 | 7.39 ± 0.07 | 7.69 ± 0.12 | 0.95 |
| L-MDS | 6.74 ± 0.19 | 7.30 ± 0.09 | 7.64 ± 0.10 | 0.90 | |
| F-MDS | 6.73 ± 0.09 | 7.22 ± 0.15 | 7.52 ± 0.23 | 0.79 | |
| SkyFinder | ERM | 2.84 ± 0.04 | 3.68 ± 0.03 | 5.27 ± 0.04 | 2.43 |
| L-MDS | 2.84 ± 0.02 | 3.54 ± 0.02 | 4.76 ± 0.07 | 1.92 | |
| F-MDS | 2.86 ± 0.03 | 3.56 ± 0.02 | 4.74 ± 0.08 | 1.88 | |
| PovertyMap | ERM | 0.416 ± 0.026 | 0.504 ± 0.009 | 0.586 ± 0.013 | 0.170 |
| L-MDS | 0.407 ± 0.002 | 0.486 ± 0.009 | 0.562 ± 0.012 | 0.155 | |
| F-MDS | 0.417 ± 0.018 | 0.488 ± 0.003 | 0.551 ± 0.010 | 0.134 |
Under training-aligned dependence, ERM performs comparably to L-MDS and F-MDS across all three datasets. However, when the attribute–target correlations are reversed, ERM experiences a larger increase in MAE, particularly on SkyFinder and PovertyMap, where both L-MDS and F-MDS achieve lower errors and smaller performance gaps. These results suggest that our methods reduce reliance on unstable spurious correlations and improve robustness beyond the original balanced test distribution.
We further evaluate ColoredRotatedMNIST under varying strengths of spurious color–angle correlations ().
| Method | Train-aligned | Balanced | Reversed | Reversed − Train | |
|---|---|---|---|---|---|
| 0.75 | ERM | 7.29 ± 0.26 | 8.90 ± 0.10 | 9.68 ± 0.10 | 2.39 |
| L-MDS | 8.10 ± 0.17 | 8.65 ± 0.06 | 8.99 ± 0.09 | 0.89 | |
| F-MDS | 8.08 ± 0.15 | 8.55 ± 0.04 | 9.08 ± 0.15 | 1.01 | |
| 0.90 | ERM | 7.27 ± 0.18 | 13.15 ± 1.16 | 16.01 ± 1.43 | 8.74 |
| L-MDS | 8.86 ± 0.60 | 10.68 ± 0.86 | 11.22 ± 0.37 | 2.36 | |
| F-MDS | 8.95 ± 0.45 | 10.48 ± 0.34 | 11.97 ± 0.36 | 3.02 | |
| 0.95 | ERM | 6.75 ± 0.07 | 22.28 ± 2.36 | 22.72 ± 2.26 | 15.97 |
| L-MDS | 9.35 ± 1.26 | 14.22 ± 1.31 | 15.30 ± 2.24 | 5.95 | |
| F-MDS | 9.17 ± 0.33 | 13.23 ± 1.01 | 15.25 ± 1.15 | 6.08 |
As the spurious correlation strengthens, ERM becomes increasingly sensitive to changes in attribute–target dependence, with its reversed-minus-training MAE gap increasing from 2.39° to 15.97°. In contrast, L-MDS and F-MDS consistently exhibit substantially smaller gaps across all three settings, demonstrating improved robustness under stronger spurious correlations.
Response to Reviewer CcXZ (1/8)
We sincerely thank the reviewer for the thoughtful and detailed feedback! We appreciate your recognition of the value of studying spurious correlations in continuous regression and the practical simplicity of our proposed methods. Below, we provide additional explanations and experimental results to address your concerns. We hope these clarifications strengthen our submission.
Thank you for pointing out these issues. We clarify our hyperparameter selection protocol and the dataset statistics below.
Hyperparameter selection. All hyperparameters were selected using the validation split only, with the test set reserved for final evaluation, following the protocol in [6]. For L-MDS, we search kernel size k∈{5,9,15} and σ∈{1,2,3}. For F-MDS, we search k∈{3,5,9} and σ∈{1,2}, together with inverse and square-root-inverse reweighting. Each configuration is trained with five random seeds. For each dataset and method, we select the configuration with the lowest mean validation MAE across the five runs and report its test performance over the same five seeds.
The original Appendix D reported the implementation defaults (k=5, σ=2), rather than the dataset-specific settings used for the main results. We have clarified this distinction in Appendix D (Table 46) and provided the exact configurations:
For reproducibility, we also provide the remaining implementation details in Appendix D (Table 47):
Dataset statistics. The reviewer also pointed out two discrepancies in our dataset descriptions, which we clarify below.
UTKFace. The reviewer is correct that 10,222 / 17,620 is 58%. The counts of 10,222 (42.4%) and 1,711 (7.1%) in the original Appendix C.2 refer to the full dataset of 24,103 images before splitting. In the training split of 17,620 images, the White and Others groups contain 8,392 (47.6%) and 830 (4.7%) images, respectively, consistent with Table 43.
PovertyMap. Our benchmark contains 20 attribute groups across 18 countries, with Tanzania and Zimbabwe each divided into rural and urban groups. The 1,159 images in Table 43 correspond to rural Tanzania, while the 1,521 images reported in Appendix C.4 include both rural (1,159) and urban (362) Tanzania. The smallest group is Togo, with 97 images. All weights, affinities, and shot regimes were computed using these 20 attribute groups.
We have corrected and clarified the dataset statistics in Appendices C.2 and C.4, clarified the PovertyMap attribute definition in Section 4, and included the selected hyperparameters and implementation details in Appendix D (Tables 46–47). To support the community and enable future research, we remain committed to fully open-sourcing our research.