acceptodds
Withdrawn from ICLR 2027

Endpoint Exposure and Constraint Coverage in Mixup under Spurious Correlations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Selective Mixup changes both interpolation and endpoint sampling. We study these effects using two clean-endpoint baselines. The policy-matched baseline follows the original pair sampler and coefficient weights, whereas the marginal-matched baseline reproduces pooled group-position exposure with independently sampled endpoint groups. For binary labels and attributes, we define Disentangling Pair Exposure (DPE) as the probability that a sampled pair differs in exactly one variable. DPE and complementary same-polarity coverage jointly determine endpoint-polarity balance. In a linear squared-loss model with lambda drawn from a Beta(alpha, alpha) distribution and finite alpha > 0, Mixup and endpoint training uniquely identify the noiseless core and spurious signal coefficients if and only if both endpoint polarities have positive mass. Positive DPE is sufficient but not necessary for noiseless signal identification, while pair topology can change objective curvature. Experiments on synthetic data, Colored MNIST, biased CIFAR-10, Waterbirds, and CelebA show that endpoint sampling alone can produce substantial changes in worst-group accuracy. With balanced synthetic endpoint groups at source agreement probability rho = 0.98, mean Mixup worst-group accuracy ranges from 90.34% to 90.59% as DPE varies from zero to one. The remaining Mixup–endpoint differences depend on the pair relation and interpolation space. These results motivate evaluating selective Mixup together with clean-endpoint baselines derived from its sampling policy.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Mixup, spurious correlations, group robustness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xD1rOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The author argue that selective mixup methods (e.g., LISA) confound the effect of interpolation and sampling of training samples, and that both can have an effect in improving worst-group accuracy (WGA, per the author's acronym). This is because pairing by both label and spurious attribute oversamples rare groups, which is known to already help by rebalancing the samples.

To separate the two effects, the authors propose the so-called "clean-endpoint baselines", which run the same pair sampler on unmixed examples. They classify pairs in a confusion-matrix style (SS/SD/DS/DD) based on whether they have the same label and attribute, and introduce DPE as the measure of the samples that differ in exactly one of the two. They then empirically show that much of WGA changes are due to endpoint training alone.

Critical strengths
  1. Clean experiments: the paper compares each mixup method against a baseline with the same exact settings but trains on two original images separately. This means that any difference must be due to the interpolation. The same goes for the synthetic experiment.
  2. All experiments are ran multiple times (although seeds are confusing, see weaknesses). The code is provided in the supplementary material.
Critical weaknesses
  1. The paper does not frame itself in the literature. There is no dedicated Related Work section, and section 2 is a lengthy discourse on generic preliminaries and related work with not a single citation. In the journey of trying to understand what the author's contributions were, I found this paper [1], which seems to be extremely correlated, both in narrative and conclusion. The fact that not only this paper, but also the other literature, is missing, makes it extremely hard to understand what the author's contributions are. They state them briefly in the introduction, but that segment is not informative, and if all of section 2 is established literature, the contribution of the paper is narrow.

  2. The experimental settings are very confusing. The authors claim to be using 5 different datasets, but only 3 are reported in the main text, and 2 in the appendix (Colored MNIST and biased CIFAR). To the best of my understanding, they are also used for different experiments (endpoint decomposition and input vs feature comparison). This needs to be mentioned explicitly and explained in the paper, because it is not clear. Also, each dataset use a different number of seeds, which is confusing. Why can't they all be standardized to the same task and seeds?

[1]. @InProceedings{pmlr-v235-teney24a, title = {Selective Mixup Helps with Distribution Shifts, But Not ({O}nly) because of Mixup}, author = {Teney, Damien and Wang, Jindong and Abbasnejad, Ehsan}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {47948--47964}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/teney24a/teney24a.pdf}, url = {https://proceedings.mlr.press/v235/teney24a.html}, }

Questions for authors

See the two weaknesses. The framing in the literature and understanding exactly what the contribution is and what, instead, is established literature, is particularly critical.

Reviewer_qex2OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates how selective Mixup affects model robustness through two factors: endpoint sampling and interpolation. The authors introduce several new concepts to facilitate the theoretical analysis of pair relationships and endpoint sampling in Mixup. The authors provide useful theoretical insights into signal identifiability under a simplified linear squared-loss model, but the generalizability of their analysis to nonlinear networks and practical loss functions remains unclear. Experiments on synthetic and real-world benchmarks show that endpoint sampling substantially affects worst-group accuracy, motivating clean-endpoint baselines for fairer evaluation.

Critical strengths

1.The paper provides a useful theoretical analysis of how endpoint sampling and pair relationships affect signal identifiability in Mixup, offering insights into the limitations of evaluating Mixup solely through interpolation-based comparisons.

2.The experiments cover multiple synthetic and real-world benchmarks, demonstrating that endpoint sampling alone can substantially affect worst-group accuracy and motivating more controlled evaluation protocols.

Critical weaknesses
  1. Clarity of the Abstract. The abstract introduces too many technical concepts in rapid succession, making it difficult to identify the central research question and main contribution. The authors should clarify the motivation and streamline the presentation to better highlight the paper's key message.

  2. The theoretical analysis is restricted to a linear model with squared loss, whereas the experiments use nonlinear neural networks and practical classification losses. The paper does not sufficiently establish how its theoretical conclusions extend to these settings, limiting the explanatory power of the analysis.

  3. The selective Mixup strategies studied in this paper rely on known class and attribute labels, which limits their applicability in realistic settings where spurious attributes are often unavailable or difficult to annotate. Moreover, the theoretical analysis assumes a single binary spurious attribute, leaving it unclear whether and how the conclusions extend to settings involving multiple spurious attributes. Also, the paper does not sufficiently discuss these limitations or potential extensions.

Questions for authors
  1. Figure 1: Visualization of data distributions. Figure 1 is difficult to interpret because the original data distribution is not shown. It is also unclear why the blue groups appear smaller than the yellow groups under both vanilla Mixup and selective Mixup. The authors should include the original data distribution as a reference and clarify whether the observed differences arise from the source data distribution or the pairing policies.

  2. Line 66: Exact decomposition of endpoint-polarity balance. The meaning of “Together with complementary SS/DD coverage, it gives an exact decomposition of endpoint-polarity balance” is unclear. The authors should explain what is meant by an “exact decomposition” and clarify how direct-pair exposure and complementary SS/DD coverage jointly determine endpoint-polarity balance.

  3. Line 73: Balanced synthetic endpoint exposure. The term “balanced synthetic endpoint exposure” is not sufficiently clear. The authors should specify what is being balanced, whether the balance refers to polarity marginals or group-level endpoint exposure, and how this condition is implemented in the synthetic experiments.

  4. Notation and symbol redundancy. The paper introduces numerous symbols, some of which appear redundant or potentially confusing. For example, the relationship between (\rho) and (e_-), as well as that between (\pi_g) and (\pi_{y,a}), should be clarified. The notation for polarity and attribute values also needs to be used more consistently. The authors should consider consolidating redundant definitions and avoiding the reuse of symbols for different quantities.

  5. Line 160: Balanced SS/DD mass across polarities. The phrase “the balanced SS/DD mass available across polarities” is difficult to interpret. The authors should clarify whether “balanced” refers to equal total mass across the two polarities or to a particular allocation of SS and DD pairs, and provide a precise mathematical definition.

  6. Introducing new variables and concepts. The paper introduces many new variables and concepts in a relatively short space, making the theoretical analysis difficult to follow. Figures or schematic examples, similar to Figure 2, would help illustrate the relationships among endpoint exposure, pair topology, DPE, and complementary coverage. A compact summary of the main variables and their relationships would also improve readability.

  7. Section 2.2: Direct and complementary support. The terms “direct support” and “complementary support” are difficult to interpret from the current presentation. In addition, the motivation and content of Section 2.2 are not sufficiently aligned with its title. The discussion of different notions of balance around Line 182 would benefit from clearer definitions and a more systematic explanation of how these notions differ and relate to one another.

  8. Section 2.3: Relationship between the fixed-marginal DPE bound and the only-SD/only-DS examples. The connection between the fixed-marginal DPE bound and the only-SD/only-DS examples is unclear. The bound assumes that both endpoint positions preserve the source polarity marginal, whereas these examples analyze pooled endpoint group exposure. The authors should clarify that these pairing strategies change at least one endpoint's polarity marginal, thereby allowing DPE to exceed the bound, and explain how this relates to the derived endpoint exposure formulas.

  9. Overall organization of the sections. The paper contains many subsections, but the relationships among them are not always clear. The authors should provide a clearer roadmap at the beginning of each major section, explaining how the subsections build on one another and how each contributes to the paper's overall argument. Brief summaries or transitions between subsections would also help improve the logical flow.

  10. Section 3: Motivation for the linear population model. The motivation for introducing the linear population model is unclear. The authors should better explain why the zero-DPE setting is considered first and how signal-coefficient identifiability connects to the preceding analysis of endpoint exposure and pair topology.

  11. Connection between the theoretical analysis and empirical robustness. The connection between the theoretical analysis and the empirical results remains incomplete. The theory establishes endpoint-support conditions for signal-coefficient identifiability under a simplified linear model with squared loss, whereas the main experiments use deep neural networks trained with binary cross-entropy and evaluate robustness through worst-group accuracy (WGA). Although the controlled experiments support the empirical importance of balanced endpoint exposure, the paper does not sufficiently explain how the theoretical identifiability results translate into improved representation learning or WGA in these practical settings. A clearer discussion of the scope and limitations of the theory, together with a more direct validation of its implications for the empirical results, would strengthen the paper.

Reviewer_sEbpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies why selective Mixup improves worst-group accuracy (WGA) under spurious correlations, separating the effect of which examples are paired from the effect of interpolation. It introduces two references that train on the sampled pairs without mixing them: a policy-matched reference that keeps the original pair sampler and mixing coefficients as loss weights, and a marginal-matched reference that samples endpoint groups independently from the policy's overall group exposure. For a binary label and a binary spurious attribute, it defines Disentangling Pair Exposure (DPE), the fraction of pairs that differ in exactly one of the two, and analyzes a linear squared-loss model in which Mixup and endpoint training recover the core signal if and only if both label-attribute agreement and disagreement appear among the endpoints.

Critical strengths
  • The paper separates the question of whether selective Mixup helps through interpolation or through the sampling distribution it induces into measurable components (pair topology, endpoint exposure, and interpolation space), which is a useful lens for interpreting this family of robustness methods.
  • The experimental reporting is careful: matched seeds, paired confidence intervals computed within seed, achieved rather than nominal pair statistics, and detailed implementation and logging descriptions.
Critical weaknesses
  1. The paper is very hard to read, and its core message is buried under new terminology. The main finding, that selective Mixup improves worst-group accuracy largely by changing which groups are sampled rather than by interpolation, could be stated in one sentence, but the paper wraps it in a large set of new terms. For example, the third sentence of the abstract ("The policy-matched baseline follows the original pair sampler and coefficient weights, whereas the marginal-matched baseline reproduces pooled group-position exposure with independently sampled endpoint groups") cannot be understood without Section 2, and the abstract further relies on polarity, complementary same-polarity coverage, endpoint-polarity balance, pair topology and objective curvature without defining them; the one-line definition of DPE ("differs in exactly one variable") does not say which variables are meant. Figure 1 shows no data (the caption states that "bars are illustrative"), and its labels (e.g., "much of the gain remains") state the conclusion rather than illustrate a mechanism. The burden is aggravated by the fact that the most prominent new quantity, DPE, ultimately plays a secondary role.

  2. The central claim and method were already presented by Teney et al. (ICML 2024), which is not discussed. "Selective Mixup Helps with Distribution Shifts, But Not (Only) because of Mixup" shows that the gains of selective Mixup largely come from the implicit resampling induced by pair selection, and its "selective sampling" ablation (training on the selected pairs without mixing them) is essentially the policy-matched endpoint reference proposed here; it also reports WGA on Waterbirds. Together with Idrissi et al. (2022), who show that simple group balancing is competitive, this leaves the main empirical message of the paper largely known. What remains new (coefficient-weighted endpoint losses, the marginal-matched reference, and the binary coverage identities) is not positioned against this prior work, and it is unclear whether any of it changes the earlier conclusions.

  3. The two endpoint references disagree substantially, which undermines the decomposition. Under the population assumptions of Eq. (5)–(6), the policy-matched and marginal-matched references optimize the same objective when E^pos=Epos\widehat{E}^{\text{pos}} = E^{\text{pos}}. Yet on CelebA only-sens they reach 87.44%87.44\% and 74.00%74.00\% WGA (Table 2), a gap of about 13 points that is as large as the policy contrasts being interpreted (e.g., the Mixup contrast of +12.24+12.24 in Table 1), and on Waterbirds sens08 the order reverses (70.27%70.27\% vs. 74.45%74.45\%). The paper attributes this to finite-run factors (position-specific marginals, pair coupling, coefficient weights, minibatch composition) but does not identify which one is responsible. Since Eq. (15) decomposes magnitudes, the "endpoint" term depends strongly on an implementation choice, and the population identity that motivates the references does not predict their empirical behavior.

  4. The empirical scope is far narrower than that of the method being analyzed. LISA, whose pair relations the paper adopts, was evaluated on nine benchmarks covering subpopulation shift (CMNIST, Waterbirds, CelebA, CivilComments) and domain shift (Camelyon17, FMoW, RxRx1, Amazon, MetaShift), and Teney et al. additionally used CivilComments and Wild-Time datasets. This paper evaluates only Waterbirds and CelebA among real benchmarks, together with synthetic data, Colored MNIST and biased CIFAR-10, and restricts both the analysis and the experiments to binary labels and a single binary attribute. It therefore does not test whether the conclusions hold with many domains or classes (e.g., FMoW, RxRx1), in text (CivilComments, Amazon), or under domain shift (Camelyon17, MetaShift), which is where selective Mixup was proposed to help. In addition, Waterbirds and CelebA use an ImageNet-pretrained ResNet-18 rather than the ResNet-50 used by LISA, so absolute WGA cannot be compared with published numbers.

Questions for authors
  1. What does this work establish beyond Teney et al. (2024)? Please compare the policy-matched reference with their selective-sampling ablation and state whether any conclusion differs.
  2. Which factor explains the gap between the two endpoint references on CelebA only-sens: the coefficient weights, the position-specific marginals, the pair coupling, or the two-pass batch-normalization updates? An ablation that changes one factor at a time would clarify which reference should be used in Eq. (15).
  3. Do the main contrasts hold on the other LISA benchmarks, in particular MetaShift, CivilComments and a multi-domain dataset such as FMoW or Camelyon17, and with ResNet-50 on Waterbirds and CelebA? How would DPE and the endpoint references be defined beyond a binary label and a binary attribute?
  4. Does the endpoint reference still match Mixup when it processes the same number of examples per update (BB instead of 2B2B)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.