acceptodds
Under review as a conference paper at ICLR 2027

Imbalanced Graph Classification via Graph Similarity Learning

Abstract

Graph classification is a fundamental task prevalent in fields such as molecular property prediction, bioinformatics, and social network analysis. However, real-world graph datasets suffer from severe graph size imbalance, which under limited label supervision induces size-driven similarity bias and semantic misalignment, resulting in significant performance gaps across size groups. In response, we propose a unified graph similarity-based learning framework **SimGraph** integrating balanced pairwise graph similarity (BPGS) learning and size-adaptive instance normalization (SAIN): BPGS formulates similarity estimation as pairwise binary classification with an asymmetric scaling scheme and a learnable decision boundary, while SAIN suppresses size-related variations in graph representations. We characterize the design-motivated properties of SimGraph, including its false-negative-free pairwise construction and bounded gradients, which provide principled support for stable optimization under size-imbalanced graph distributions. Extensive experiments on binary and multi-class benchmarks spanning molecular, biological, social, and heterogeneous graphs demonstrate that SimGraph achieves state-of-the-art performance on imbalanced graph classification with limited labels, with an average accuracy improvement of **4.2%** in size-imbalanced settings and up to **13%** on individual datasets.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Imbalanced Classification, Graph Neural Networks

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KtMMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes SimGraph for imbalanced graph classification. The framework combines Balanced Pairwise Graph Similarity (BPGS) learning and Size-Adaptive Instance Normalization (SAIN). BPGS treats graph similarity estimation as a pairwise binary classification problem, using asymmetric scaling and a learnable decision boundary. SAIN aims to reduce the effect of graph size on graph representations. The authors conduct extensive experiments on multiple datasets and compare their method against several baselines. The results show that SimGraph achieves state-of-the-art performance.

Critical strengths

The experiments are comprehensive, covering multiple datasets, including large-scale datasets, and different models. The authors also provide the source code, which is helpful for reproducibility.

Critical weaknesses
  • The motivation and justification for the proposed method are not sufficiently clear. The paper identifies size-induced similarity bias as a key problem, but I did not find convincing empirical evidence that directly demonstrates this bias. Figure 1 only provides an illustration, rather than evidence from real datasets. Moreover, most of the proposed components seem incremental, with relatively minor modifications to existing techniques. The paper does not clearly explain how these design choices specifically help reduce size-induced similarity bias. For example, the BPGS loss is essentially a standard BCE loss with an additional scaling factor, while the memory bank is already a common technique in contrastive learning. For SAIN, I am not convinced that the proposed operation makes graph embeddings size-invariant. Since the affine transformation uses graph size information, could it instead introduce more size information into the embeddings? Although the ablation studies show the contributions of different components to the final performance, they do not directly demonstrate how individual design choices in these components reduce similarity bias. I would like to see stronger empirical or theoretical justification for these design choices.

  • The paper is difficult to follow, and the presentation needs improvement. The font size in some tables is too small to read comfortably. Similarly, the figure captions are very small, and the spacing between the figures and their captions is too narrow. These formatting choices appear to violate the conference's formatting requirements. In addition, the method section is somewhat lengthy and tedious to follow. I suggest shortening this section to make room for larger and more readable figures and tables.

Questions for authors
  1. Can the authors provide empirical evidence demonstrating the existence of size-induced similarity bias on real datasets? Figure 1 is only an illustration, and it would be helpful to see whether graph size actually affects pairwise similarity in the way described in the paper.

  2. How does the asymmetric scaling factor in BPGS specifically address size-induced similarity bias? What makes this design more suitable than the standard BCE loss? Have you tried to replace the BPGS loss with standard BCE loss?

  3. Since memory banks are commonly used in contrastive learning, what is their specific role in addressing size-induced similarity bias in this framework?

  4. Why does SAIN produce size-invariant graph embeddings when the affine transformation explicitly depends on graph size? Could this operation instead introduce additional size information into the representations?

  5. Can the authors provide more direct empirical or theoretical evidence showing that individual design choices in the proposed components actually reduce similarity bias, beyond the performance improvements reported in the ablation studies?

  6. Can the authors evaluate SimGraph with smaller labeled training sets, such as 5%, 3%, and 1%, and compare its performance against existing semi-supervised learning methods under these settings?

3 comments

We thank Reviewer KtMM for the constructive suggestions regarding empirical motivation, component justification, ultra-low-label benchmarks, and presentation.

W1 & Q1: Empirical Evidence of Size-Induced Bias

Evaluating held-out pairs stratified by class pairs, standard CE exhibits lower similarity and higher FNR on cross-size vs. near-size same-class pairs (62.36% vs. 47.06% on D&D; 61.04% vs. 35.45% on RDT-B), and higher FPR on near-size vs. cross-size different-class pairs (49.96% vs. 30.87% on D&D; 49.02% vs. 34.60% on RDT-B). We will integrate these real-dataset statistics into Figure 1.

W1 & Q2: Asymmetric Scaling Factor vs. Standard BCE

Setting r=1r=1 in BPGS exactly recovers balanced pairwise BCE. Setting r≠1r \neq 1 asymmetrically bounds positive (1/r1/r) and negative (rr) gradients, preventing overfitting on scarce positives while sharpening hard-negative separation (Figure 10). Under aligned tuning budgets (Table 26), CE + BPGS (w/o SAIN) outperforms CE by +9.0–12.7 pp and SupCon/ProGCL by +4.4–7.4 pp across four datasets.

W1 & Q3: Specific Role of the Memory Bank

Under sparse labels, mini-batches rarely cover diverse cross-size positive pairs for minority classes. Using an identical EMA encoder, our memory bank expands distinct cross-size (Δlog⁡n≥log⁡2\Delta \log n \ge \log 2) positive pairs per epoch from 788.8 to 1,969.1 (D&D) and from 1,746.8 to 6,703.8 (RDT-B), improving BACC from 64.88% (in-batch) to 65.43% (D&D) and from 70.31% to 73.52% (RDT-B).

W1 & Q4: SAIN and Size Information

We agree that conditioning γ(n),β(n)\gamma(n), \beta(n) on size performs adaptive size modulation rather than unconditional size erasure: InstanceNorm removes uncontrolled size-coupled moment shifts, while the affine transform retains task-predictive cues (Appendix I.1, Table 32). Boundary tests (Table 36) confirm that SAIN prevents size-shortcut overfitting under size_sep (+8.57 pp on IMDB-B, +13.66 pp on RDT-B) while preserving predictive signals under size_mix (+4.09 pp on D&D). We will revise "size-invariant" to "size-modulated" in the main text.

W1 & Q5: Direct Component-Level Evidence

Beyond ablation accuracy, our held-out pair FNR/FPR statistics, cross-size bank pair counts, norm–size correlation drops (Table 32), three-panel angular/radial decompositions (Figures 6–8), and subgroup improvements (lifting RDT-B worst-cell accuracy from 48.13% to 52.36%) directly demonstrate how each module mitigates size-induced bias.

Q6: Evaluation with 5%, 3%, and 1% Labeled Training Sets

We evaluated SimGraph on D&D across 10% (118 graphs), 5% (59), 3% (35), and 1% (12) labeled budgets (10 seeds per split), comparing against CE, SupCon, and a semi-supervised Mean Teacher (MT) GNN consistency baseline (Test BACC %):

Label BudgetSubsetSimGraph (Ours)CESupConMean Teacher
10%066.94 ± 3.0065.67 ± 1.5365.02 ± 1.8663.93 ± 1.82
5%0 / 163.01 / 64.9462.86 / 61.9361.93 / 62.7462.05 / 62.10
3%0 / 162.09 / 64.2661.69 / 63.4161.96 / 64.2558.49 / 60.90
1%0 / 161.90 / 58.9861.83 / 56.7959.49 / 58.7560.46 / 58.22

SimGraph consistently achieves the best performance across all ultra-low label budgets, outperforming CE by +1.58 / +0.62 / +1.13 pp, SupCon by +1.64 / +0.07 / +1.33 pp, and Mean Teacher by +1.90 / +3.48 / +1.10 pp on average at 5% / 3% / 1%, respectively.

W2: Readability and Formatting

We will streamline Section 3 to make it concise, enlarge font sizes across all tables and figure captions, adjust spacing to strictly adhere to formatting guidelines, and fix all citation/typographical errors.

Thank you for the detailed response and additional experimental results. While these clarifications are helpful, I still have concerns about whether the proposed components directly mitigate size-induced similarity bias, particularly the specific contributions of asymmetric scaling in BPGS and size-adaptive normalization in SAIN.

Furthermore, the manuscript has not yet been revised to incorporate the additional evidence and clarifications provided in the rebuttal.

Therefore, I will retain my original rating.

Dear Reviewer ktMM,

Thank you for identifying the two remaining issues: the connection between our components and size-induced similarity bias, and the absence of these clarifications from the manuscript. We agree that classification improvements alone do not establish debiasing. We have now revised the manuscript, with changes highlighted in blue, to distinguish the observed bias, the optimization rationale, and the supporting empirical evidence.

1. The evidence is now in the main text. Table 1 in Section 1 (p. 2) reports held-out pairwise measurements, rather than only an illustration. With the CE encoder, cross-size versus near-size same-class FNR is 62.36% versus 47.06% on D&D and 61.04% versus 35.45% on RDT-B. For different-class pairs, near-size versus cross-size FPR is 49.96% versus 30.87% and 49.02% versus 34.60%, respectively. These establish the two observed error modes. Section 3 now explains the component mechanisms and their limits, while Section 4.3 (pp. 9–10) includes the auxiliary-loss comparison (Table 6), SAIN boundary tests (Table 7), and smaller-label results (Table 8). We also shortened the theoretical discussion and enlarged the large-scale result tables.

2. Asymmetric scaling: its exact role, rather than a size-specific guarantee. Given identical scores, pair masks, and balance weights, setting r=1r=1 exactly recovers balanced pairwise BCE. For z=s+bz=s+b, the weighted score gradients are

g_-(z)=(1-\alpha)r\sigma(rz).$$ For $r>1$, a different-class pair with $z\ge0$ receives stronger repulsion than under $r=1$; this includes near-size false positives. The positive gradient is bounded by $\alpha/r$, and the positive/negative curvatures at $z=0$ are $\alpha/(4r^2)$ and $(1-\alpha)r^2/4$. Thus, scaling provides smoother positive attraction and sharper negative separation near the boundary. However, **$r$ does not itself identify size-confounded pairs**, and these score-level properties do not guarantee reduced overfitting or size independence. Cross-size constraints come from labeled pair coverage, expanded by the memory bank. Table 6 supports the combined pairwise design: without SAIN, CE+BPGS exceeds SupCon by 3.79–7.41 percentage points and ProGCL by 2.11–6.62 points across four datasets. We explicitly distinguish this comparison from an isolated test of $r$. The existing $r$-sensitivity figure measures class recall, not size-stratified pair errors; it supports the optimization trade-off but is not presented as direct evidence that $r$ alone removes size bias. **3. SAIN: size-conditioned calibration, not unconditional size erasure.** Section 3.2 now uses **“size-modulated”** rather than “size-invariant.” Normalization standardizes per-embedding moments; the affine transform can retain size information. BPGS supplies cross-size pair constraints on that transform, but cannot guarantee that all size shortcuts disappear. We therefore report complementary diagnostics. On RDT-B Low, norm–size correlation decreases from +0.777 to +0.186; on RDT-B High, it decreases only from +0.899 to +0.751 (Appendix I.1, Table 35). In a separate High-imbalance visualization, norms compress from roughly 0–100 to 10.9–11.6 and directional class separation improves (Appendix I.3, Figure 7). Table 7 also reports BACC gains of 8.57 points on IMDB-B and 13.66 points on RDT-B under size-biased class selection, alongside the negative results under size-OOD splits. These support useful calibration in the evaluated settings, not universal invariance. Accordingly, we recommend validation-based selection and caution when size is label-defining. We appreciate the distinction you raised between a plausible mechanism and evidence that isolates its effect. The revision now makes that distinction explicit and places the relevant results in the manuscript for direct inspection.
Reviewer_dNCcOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses primarily on the problem of graph-scale imbalanced classification under sparse labelling. To address the potential bias in similarity estimation arising from this issue, we propose SimGraph. Specifically, SimGraph integrates Balanced Proportional Graph Similarity (BPGS) and Size-Adaptive Instance Normalisation (SAIN) into a unified framework. This framework utilises label-supervised asymmetric scaling and learnable decision boundaries to shape the relationship between representations and mitigate the impact of graph size on statistical features. Furthermore, the framework introduces virtual graph augmentation and adaptive loss weighting to address training challenges under sparse data conditions. Extensive experiments were conducted on a wide range of datasets, and the results confirm the framework’s effectiveness.

Critical strengths

(1)This paper provides a clear and effective motivation. The paper points out that differences in graph size may interfere with semantic similarity in the representation space, leading to underestimated similarity between graphs of the same class but different sizes, whilst graphs of different classes but similar sizes are overestimated in terms of similarity. This issue is of particular concern under conditions of sparse labelling and size imbalance. Taking ‘size-induced similarity bias’ as a starting point, the paper provides an intuitive and coherent motivation for the design of pairwise similarity learning and size-conditioned normalisation.

(2)The paper conducted evaluations across multiple graph classification datasets, varying degrees of class and size imbalance, and two backbone networks—GIN and GCN—further extending the scope to multi-classification, heterogeneous graph and large-scale molecular graph tasks. The reported results demonstrate that the method achieves performance improvements across a variety of settings; component ablation, similarity function comparisons and hyperparameter analysis also provide supplementary evidence, aiding in the understanding of the roles of individual modules and the method’s scope of application.

Critical weaknesses

(1)In the paper, the de-biasing mechanism claimed in the paper has not been fully substantiated. SAIN explicitly uses graph size as a condition for generating representations, whilst BPGS is trained under label supervision from the same training distribution; consequently, the improvement in classification performance does not actually demonstrate whether semantic de-biasing has genuinely been achieved, or whether it merely makes more effective use of the correlation between size and labels. An improvement in performance does not necessarily mean that de-biasing has been effective; it may simply be that the model has become better at ‘exploiting’ the statistical correlations between size and labels.

(2)Typos error: (1) Duplicate terms: On line 394, ‘CSSL cssl’ contains an error: there are two instances of ‘cssl’; on line 398, ‘We employ GradNorm gradnorm’ contains two instances of ‘gradnorm’; on line 352, ‘through mixup mixup between …’ contains two instances of ‘mixup’; on line 361, ‘graphacl’ is not cited; on line 2377, ‘TablesTables 1 and 2’ contains two instances of ‘table’; on line 302, ‘style transfer adain’ is not cited. There are many other errors; please check carefully. (2) Inconsistent tables: For the same metrics—GIN, high size imbalance and complete SimGraph—different tables provide differing standard deviations. For example, on the ptrmr dataset, Table 1 reports 1.57, Table 22 reports 1.27, and Table 26 reports 1.28. The same issue occurs in the D&D, IMDB-B and RDT-B datasets.

Questions for authors

(1)I’m actually a little unclear on this point: in Equation (9), how exactly is the Gaussian covariance estimated and regularised? Since in VGA there are only M virtual prototypes per class, and M=20 is typically far smaller than the embedding dimension d, in which case the full empirical covariance is almost certainly singular—and may not even be invertible directly. So, in the actual implementation, is a diagonal approximation, diagonal loading, shrinkage, shared covariance, low-rank approximation or some other method used? I hope the author can clarify the specific approach and the corresponding hyperparameters; otherwise, it is difficult for readers to judge how this negative KL term is actually calculated and whether it is stable.

(2)Furthermore, there is another issue that concerns me even more: the negative KL term itself has no lower bound; minimising it essentially drives the inter-class KL towards larger values. Might this therefore circumvent genuine inter-class semantic separation, instead relying on infinitely increasing the distance between class means, or causing the covariance to degenerate or explode in order to minimise the loss? If this is the case, are there any explicit constraints or regularisation terms to prevent such degenerate solutions—for example, constraints on the mean norm, the condition number of the covariance matrix, the trace or the determinant? Or do terms such as CE, BPGS and orthogonal loss already suffice to keep it in check? This point directly relates to whether VGA is actually generating meaningful virtual samples or merely exploiting loopholes in the objective function. I hope the authors can provide further clarification, ideally accompanied by some ablation or diagnostic experiments.

1 comment

We thank Reviewer dNCc for the rigorous insights on semantic debiasing, covariance conditioning, and table consistency.

W1: Predictive Gains vs. Semantic Debiasing

We agree that accuracy improvements alone do not prove semantic debiasing. We substantiate debiasing through three direct diagnostics: (1) on held-out pairs, standard CE suffers severe size-induced bias (cross-size vs. near-size FNR of 62.36% vs. 47.06% on D&D; 61.04% vs. 35.45% on RDT-B); (2) SAIN improves both overall BACC (68.18% vs. 65.23% on D&D; 72.23% vs. 70.54% on RDT-B) and worst class×size subgroup accuracy on RDT-B (52.36% vs. 48.13%) over standard normalization; and (3) under size_sep (Appendix I.4, Table 36), where training size strongly correlates with labels as a shortcut, SAIN yields +8.57 pp (IMDB-B) and +13.66 pp (RDT-B) BACC by preventing size-cue overfitting, while SimGraph exceeds size-only predictors by 14.6–17.0 pp (Table 28). We will frame SAIN precisely as task-guided size calibration rather than unconditional invariance.

W2: Typography and Table Inconsistencies

We apologize for the typographical errors, missing citations, and standard deviation transcription discrepancies across Tables 1, 22, and 26. We have audited all logs and will unify all table entries and citations in the revision.

Q1: Covariance Estimation and Regularization in Eq. (9)

Our implementation retains full empirical covariance with diagonal loading: Σ^c=1M−1∑m=1M(vcm−μc)(vcm−μc)⊤+ϵI\widehat{\Sigma}_{c} = \frac{1}{M-1}\sum_{m=1}^{M}(v_{cm}-\mu_{c})(v_{cm}-\mu_{c})^{\top} + \epsilon I (ϵ=10−3\epsilon=10^{-3}). Since rank(Σcemp)≤M−1<d\text{rank}(\Sigma_{c}^{\text{emp}}) \le M-1 < d (M=20,d=32M=20, d=32), diagonal loading ensures strict positive definiteness. We compute the analytic Gaussian KL via float64 Cholesky solves and log-determinants rather than explicit matrix inversion (verified within 4.1×10−84.1\times 10^{-8} relative discrepancy). Sensitivity tests on D&D confirm stable BACC across ϵ∈{10−4,10−3,10−2}\epsilon \in \{10^{-4}, 10^{-3}, 10^{-2}\} and a diagonal-covariance variant (65.88%–66.73%). We will add these specifications to Section 3.3.1.

Q2: Unbounded Negative KL and Bounded Stabilization

You raise an important theoretical point: diagonal loading ensures invertibility, and Lorth\mathcal{L}_{\text{orth}} acts on L2L_{2}-normalized prototypes, meaning neither places a strict analytical lower bound on Ldiv\mathcal{L}_{\text{div}}. While our training trajectories remained numerically finite without prototype-norm runaway (post-warmup norm ratio ≈1.0005\approx 1.0005), unconstrained optimization can induce ill-conditioned covariances. Following your suggestion, we introduced an explicit bounded-separation regularization applying prototype norm projection and normalized covariance inputs. This strictly bounds the objective, restricts the maximum covariance condition number below 48.2, and improves D&D BACC to 67.33 ± 1.32%, ensuring well-conditioned, semantically valid virtual prototypes. We will incorporate this bounded formulation into Eq. (9).

Reviewer_W3LpOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work introduces size induced similarity bias in imbalanced graph classification, which has seen increased attention due to the number of real world applications. Specifically, they propose SimGraph, which combines balanced pairwise similarity learning, size-adaptive normalization, and graph augmentations to learn better graph representations under imbalance. Experiments on multiple datasets show generally strong improvements over many existing representative baselines.

Critical strengths

(S1) The paper addresses an interesting problem of size-induced similarity bias in imbalanced graph classification, which is not well explored in existing methods that have focused on imbalance learning on graphs.

(S2) The proposed loss is seen as novel, which combines pairwise supervision with asymmetric gradient scaling and a learnable decision boundary to better handle class imbalance.

(S3) The size-adaptive instance normalization module provides a reasonable way to account for graph size differences during representation learning, but additional experimentation could potentially showcase the benefits a bit more.

(S4) Experiments across different datasets, imbalance settings, and GNN backbones are generally consistent in their improvements.

Critical weaknesses

(W1) For the overall framework, in some sense it can be seen as several components building on existing techniques including pairwise learning, normalization, and graph augmentation. The theoretical results also seem relatively straightforward, but I'm still positive about the contributions and results.

(W2) While ablations are provided, it's not fully clear how much improvement comes from the specific BPGS design rather than using standard supervised contrastive learning (although baselines are compared).

(W3) SAIN may remove useful size information when graph size is predictive of the label. The additional experiments also show settings where it hurts performance, raising questions about its general applicability.

Questions for authors

Q1. How would BPGS compare with standard balanced pairwise BCE or supervised contrastive learning under the same framework?

Q2. Can the authors directly evaluate learned similarities between graphs of different sizes to better support the motivation?

Q3. Is there a practical way to determine when SAIN should be used (given that graph size could contain useful class information)?

1 comment

We sincerely thank Reviewer W3Lp for the positive assessment and insightful feedback on component attribution and SAIN's applicability.

W1: Relation to Existing Components and Theoretical Contribution

While pairwise learning, normalization, memory banks, and mixup are established primitives, our core contribution is their synergistic adaptation to resolve size-induced similarity bias under sparse labels: SAIN calibrates size-coupled representation moments, BPGS enforces balanced cross-size pairwise geometry, and VGA alleviates minority-class scarcity. We will clarify this division of roles in Section 3 and frame our theoretical analyses strictly as conditional design properties rather than unconditional guarantees of size independence.

W2 & Q1: Comparison with Pairwise BCE and Supervised Contrastive Learning

Mathematically, given identical similarity scores zz, masks, and balance weights α\alpha, setting r=1r=1 in BPGS exactly recovers standard balanced pairwise BCE. Introducing r≠1r \neq 1 yields asymmetric gradients (−αrσ(−z/r)-\frac{\alpha}{r}\sigma(-z/r) for positives; (1−α)rσ(rz)(1-\alpha)r\sigma(rz) for negatives), smoothing scarce positive attraction while sharpening hard-negative repulsion without softmax denominator competition. Under identical GIN backbones, splits, and aligned tuning budgets (Appendix G.3, Table 26), BPGS substantially outperforms label-aware contrastive baselines under high size imbalance (Accuracy %):

MethodPTC-MRD&DIMDB-BRDT-B
CE only (GIN)48.41 ± 1.2560.68 ± 1.4562.60 ± 0.8967.41 ± 1.42
CE + SupCon52.46 ± 1.2465.94 ± 1.2563.53 ± 0.8469.14 ± 1.31
CE + ProGCL53.02 ± 1.4266.73 ± 1.6365.21 ± 0.9271.26 ± 1.26
CE + BPGS (w/o SAIN)57.42 ± 1.1973.35 ± 1.3667.32 ± 0.7675.03 ± 1.23
Full SimGraph60.51 ± 1.2877.32 ± 1.1769.00 ± 0.9577.94 ± 1.06

Even without SAIN, CE + BPGS surpasses SupCon and ProGCL by 4.4–7.4 pp, confirming that per-pair asymmetric BCE with memory-bank pairing drives the primary gains. We will highlight Table 26 in the main text.

Q2: Real-Data Similarity Evidence

Evaluating held-out graph pairs stratified by unordered class pairs (using validation-selected thresholds), the CE baseline exhibits a positive cosine similarity bias (near-size minus cross-size) of +0.0531 (D&D) and +0.0922 (RDT-B) for same-class pairs, and +0.0793 and +0.1098 for different-class pairs. Consequently, cross-size vs. near-size false-negative rates (FNR) reach 62.36% vs. 47.06% (D&D) and 61.04% vs. 35.45% (RDT-B), while near-size vs. cross-size false-positive rates (FPR) reach 49.96% vs. 30.87% and 49.02% vs. 34.60%. We will add these real-data measurements alongside Figure 1.

W3 & Q3: Useful Size Information and When to Use SAIN

Because γ(n)\gamma(n) and β(n)\beta(n) depend on size, SAIN acts as an adaptive size-modulation layer rather than an unconditional size-erasure operator: InstanceNorm strips uncontrolled size-dependent moment shifts, while the affine transform re-injects predictive size signals via task gradients (preserving controlled norm–size correlations, Table 32). Under BPGS, SAIN improves BACC over standard normalization on both D&D (68.18% vs. 65.23%) and RDT-B (72.23% vs. 70.54%), while lifting RDT-B worst class×size subgroup accuracy from 48.13% to 52.36%. As characterized in our boundary analysis (Appendix I.4, Tables 36–37), SAIN consistently helps under balanced (size_mix, up to +4.09 pp) and spurious-shortcut (size_sep, up to +13.66 pp) regimes, and only hurts when size is simultaneously OOD-shifted and genuinely class-determinative (size_ood on D&D). Practically, we recommend computing our label-free shift metric Δ^OD=KLsym(Psizetrain∥Psizetest)\hat{\Delta}_{\text{OD}} = \text{KL}_{\text{sym}}(P_{\text{size}}^{\text{train}} \parallel P_{\text{size}}^{\text{test}}) alongside subgroup validation metrics: SAIN is recommended when Δ^OD>2.0\hat{\Delta}_{\text{OD}} > 2.0 unless domain priors indicate size directly defines labels. We will move this guideline to Section 3.2 and refine "size-invariant" to "size-modulated."

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.