- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes COMPASS for long-tailed adversarial training. It adjusts the per-sample perturbation budget based on clean-prediction reliability and class-level robustness gaps, and then retrains the final classifier on class-balanced data after training. The paper reports improved tail-class robustness across several datasets and architectures.
Critical strengths
None.
Critical weaknesses
W1. The paper reads more like a technical report than a research paper. The introduction claims that strong perturbations on clean-unstable samples harm long-tailed AT, but the paper never shows that existing methods actually fail for this reason. The only related evidence, Fig. 3(b), is measured within COMPASS after the method is introduced. The paper therefore describes what was done and the resulting numbers, but offers no clear claim or insight that a reader can take away beyond the specific combination of techniques.
W2. The method does not distinguish existing components from the proposed contribution. The prior-calibrated loss is logit adjustment as in BSL, the class-wise perturbation budget and TRADES weight follow CFA, and TCR is essentially cRT with an additional interpolation step. The method section presents all of these as parts of COMPASS without separating them from what is new. As a result, it is unclear what the authors actually propose and what the paper is meant to claim.
W3. It is unclear whether the reliability gate itself helps, although it is the core of the paper’s motivation. The introduction and abstract build the paper around the claim that strong perturbations should be applied only to clean-reliable samples, so the reliability gate is the component that most directly supports this claim. However, in Table 4, comparing “Base + RSR” with “Base + RSR w/o Rel.” shows that the gate mostly increases clean accuracy, while robust accuracy stays about the same. But the gate only makes the training attack budget smaller, and a smaller budget usually gives the same result. Fig. 3(b) also just shows that samples given a larger budget become more robust.
W4. Instead, most of the gain comes from TCR, which trades head robustness for tail robustness. In Table 4, the head-class accuracy implied by the all and tail results drops sharply after TCR, while overall robustness barely improves. The tail gains thus mainly reflect classifier rebalancing rather than more robust representations.
W5. The baselines may not share the same base loss. COMPASS includes a BSL-style loss in Eq. (1), but the other plug-in baselines appear to use standard CE. For example, PGD-AT + RLT closely matches AT + RLT in the RLT paper, not AT-BSL + RLT. If so, part of the gain may come from the BSL loss itself, and AT-BSL + RLT would be a more appropriate baseline.
Minor
- Tables 1–3 are hard to read, and the overall trend is not visible at a glance. Keeping only the TRADES results in the main text would help.
- Have the authors considered comparing with PAT (Zhang et al., ICML 2026), a recent long-tailed AT method?
- No statistical report is provided.
- In Eq. (5) and Eq. (11), should max be min?
Questions for authors
None
13 comments
Reply to Reviewer BHRb– Weakness 2
Weakness2. The method does not distinguish existing components from the proposed contribution. The prior-calibrated loss is logit adjustment as in BSL, the class-wise perturbation budget and TRADES weight follow CFA, and TCR is essentially cRT with an additional interpolation step. The method section presents all of these as parts of COMPASS without separating them from what is new. As a result, it is unclear what the authors actually propose and what the paper is meant to claim.
Response. Thank you for raising this concern. We respectfully clarify that COMPASS does not claim logit adjustment, class-wise feedback, or classifier retraining as new in isolation. Its contribution lies in two mechanisms that change how robust supervision is allocated during training and how classifier bias is corrected afterward.
RSR innovation and distinction from BSL/CFA. RSR addresses the within-class ambiguity left by BSL/CFA. It factorizes robust allocation into class-level demand and sample-level readiness: class-wise robust deficiency determines where additional supervision is required, while class-relative clean reliability determines through which samples that demand should be realized. Consequently, two examples from the same under-robust class can receive different effective budgets. The novelty is this conditional cross-granularity allocation, rather than class-wise adaptation or confidence estimation alone. The controlled intervention above supports its necessity: the full budget improves high-reliability PGD-20 accuracy by 4.90 points, but decreases both natural and robust accuracy for low-reliability samples.
TCR innovation and distinction from direct classifier retraining. Conventional decoupled retraining freezes the representation, fits a balanced classifier, and deploys it as a replacement for the original classifier. TCR instead treats classifier correction as a robustness-aware selection problem. It preserves the original adversarial classifier and the balanced classifier as two endpoints, interpolates both their weights and biases, and selects the interpolation coefficient solely on a training-derived split. The selection criterion jointly evaluates natural and robust performance over all and tail classes. Thus, the contribution is not interpolation as an arithmetic operation, but a train-only mechanism for determining how much prior correction can be introduced without fully discarding the adversarially learned decision structure.
Unified claim. COMPASS contributes reliability-conditioned robust allocation and robustness-aware partial classifier correction. We will revise the method section, contribution list, abstract, and related work to mark this boundary explicitly and consistently.
Reply to Reviewer BHRb– Weakness 3
Weakness 3. It is unclear whether the reliability gate itself helps, although it is the core of the paper’s motivation. The introduction and abstract build the paper around the claim that strong perturbations should be applied only to clean-reliable samples, so the reliability gate is the component that most directly supports this claim. However, in Table 4, comparing “Base + RSR” with “Base + RSR w/o Rel.” shows that the gate mostly increases clean accuracy, while robust accuracy stays about the same. But the gate only makes the training attack budget smaller, and a smaller budget usually gives the same result...
Response. Thank you for raising this point. The reliability gate is designed to prevent class-level robust feedback from assigning an enlarged perturbation budget indiscriminately to all samples of an under-robust class. Its contribution should therefore be assessed by whether it avoids the clean-accuracy cost of uniform class-wise strengthening while retaining its robust benefit.
Table 4 supports this intended role. Relative to RSR without reliability conditioning, the complete RSR improves Nat.(all) and Nat.(tail) by 5.02 and 5.87 points, respectively. Meanwhile, PGD(tail), C&W(tail), and AA(tail) improve by 0.45, 1.97, and 1.19 points. Overall robustness remains comparable: the changes in PGD(all), C&W(all), and AA(all) are -0.67, +0.69, and -0.14 points, respectively. Thus, the reliability gate substantially improves natural accuracy and tail robustness without materially reducing aggregate robustness.
We directly test the reviewer's alternative explanation by replacing the gate with fixed budgets from to . We also include two strength-matched controls: Class-mean matched assigns every sample of a class its mean gate value, while Reliability shuffled preserves the batch-level distribution of gate values but randomly permutes their sample assignments.
Reliability gate versus non-conditional budget controls on CIFAR10-LT. All entries are accuracies in percent.
| Allocation | Nat.(all) | PGD(all) | Nat.(tail) | PGD(tail) |
|---|---|---|---|---|
| Uniform 0.3 epsilon | 70.10 | 27.79 | 66.50 | 21.30 |
| Uniform 0.5 epsilon | 66.63 | 30.99 | 62.31 | 23.98 |
| Uniform 0.7 epsilon | 65.77 | 34.55 | 59.36 | 25.35 |
| Uniform 0.9 epsilon | 64.07 | 36.08 | 57.16 | 26.56 |
| Class-mean matched | 65.62 | 34.35 | 61.16 | 26.55 |
| Reliability shuffled | 66.82 | 35.13 | 60.70 | 26.18 |
| Reliability gate (ours) | 68.07 | 35.38 | 62.25 | 27.52 |
The fixed-budget sweep exhibits a clear clean--robust trade-off rather than reproducing RSR. Reducing the budget to 0.3 epsilon raises natural accuracy but lowers PGD(all/tail) to 27.79/21.30%; increasing it to 0.9 epsilon recovers PGD(all) but lowers Nat.(all/tail) to 64.07/57.16%. No fixed budget matches the gate's joint result. The two matched controls rule out more specific alternatives: Class-mean matched preserves the average budget of each class, while Reliability shuffled preserves the batch-level budget distribution. Both underperform the gate, indicating that the gain depends on assigning each budget according to the corresponding sample reliability, rather than on the average strength or variability of the training attack alone.
We separately test whether the Base+RSR robust gain can be explained solely by the increase in clean-correct predictions. On the same 10,000 CIFAR10-LT test examples, we compare Base with Base+RSR, using a separately generated 20-step PGD attack for each model. Let and denote clean-correct and robust-correct predictions, respectively.
Tail-sample state transitions from Base to Base+RSR.
| Base state | Base+RSR state | Count |
|---|---|---|
| Clean-incorrect | Clean-correct, PGD-incorrect | 820 |
| Clean-incorrect | Clean-correct, PGD-correct | 86 |
| Clean-correct, PGD-incorrect | Clean-correct, PGD-correct | 650 |
| Clean-correct, PGD-correct | Clean-correct, PGD-incorrect | 217 |
| Clean-correct, PGD-correct | Clean-incorrect | 6 |
Among the tail samples misclassified by Base, RSR makes 906 clean-correct: 820 remain adversarially incorrect, while only 86 become both clean- and robust-correct. Meanwhile, RSR converts 650 samples that were already clean-correct but adversarially incorrect into robust-correct predictions. Accounting for the 217 robust-correct samples that become adversarially incorrect and the 6 that become clean-incorrect, the total net increase in robust-correct tail predictions is
The net contribution from samples already clean-correct under Base is . Hence, most of the robust improvement comes from correcting adversarial vulnerability in existing clean decisions, rather than from the increase in clean-correct samples. We will revise the paper to describe the gate as reliability-conditioned allocation.
Reply to Reviewer BHRb– Weakness 1
Response to Reviewer BHRb
We thank the reviewer for the detailed and constructive assessment. The review identifies three issues that were not sufficiently separated in the submission: the empirical basis of the reliability hypothesis, the boundary between inherited techniques and our contributions, and the respective roles of RSR and TCR. We address these points below.
Weakness 1. The paper reads more like a technical report than a research paper. The introduction claims that strong perturbations on clean-unstable samples harm long-tailed AT, but the paper never shows that existing methods actually fail for this reason. The only related evidence, Fig. 3(b), is measured within COMPASS after the method is introduced. The paper therefore describes what was done and the resulting numbers, but offers no clear claim or insight that a reader can take away beyond the specific combination of techniques.
Response to the Weakness 1. We agree that Fig. 3(b) is correlational and, by itself, does not establish the failure mode used to motivate RSR. Our scientific claim is narrower than the original wording: class-level robust deficiency is not sufficient to determine the appropriate adversarial pressure for every sample in that class. The allocation problem has two conditions: where robustness is lacking and which samples can receive stronger supervision. RSR is designed around this distinction.
To test this premise directly, we conducted a controlled intervention on CIFAR10-LT with ResNet-18. The model (RLT+PCC) was retrained from random initialization, and all reliability statistics and group thresholds were derived exclusively from the training set. For a training sample (xi,yi) with logits zi, we compute the clean margin
mi=zi,yi−maxk=yizi,k
and its class-relative reliability
\qquad \tau_c=\frac{1}{|\mathcal D_c|}\sum_{i\in\mathcal D_c}m_i,$$ where $\tau_c$ is the mean training margin of class $c$ and $T_r=0.5$. Tail classes are the least frequent 80% of classes. Quintiles of their training reliabilities produce the fixed boundaries 0.157, 0.374, 0.618, and 0.846. We define the low-reliability group as the bottom 40% (reliability at most 0.374) and the high-reliability group as the top 40% (reliability at least 0.618); the middle 20% is excluded to obtain two clearly separated groups. Starting from the same final-epoch model, five paired runs compare a weak perturbation budget of $0.3\epsilon$ with the full budget $\epsilon$, while holding the data, objective, optimizer, and three-epoch adaptation schedule fixed. **(a) Reliability and pre-intervention vulnerability.** Q1--Q5 denote increasing reliability quintiles. Samples is the number of test examples, Mean trust is the group mean reliability, and Attack success is 100 minus PGD-20 accuracy. All accuracy values are percentages. | Bin | Samples | Mean trust | Nat. | PGD-20 | Attack success | |:---:|--------:|-----------:|-----:|-------:|---------------:| | Q1 | 3,122 | 0.042 | 12.46 | 0.10 | 99.90 | | Q2 | 1,378 | 0.261 | 77.14 | 1.09 | 98.91 | | Q3 | 1,245 | 0.497 | 100.00 | 18.31 | 81.69 | | Q4 | 1,110 | 0.735 | 100.00 | 67.48 | 32.52 | | Q5 | 1,145 | 0.937 | 100.00 | 93.45 | 6.55 | **(b) Effect of increasing the perturbation budget.** Weak and Strong use $0.3\epsilon$ and $\epsilon$, respectively. Results are mean +/- standard deviation over five paired runs. Delta is Strong minus Weak in percentage points, and $p$ is from a two-sided paired $t$-test. | Group | Samples | Metric | Weak | Strong | Delta | p | |:------|--------:|:-------|-----:|-------:|------:|---:| | Low | 4,500 | Nat. | 27.52 +/- 0.54 | 26.63 +/- 0.58 | -0.89 | 0.042 | | Low | 4,500 | PGD-20 | 0.61 +/- 0.11 | 0.50 +/- 0.09 | -0.11 | 0.047 | | High | 2,255 | Nat. | 97.90 +/- 0.55 | 99.22 +/- 0.26 | +1.32 | 0.002 | | High | 2,255 | PGD-20 | 63.16 +/- 1.66 | 68.05 +/- 0.85 | +4.90 | <0.001 | Panel (a) verifies that the training-derived reliability is meaningfully related to adversarial vulnerability: PGD-20 accuracy increases monotonically from 0.10% in Q1 to 93.45% in Q5, while attack success decreases from 99.90% to 6.55%. Panel (b) isolates the effect of perturbation strength. Increasing the budget reduces low-reliability Nat. and PGD-20 accuracy by 0.89 and 0.11 points, respectively, but improves the corresponding high-reliability metrics by 1.32 and 4.90 points. Thus, the same increase in adversarial pressure has opposite effects depending on clean reliability. This result provides the missing empirical premise for reliability-conditioned allocation. We will reorganize the introduction around this finding, define "clean instability" precisely, and present Fig. 3(b) only as a diagnostic visualization rather than causal evidence.