acceptodds
Under review as a conference paper at ICLR 2027

ReAL: Robustness-Enhanced Active Learning

Abstract

Active learning saves labels by selecting a small part of an unlabeled pool. For adversarial training, a useful batch should represent both the original learning signals and how they change under input attacks. We propose ReAL, which describes every candidate by its clean last-layer gradient and its attack-induced gradient change. Repeated -means++ seeding selects representatives of these signals, and cluster-size weights preserve the number of candidates each representative replaces. We connect the same coverage potential used for selection to approximation of the pool's robust objective. Under an explicit condition linking representation distance to true-label robust loss, its square root bounds the weighted objective's error and its minimizer's excess pool risk. This realized-batch bound holds for every selection; independent restarts additionally provide a high-probability comparison with the best achievable

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Active learning, adversarial robustness, adversarial training, batch acquisition, gradient-based acquisition, coreset approximation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_R5xNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies active learning for adversarial training. It proposes ReAL, which represents each unlabeled example using its clean last-layer gradient together with the gradient change induced by an adversarial perturbation. ReAL applies multi-start k-means++ to select a representative batch and assigns cluster-size weights to the queried examples during adversarial training. The paper also derives a conditional connection between embedding-space coverage and approximation of the robust pool objective. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show improved endpoint robust accuracy in several settings, but the gains are accompanied by inconsistent clean accuracy and substantial instability in some experiments.

Critical strengths
  • The paper addresses an important problem of improving adversarial robustness under a limited annotation budget, where both labeling and adversarial training are costly.

  • The proposed representation has a plausible robustness-related intuition, and the paper provides a sound conditional analysis connecting embedding coverage to robust-objective approximation.

  • The paper is generally clear and unusually transparent about its experimental protocol and limitations. It reports complete acquisition trajectories, multiple architectures and datasets, computational costs, seed variability, attack settings, and several negative or unstable results.

Critical weaknesses
  • The theoretical guarantee relies on a strong assumption that is neither derived from the proposed representation nor empirically validated. Specifically, Assumption 1 requires distances between embeddings constructed from current model pseudo-label gradients to uniformly control differences in true-label robust loss. This link is especially questionable when pseudo-labels are inaccurate, since such errors can propagate through acquisition-time attacks, gradient embeddings, cluster assignments, and cluster weights. The paper does not directly test the assumed representation–loss relationship. It also neither reports pseudo-label accuracy nor examines how pseudo-label quality affects acquisition behavior and downstream clean and robust performance.

  • The paper lacks the component ablations needed to identify the sources of ReAL's gains. It does not compare clean-gradient, attacked-gradient, direct-concatenation, and proposed secant embeddings, nor does it evaluate weighted versus unweighted training or single-start versus multi-start selection. Because ReAL changes both acquisition and training relative to the baselines, the current results cannot establish whether the improvements arise from the secant representation, multi-start selection, cluster weighting, or their combination.

  • The practical gains are limited and unstable in several settings. Although ReAL achieves stronger mean endpoint robust accuracy, it also yields substantially lower clean accuracy and high variance on CIFAR-100 and Tiny-ImageNet. On CIFAR-100 with ResNet-18, clean accuracy nearly collapses after the first acquisition round, while the PGD AUC improvement over Random is small relative to the observed variability. The results indicate an endpoint robustness trade-off rather than consistently improved label efficiency throughout the acquisition trajectory.

Questions for authors
  • Can the authors empirically validate Assumption 1 and report how pseudo-label accuracy affects acquisition and downstream performance?

  • Can the authors provide ablation studies comparing the proposed embedding, cluster weighting, and multi-start selection with their simpler alternatives?

  • How do the authors explain the early clean-accuracy collapse and high variance, and do additional results support stable gains across the acquisition trajectory?

2 comments
AuthorsOpenReview ↗

Assumptions, Pseudo-labels, Ablations, and Acquisition Stability Copy title

Thank you very much for your questions and constructive comments. Any references in our responses refer to the revised version of the paper, not the original submission.

1. Assumptions and pseudo-label reliability

The representation alone does not guarantee true-label regularity. Original Assumption 1 required robust-loss Lipschitz control for every pool pair and every parameter in a region. Revised Assumption 1 concerns response differences Δtr\Delta_t r at the actual two checkpoints on at least 1−α1-\alpha of assigned pairs, with bounded checkpoint losses (Sec. 5.2). Theorem 1 gives LtΨt(Bt)+2αCtL_t\sqrt{\Psi_t(B_t)}+2\alpha C_t transfer error, with Eq. (22) including old data. The proof separates regular and exceptional pairs in the cluster replacement identity, where no exact minimization is needed. Here α\alpha is response mismatch, not pseudo-label error. Both Secant blocks use one fixed pseudo-label. Sec. 5.1 preserves computed gradients even when it is wrong, without certifying true-label learning. Queried points use their acquired labels.

Appendix A.9 now examines 54 CIFAR-10/100 ResNet-18 transitions (3 seeds, 9 updates), with 2,048 uniform pre-query pool points and 500 representative loss lookups per case. True-label CE uses PGD-20$\times$1 (radius 8/2558/255, step 2/2552/255), maximizing over clean/start/all iterates. Geometry is original in 3 cases; 51 reconstruct pseudo-label PGD-5 geometry, fixing original assignments/masses. New95 is the 95th percentile of response discrepancy divided by RMS-normalized distance. It is below the analogous Old95 loss diagnostic in 27/27 and 21/27 cases, with median reductions of 28.0%/19.0%. Old95 is diagnostic, while OldMax is the constant required on sampled PGD-proxy pairs. These do not certify exact robust-loss regularity.

First-query CIFAR-100 pseudo-label accuracies are 10.21/10.94/10.25%. We record accuracy by round, correct/incorrect response groups, and checkpoint performance. Group differences are mixed. Co-occurring low accuracy and collapse do not identify causality. Direct PGD-CE transfer estimates span 0.299–2.919/0.376–4.897 CE units (CIFAR-10/100). Paired sample proxy/paired-error median ratios are 4.52/11.29, so this is not a tight certified bound. The fixed-query weighting intervention below isolates a separate training effect; pseudo-label quality was not causally manipulated.

2. Representation ablation

Secant preserves the clean gradient and exposes its attack-induced change, distinguishing candidates with similar clean signals by their adversarial response. It changes the metric despite retaining Concat's information (Sec. 4.2). Clean, Attack, Concat, and Secant share the paired CIFAR-10/ResNet-18 protocol. Only the embedding changes (Sec. 6.1, Table 1). Secant achieves the highest mean robust AUC, 19.96±0.8419.96\pm0.84, and the smallest SD of that metric among embeddings. Clean's higher robust endpoint motivates reporting both metrics.

3. Weighting and multi-start selection

Embedding defines geometry, seeding finds representatives, and weighting preserves pool mass. Equal weights distort unequal cluster proportions even at zero coverage error (Sec. 4.4). The new one-factor comparisons address the original confounding concern. Cluster improves robust endpoint and AUC over Unit on CIFAR-10, while the restart study tests 1/2/4/8/16 independently of weighting. Restarts reuse attacks/embeddings. With eight restarts, robust AUC is near its best mean with the smallest observed SD. This balances coverage and cost without assuming monotonic accuracy gains.

4. Early collapse and variability

In the CIFAR-100 first-query intervention, queries and training randomness are identical. Changing only weights reproduces deterioration under Cluster (Sec. 6.2; Appendix A.8). This isolates a weighting effect without changing acquired samples. One representative receives 22–24% of total normalized training weight, showing that fixed total mass can still produce highly concentrated supervision. Later acquisition paths diverge, so this does not explain every fluctuation or rule out pseudo-label effects on the common geometry.

5. Label efficiency and clean–robust trade-off

On CIFAR-10, ReAL has the highest mean clean, PGD, APGD, and robust-AUC scores, supporting endpoint and measured trajectory gains (Sec. 6.2, Fig. 1, Table 2). CIFAR-100/RN18 shows later robust benefit after an early weighting penalty, with lower clean accuracy and modest AUC gain. Its endpoint and Tiny 9,000/10,000-label ReAL models include post-hoc LR0.01 reruns against LR0.1 baselines (Appendix A.5). These comparisons do not isolate acquisition effects. We distinguish endpoint robustness from uniform label efficiency. Full curves and three-seed SDs show gains and instability without a statistical-significance claim.

6. Summary of revisions

We updated the theoretical assumptions, ablations, and weighting-related analysis. We appreciate your careful review.

Thank you for the detailed response and additional experiments. The revision substantially addresses the missing component-ablation concern, clarifies the conditional scope of the theoretical analysis, and provides further diagnosis of the early CIFAR-100 degradation. However, several issues that affect the contribution remain:

  • The ablation results do not establish a consistently dominant component or clear marginal gains for each module. Moreover, they do not include the post-hoc learning-rate reduction or other learning-rate choices.
  • The CIFAR-100 ReAL endpoint uses a post-hoc learning-rate reduction to 0.01, whereas the baselines retain 0.1. Without a matched-schedule comparison, the reported endpoint gain cannot be attributed solely to ReAL.
  • The reported gains are not consistently stable, with early clean-accuracy degradation, a clean–robust trade-off, and only modest AUC improvements relative to the observed variability.

These concerns limit the strength of the empirical evidence and the contribution of the proposed method. I therefore maintain my original rating.

Reviewer_1owsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes REAL, an active learning method for adversarial training. REAL combines clean and adversarial gradients into a joint embedding, selects samples using k-means++ with multiple restarts, and assigns cluster-based weights during robust training. The authors provide a coreset approximation bound under a Lipschitz assumption and evaluate REAL on CIFAR-10/100 and Tiny-ImageNet. Results show improvements in robust accuracy, although these often come with lower clean accuracy and substantial variance.

Critical strengths
  1. The paper is well-written and all the notations are clear.
  2. The proposed method is intuitive and looks novel to me.
  3. The theoretical analysis is straightforward and provides a clear interpretation of the cluster-weighting scheme.
Critical weaknesses
  1. Theorem 1 applies to any embedding satisfying Assumption 1 and does not justify the proposed gradient representation over simpler alternatives. Moreover, the assumption may be unrealistic since embeddings use pseudo-labels while robust losses depend on true labels. This is particularly concerning in low-label settings where pseudo-label accuracy is poor. No empirical validation of the assumption is provided.

  2. REAL changes the embedding, selection procedure, and training weights simultaneously. Without ablations, it is unclear which component drives the improvements. Key comparisons, such as REAL without weighting, BADGE with cluster weighting, alternative embeddings, and fewer k-means++ restarts, are missing.

  3. REAL shows sharp clean-accuracy drops and high variance in several settings. The unnormalized cluster weights may cause unstable gradient updates. Since higher robust accuracy often comes at the cost of clean accuracy, it is unclear whether REAL improves acquisition or simply shifts the clean–robust trade-off.

  4. Many improvements are small relative to the variance across only three seeds, especially for robustness AUC and Tiny-ImageNet.

  5. The evaluation should also consider the accuracy against AutoAttack.

Questions for authors
  1. Can you compare REAL with unit weights, BADGE with cluster weights, alternative gradient embeddings, and one versus eight k-means++ restarts? Which component contributes most to the improvement?

  2. What is the distribution of cluster weights across rounds? Could large weights explain the early clean-accuracy collapse? Have weight clipping or normalized weighted losses been tested?

  3. Why does CIFAR-100/WRN use a lower learning rate than the other settings? How sensitive are REAL and the baselines to this choice?

  4. Can you provide empirical evidence supporting Assumption 1, particularly when pseudo-labels are incorrect? Is there any reason to expect the proposed embedding to yield a smaller Lipschitz constant than simpler representations?

  5. Can you provide the clean–robust Pareto comparisons?

1 comment
AuthorsOpenReview ↗

Assumptions, Ablations, Stability, and Robustness

Thank you very much for your constructive comments. References below are to the revised version.

1. Theoretical assumptions and justification

We agree that Theorem 1 is embedding-agnostic and does not establish the secant embedding as the optimal representation. We added a comparison on the same fixed ReAL assignments and checkpoints. With each metric normalized by its sample RMS, Secant New95 is lower than Clean-gradient in 25/27 and 22/27 CIFAR-10/100 cases, with median Clean95/Secant95 ratios of 1.166/1.034 (Appendix A.9). This does not establish uniform LL or compare BADGE acquisition or downstream training.

Original Assumption 1 required true-label robust-loss Lipschitz control for all pool pairs and parameters in a region. The revision bounds response differences Δtr\Delta_t r at two actual checkpoints on at least 1−α1-\alpha of assigned pairs, with bounded checkpoint losses (Sec. 5.2). Theorem 1 gives LtΨt(Bt)+2αCtL_t\sqrt{\Psi_t(B_t)}+2\alpha C_t transfer error. Eq. (22) includes previously labeled data. The bound uses Cauchy–Schwarz for regular pairs and bounded losses for exceptions. Neither exact minimization nor pseudo-label correctness is assumed.

Sec. 5.1 bounds the error in the mean computed attacked gradient without correct pseudo-labels, but not unknown true-label gradients. Appendix A.9 analyzes 54 CIFAR-10/100 ResNet-18 transitions (3 seeds, 9 updates), with 2,048 uniform pre-query pool points and 500 representative loss lookups per case. True-label cross-entropy uses PGD-20$\times$1 (radius 8/2558/255, step 2/2552/255), maximizing over clean/start/all iterates. All 3 cases use the original geometry, while the remaining 51 use reconstructed PGD-5 geometry with fixed assignments/masses. New95 is the 95th-percentile response-discrepancy/RMS-normalized-distance ratio. It is lower than the analogous Old95 loss diagnostic in 27/27 CIFAR-10 and 21/27 CIFAR-100 cases, with median reductions of 28.0%/19.0%. Median NewMax/New95 ratios (5.19/5.08) show substantial tail sensitivity. These finite-PGD sample diagnostics do not certify exact-loss regularity or the full-pool bound. We find mixed differences across correct/incorrect pseudo-label groups, so label correctness alone does not guarantee regularity.

2. Component-wise ablations

Sec. 6.1/Table 1 varies one component under paired CIFAR-10/ResNet-18 warm start (500–3,000 labels). Representation: Secant has the highest mean robust AUC and smallest SD among the four, although Clean has a higher robust endpoint. Weighting: Cluster improves robust endpoint accuracy and AUC over Unit on CIFAR-10, supporting pool-mass preservation. BADGE+Cluster is not reported, and the Clean control differs in first-center and restart rules. Restarts: We test 1/2/4/8/16. Eight restarts give AUC 19.96±0.8419.96\pm0.84 versus 19.66±1.6119.66\pm1.61 for one, close to four restarts' best mean. Eight balances cost/variability, but these tests do not establish additive gains.

3. Cluster weights and training stability

The new-example weights already average one, but batch-size loss normalization only preserves pool mass, not cap individual weights. Our CIFAR-100 intervention fixes first-query examples and training randomness and reproduces the early deterioration under Cluster rather than Unit (Sec. 6.2; Appendix A.8). The concentration audit provides a mechanism with identical queries. Clipping changes the intended mass-preserving objective, and its benefit and full round-wise weight distributions remain unreported. Later variability may have other causes.

4. Learning rate and protocol

CIFAR-100/WRN uses LR0.1 initially and 0.01 later for every method (Appendix A.5). Displayed ReAL models at CIFAR-100/RN18's 5,000 and Tiny's 9,000/10,000 labels are post-hoc LR0.01 reruns; baselines retain 0.1. These endpoints do not isolate acquisition effects. The 54-transition audit uses original LR0.1 updates (Appendix A.9). No LR sensitivity grid is reported.

5. Clean–robust trade-off and acquisition quality

ReAL has the highest mean clean/robust endpoints and robust AUC on CIFAR-10 (Sec. 6.2, Fig. 1, Table 2). CIFAR-100 instead shows stronger final robustness, an early concentration penalty, lower clean accuracy, and modest AUC gains. We report trajectories and paired outcomes. These are operating points with the LR exceptions above, not a swept Pareto frontier. Three-seed SDs do not establish significance, especially for small Tiny-ImageNet differences.

6. Additional attack evaluation

Sequential APGD-CE+DLR tests dependence on fixed-step PGD with adaptive steps and complementary losses. All 90 final checkpoints, 100 iterations per component, 8/2558/255 (Sec. 6.2; Appendix A.7). ReAL's CIFAR-10/100 ResNet-18 robust endpoint advantage remains under this evaluation. Targeted APGD, FAB, and Square are not included, so this is not full AutoAttack.

7. Summary of revisions

We updated the theory, ablations, and controlled failure analysis. We appreciate your careful review.

Reviewer_R1EJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

REAL selects representative training examples using both clean last-layer gradients and attack-induced gradient changes, with cluster-size weights preserving the pool’s distribution. It connects this coverage-based selection to the robust objective and derives an error bound under an explicit condition relating representation distance to robust loss. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show improved adversarial robustness over most evaluated baselines, alongside trade-offs in clean accuracy and variability across random seeds.

Critical strengths

(1) The core idea is straightforward and easy to understand, and the paper presents it clearly.

(2) The authors conduct extensive experiments across multiple datasets, and the promising results provide encouraging evidence of the method’s effectiveness.

Critical weaknesses

(1) The paper’s main contribution is difficult to identify. Even after reading the full manuscript, it remains unclear what the central technical advance is. The paper also does not clearly distinguish its approach from existing methods, such as uncertainty- and gradient-based batch acquisition or coreset- and coverage-based active learning. The authors should explain how their method differs from these approaches and what specific limitation it addresses; this would make the paper’s novelty and contribution easier to assess.

(2) It is unclear why the proposed methodology requires the cluster-weighted update or how much it contributes to the results. The paper also does not discuss computational cost in the experimental section. Please explain the role of cluster weighting, evaluate its effect (for example, through an ablation study), and report the method’s computational cost so readers can assess its practicality and scalability.

(3) The paper’s writing needs improvement. Please proofread the full manuscript, as it still contains typos and grammatical errors that affect readability.

Questions for authors

(1) What is the paper’s main contribution, and how does the proposed method differ from existing approaches?

(2) What is the computational cost of the proposed method, and how does it compare with that of the baselines?

1 comment
AuthorsOpenReview ↗

Clarification of Main Contributions, Cluster Weighting, and Computational Efficiency

Thank you very much for the constructive assessment and suggestions. They helped us sharpen the contribution and strengthen its supporting evidence. Any references in our responses in the following refer to the revised version of the paper, not the original submission.

1. Novelty and contributions

ReAL targets the acquisition signal needed by adversarial training, where examples with similar clean gradients can respond differently to an attack. The proposed secant embedding [gc;ga−gc][g_c;g_a-g_c] separates the clean signal from its attack-induced change, whereas uncertainty scores summarize confidence, BADGE covers clean gradients, and feature coresets cover learned features (Secs. 2 and 4). This geometry lets selection account for both ordinary learning and adversarial response. The concatenated gradient embedding [gc;ga][g_c;g_a] retains the same information but mixes those components in its distance, and the secant embedding changes how they are compared. Selection and mass-preserving training then form one approximation of the unlabeled pool. Sec. 5.1 proves that small coverage controls the error in the pool's mean computed acquisition gradient, even with incorrect pseudo-labels. Sec. 6.1/Table 1 supplies the controlled comparison, where the secant embedding has the highest mean robust AUC and smallest SD among the four embeddings.

2. Role of cluster weighting

Coverage determines which examples represent the pool, while weighting determines how much each contributes to training. If two perfectly represented groups have unequal sizes, training their representatives equally still changes their proportions. Cluster weights remove this mismatch by preserving assigned pool mass exactly (Sec. 4.4, Eq. 14). They connect representative selection to the intended pool objective, even when coverage error is zero. Multipliers qwt(b)/Ntq w_t(b)/N_t average to one across the selected examples, reallocating a fixed new-data weight budget while old examples retain unit weights. The controlled CIFAR-10 comparison shows that Cluster improves mean robust endpoint accuracy and AUC over Unit (Sec. 6.1, Table 1). The CIFAR-100 analysis identifies when concentration needs care (Sec. 6.2), explaining the mechanism's operating limits.

3. Computation and efficiency

Practicality comes from reusing attacks and embeddings across restarts and computing distances through factorized last-layer gradients. For NN candidates, qq queries and MM restarts, seeding uses O(MNq)O(MNq) distance evaluations, or O(MNqd)O(MNqd) arithmetic with explicit dd-dimensional vectors. Extra restarts require neither new attacks nor labels. On a dedicated RTX 4090, acquisition from 49,500 candidates with q=500q=500 takes about 40 seconds for ReAL versus about 400 seconds for SAAL and about 4.5 seconds for clean-gradient BADGE (Appendix A.6). This quantifies the cost of robustness-aware acquisition and shows a substantially smaller overhead than SAAL in this benchmark. Sec. 6.1 tests 1, 2, 4, 8, and 16 restarts. With 8 restarts, ReAL achieves robust AUC close to the highest mean obtained with 4 restarts, with the smallest observed SD, motivating a moderate restart budget. Appendix A.1 separately reports selection-only timings. Their shared-GPU variation is kept distinct from the dedicated acquisition benchmark.

4. Presentation improvements

Sec. 6 now separates design validation from cross-method results. We define the four embeddings, restarts, and Unit/Cluster weights in Sec. 6.1 and state training/attack settings and LR exceptions in Sec. 6.2. We streamlined the coverage and Secant explanations in Secs. 4.2–4.3 and clarified learning-rate notation, making the algorithm's purpose and implementation easier to follow.

5. Summary of revisions

We added controlled ablations, a matched weighting analysis, and strengthened the theoretical results with an empirical learning-response audit (Appendix A.9). Original Assumption 1 required true-label robust-loss regularity for every pool pair and parameter in a region. The revised assumption instead concerns loss changes at the actual two checkpoints on at least a 1−α1-\alpha fraction of assigned pairs, with bounded checkpoint losses, and it requires no exact minimization. In our new experiments, we further report violations of the regularity condition, pseudo-label accuracy, and transfer errors to make the theory's practical limitations explicit. Thank you so much again for helping us clarify the method and its scope.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.