acceptodds
Withdrawn from ICLR 2027

Procedural Adversarial Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

For many computer vision tasks, neural networks can be trained to produce a low average error, yet they still fail on seemingly simple scenes. Reducing these failures requires first finding them, but they may not be in a fixed dataset. We propose procedural adversarial training (PAT), a minimax framework in which a learner trains a model while an adversary searches the input space of a procedural generator. The procedural generator is an interpretable program mapping a low-dimensional vector of control parameters (e.g. layout, lighting, pose) to a rendered image and its label. We introduce the Simple-PAT algorithm, which uses zeroth-order optimization in the procedural generator's parameter space to propose adversarial training samples that are generated on demand. We also create PAT-Dojo, a collection of efficient procedural generators for surface normal estimation, chair segmentation, and human pose estimation. Against a cross-model black-box adversary with 50M queries, Simple-PAT reduces worst case error by a geometric mean of compared to standard training techniques.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer Vision, robustness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HH2VOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies procedural adversarial training by combining procedural scene generation with CMA-ES-based sampling. While the experimental results demonstrate promising robustness improvements, I have several concerns regarding the novelty and validation of the proposed approach.

Critical strengths

The paper explores adversarial robustness across different vision tasks by optimizing the parameters of procedural image generation, which offers a more practical and dynamic approach to robustness evaluation than conventional adversarial perturbations.

Critical weaknesses
  1. The main algorithm appears to be an application of CMA-ES to adversarial sampling, with limited algorithmic innovation beyond heuristic sampling and restart strategies. There is no evidence that CMA-ES is the optimal optimiser for these tasks.

  2. Defence against functional perturbations has been an active research field in recent years. The related work section also insufficiently covers recent advances in adversarial scene generation and robustness evaluation.

  3. On the experimental side, I would particularly like to see a query-matched random search baseline and end-to-end runtime comparisons to substantiate the efficiency claims. Meanwhile, it would be better to compare CMA-ES with other optimisation methods, such as DIRECT and Bayesian optimisation.

  4. While it is generally impossible to find the worst-case perturbation, adversarial robustness studies often attempt to derive some form of bounds, either theoretical or probabilistic, to estimate the worst-case scenario. Although the paper discusses the asymptotic consistency of worst-case estimation, it remains unclear how closely the reported finite-budget results approximate the true worst case. Further discussion of estimation uncertainty, convergence behaviour, or possible bounds would strengthen the robustness claims.

Questions for authors
  • If I understand correctly, the evaluation worst-case error is defined as the maximum error found across DIRECT-L, CMA-ES, and a uniformly sampled test set. This makes it difficult to assess the effectiveness of different optimisation strategies and understand which method primarily contributes to the reported worst-case performance. I suggest that the authors should at least report the results obtained by DIRECT-L, CMA-ES, and uniform random sampling separately in the appendix.

  • Although the paper describes some constraints on scene generation, it is unclear how the validity of the generated adversarial images is verified. Are there any additional checks to ensure that the generated scenes are physically plausible and that their labels remain valid?

Reviewer_JuJBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Procedural Adversarial Training (PAT), a framework that uses image rendering procedural generators with controllable parameters to generate training samples on demand. The proposed Simple-PAT algorithm employs zeroth-order optimization to search within the generator parameter space for challenging samples that maximize model loss as the intern optimization goal, then minimizes the outer loss for improving model robustness, similar to adversarial training.

Critical strengths
  1. Improving model robustness to natural adversarial examples or unexpected failures in real-world deployment is an important research problem, particularly for safety-critical systems.

  2. The use of zeroth-order optimization to search the low-dimensional parameter space of procedural generators provides a practical approach to generating challenging training samples.

Critical weaknesses
  1. The proposed framework relies on procedurally rendered training samples, which may differ substantially from real-world data. This raises concerns about whether the proposed approach adequately addresses the real-world robustness challenges highlighted in the introduction.

  2. The manuscript reads more like a technical report than a research paper, particularly in its early 3 sections. While research papers need not follow a rigid structure, a clear narrative is essential. The introduction should BRIEFLY and CLEARLY establish the research background, related literature, existing gaps for clarify the rationale for the proposed approach before presenting the method and contributions. The current introduction lacks a sufficiently focused narrative. Moreover, Sections 3 and 4 primarily describe how the proposed methods are implemented, with insufficient justification of why these particular design choices are made.

  3. The paper would benefit from discussing whether more recent zeroth-order optimization algorithms have been considered or evaluated.

Questions for authors

None beyond the concerns raised above.

Reviewer_7RbYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Procedural Adversarial Training (PAT), a framework that aims to improve worst-case error rather than focusing solely on average error. The main idea is to augment the training data with challenging and realistic procedurally generated examples, allowing the model to better identify and address difficult scenarios. To achieve this, the proposed method uses Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search the procedural parameter space for challenging examples on which the model performs poorly. These examples are then incorporated into subsequent training batches to progressively improve the model's robustness to worst-case scenarios. The method is evaluated on three computer vision tasks: surface normal estimation, object segmentation, and human pose estimation. The results demonstrate significant improvements in worst-case performance compared to several baselines, including uniform sampling, Conditional Value at Risk (CVaR), and Tilted Empirical Risk Minimization (Tilted ERM).

Critical strengths

1. Clear presentation and organization:

The paper is well-written, well-structured, and easy to follow. The contributions are clearly presented, and the experimental methodology is rigorous and well-designed to support the authors' claims.

2. Evaluation across diverse tasks:

I particularly appreciate the evaluation across three heterogeneous computer vision tasks, which helps demonstrate the effectiveness and versatility of the proposed approach.

3. Simple and computationally efficient adversarial strategy:

The proposed adversarial training strategy is simple yet effective, achieving substantial robustness improvements without introducing significant additional computational overhead.

4. Valuable benchmark contribution:

The introduction of PAT-Dojo is timely and represents a valuable contribution that could facilitate future research on procedural adversarial training and robustness evaluation.

5. Strong empirical results:

The experimental evaluation demonstrates substantial and consistent improvements in worst-case error across different tasks, providing convincing empirical evidence of the effectiveness of the proposed framework.

Critical weaknesses

1. Limited theoretical analysis:

The paper would benefit from a theoretical analysis of the convergence properties of PAT, particularly starting from the worst-case objective in Eq. (1) (line 188).

2. Dependence on task-specific procedural generators:

PAT relies on carefully designed procedural generators, which may require substantial task-specific expertise and manual engineering.

3. Limited justification of the CMA-ES design choices:

The use of diagonal covariance in CMA-ES is motivated by the small population size, but it may fail to capture dependencies between procedural parameters. An ablation comparing different covariance approximations or optimization strategies would strengthen the methodological justification.

4. Missing comparisons with related adversarial augmentation methods:

The framework shares similarities with existing approaches such as AutoAugment and Adversarial AutoAugment. Comparisons with these methods would help clarify the novelty of PAT and distinguish the benefits of procedural generation from those of adversarial sampling.

5. Limited evaluation of generalization and robustness:

The evaluation is restricted to the procedural environments used during training, making it unclear whether the improvements generalize to unseen environments or real-world distributions.

6. Trade-off between worst-case and average performance: Although PAT significantly improves worst-case error, Table 2 shows some degradation in median performance compared to uniform training. A more detailed analysis of this trade-off would help assess the practical benefits of the method.

Questions for authors
  1. How does PAT compare conceptually or empirically with approaches that actively collect additional training data from identified model failures, such as Tesla's fleet-based data collection strategy? Would PAT still provide significant benefits when large amounts of real-world data are available for targeted retraining?

  2. To what extent does constructing procedural generators require task-specific expertise and manual engineering? How sensitive is PAT to the expressiveness of these generators, and do the robustness improvements generalize to failure modes not represented by the procedural parameter space?

  3. Could the authors investigate whether the diagonal covariance approximation limits the adversary's ability to discover challenging configurations, particularly when procedural parameters are strongly correlated? A comparison with full-covariance or low-rank CMA-ES under similar computational budgets would help justify this choice.

  4. The proposed framework shares conceptual similarities with prior augmentation methods, particularly AutoAugment [1] and Adversarial AutoAugment [2]. Additionally, Learning to Generate Training Datasets for Robust Semantic Segmentation [3] explores GAN-based generation of realistic and challenging training examples to improve robustness to distribution shifts and out-of-distribution (OOD) scenarios. Could the authors clarify the methodological differences and explain to what extent the improvements arise from procedural generation rather than adversarial sampling? A comparison with these approaches would strengthen the experimental evaluation. A more detailed discussion of these approaches in the related work section would also help better contextualize the proposed contributions.

[1] AutoAugment (Cubuk et al., CVPR 2019). [2] Adversarial AutoAugment (Zhang et al., ICLR 2020). [3] Learning to Generate Training Datasets for Robust Semantic Segmentation (Hariat et al., WACV 2024).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.