acceptodds
Under review as a conference paper at ICLR 2027

TARO: Temporal Adversarial Rectification Optimization Using Diffusion Models as Purifiers

Abstract

Adversarial purification with diffusion models seeks to project adversarial examples back toward the data manifold, but balancing semantic preservation and robustness against adaptive attacks remains challenging. Recent work shows that standard diffusion purification can fail under adaptive evaluation, while test-time score-based optimization is more resilient. Existing optimization defenses, however, typically rely on a single diffusion noise regime or treat timesteps uniformly, overlooking the distinct roles of coarse and fine denoising scales. We propose Temporal Adversarial Rectification Optimization (TARO), an inference-time purification method that builds a temporally guided score prior from multiple denoising views along the diffusion trajectory. TARO forms a coarse-to-fine residual target: high-noise experts provide globally smoothed structure with reduced adversarial sensitivity, while low-noise experts restore image-specific, class-relevant details. A guidance strength controls this temporal correction, allowing TARO to balance robust global rectification with semantic preservation. Empirically, TARO improves robust accuracy across datasets and adaptive threat models in a zero-shot setting, while remaining compatible with complementary adversarial-likelihood objectives for further robustness gains.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Adversarial Diffusion Purification, Purification with Optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wdHSOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

TARO treats diffusion denoising outputs at different noise levels as complementary experts, constructing a purification target through weighted coarse-to-fine corrections. It optimizes the input image at test time to remove adversarial perturbations. The authors evaluate clean accuracy and robustness through experiments on multiple datasets, ablation studies, and adaptive attacks.

Critical strengths

The authors consider adaptive attack evaluation for optimization-based purification and include fully unrolled attack experiments, helping assess whether the apparent robustness stems from inadequate attacks. Overall, the experimental setup is reasonably designed.

Critical weaknesses
  1. The method mainly combines denoising outputs from different timesteps for rectification, but the theoretical analysis does not sufficiently explain the empirical findings. The gains over the strongest evaluated baselines are modest, and the comparisons cover work only through 2025 without discussing more recent advances. Overall, the paper overstates its contributions and empirical advantages.

  2. The paper lacks clear organization and visual explanations. The presentation is overly lengthy and difficult to follow.

Questions for authors
  1. Which key empirical findings does the theoretical analysis explain? How does it help us understand when the method is effective and why its gains are limited in some settings?

  2. Considering all the experiments, what are the method’s main advantages over the strongest baselines? Do the results justify the paper’s strong claims, and how does the method relate to more recent work?

  3. Please concisely summarize the core technical novelty and the most important findings, identifying the evidence supporting each. The current presentation is lengthy, making these key points difficult to grasp.

1 comment
AuthorsOpenReview ↗

Response to Reviewer

We thank Reviewer for recognizing the value of our adaptive and fully unrolled evaluation protocol, and for the constructive feedback on connecting our theoretical analysis to our empirical findings, comparing against 2026 baselines, and streamlining the paper's presentation. We have substantially revised the manuscript to address each point.

1. Which empirical findings does the theoretical analysis explain, when is TARO effective, and why are gains smaller in some settings?

Proposition 1 (Section 3.4) decomposes the error of the TARO target uγ=uc+γ(uf−uc)u_\gamma = u_c + \gamma(u_f - u_c) into an effective bias term and a cross-scale variance term, while Appendix A.1, under the conditions stated, shows that under a local Gaussian model, extrapolating (γ>1\gamma > 1) sharpens the effective precision Λaff=Λ~f+(γ−1)(Λ~f−Λ~c)⪰Λ~f\Lambda_{\text{aff}} = \tilde{\Lambda}_f + (\gamma - 1)(\tilde{\Lambda}_f - \tilde{\Lambda}_c) \succeq \tilde{\Lambda}_f. Together, this analysis explains the core empirical phenomena:

  • Why moderate extrapolation (γ>1\gamma > 1) helps (Table 7): Moderate extrapolation (γ=1.2\gamma=1.2) achieves 86.56% / 87.30% robustness under ℓ∞/ℓ2\ell_\infty / \ell_2, outperforming fine-only denoising (84.99% / 86.01%) and equal-weight averaging (70.04% / 71.42%). Excessive extrapolation (γ=3\gamma=3) reduces robustness to 80.59% / 78.38%, consistent with the tradeoff between correcting oversmoothing bias and amplifying cross-scale variation.
  • Improved Results : TARO-2 improves robustness under fully unrolled AutoAttack-EOT (83.07% vs. 74.22% for Score-Opt-O), DiffBreak (86.97% vs. 80.20%), CIFAR-100 BPDA-EOT (72.26% vs. 66.20% for AAOpt), and Imagenette (77.63% vs. 69.5% for Score-Opt-O), alongside smaller gains under PGD (Tables 1, 3–5, and 11). Overall, we see improvement in all scenarios in the revised manuscript.

2. Main advantages over the strongest and most recent (2026) baselines, and calibrating claims

  • Zero-shot generalization across threat models without auxiliary training: Unlike AAOpt (Wesego & Rooshenas, 2025), which requires training a separate perturbation likelihood model on generated adversarial examples, TARO is zero-shot, relying solely on a single pretrained clean-image diffusion model. Under unified matched evaluation, pure TARO-2 now outperforms AAOpt in all settings.
  • Comparison with 2026 methods (Tables 1 & 2): We added comparisons against two recent 2026 purification defenses: MultiDAP (Bai et al., ICML 2026), which uses multimodal diffusion with learned prompts, and CMAP (Zhang et al., TPAMI 2026), which optimizes latent consistency models. On CIFAR-10 PGD-EOT (Table 1), TARO-2 performs better.
  • Calibrating claims and streamlining presentation: Following your feedback, we revised the Abstract, Introduction, and Contributions to focus strictly on zero-shot temporal rectification, moved the hybrid TARO+AA experiments to Appendix A.3 (Table 9) so the main text evaluates pure TARO on equal footing, and unified the purification loss (Equation 4, β=0.5\beta = 0.5) so the exact same computation is used during both deployment and attack generation across all experiments.

3. Concise summary of core technical novelty, key findings, and supporting evidence

  • Core Novelty — Affine Coarse-to-Fine Temporal Target: Rather than committing to a single diffusion noise scale per optimization step, TARO evaluates a clean diffusion denoiser at ordered scales (tf<tct_f < t_c) and forms a variance-corrected affine target xtarget=uK+∑j=1K−1γK−j(uj−uj+1)x_{\text{target}} = u_K + \sum_{j=1}^{K-1} \gamma^{K-j}(u_j - u_{j+1}), optimized via a matched partial stop-gradient objective.
  • Finding 1 — Structured coarse-to-fine extrapolation is essential, not extra compute or naive averaging:
    • Outperforms single experts and convex averaging (Table 7)
    • Not an artifact of extra compute (Appendix A.7, Table 12): Allocating the same or greater compute to single-scale TARO-1 via 2×2\times optimization iterations (79.29%), 2×2\times or 3×3\times defense EOT samples (84.96% / 84.70%), or random two-timestep averaging (53.12%) fails to match TARO-2 (86.65%).
  • Finding 2 — The multi-scale temporal prior is modular and transfers across methods and corruptions:
    • Improves other optimization purifiers (Appendix A.9, Table 14): Plugging TARO's multi-scale prior into AAOpt improves its performance.
    • Generalizes to natural corruptions (Appendix A.10, Table 15)
  • Finding 3 — Optimization-based purifiers require graph-consistent higher-order evaluation (Section 4.3, Appendix A.4–A.5):
    • Detaching inner-loop gradients creates false robustness (e.g., CLIPure collapses from 91.1% / 88.0% to 0.0% under graph-consistent evaluation, Table 10), whereas TARO-2 achieves 83.07% under fully unrolled AutoAttack-EOT (Table 4).

We would be grateful if you could reassess your initial rating and consider a more positive assessment. If there are any questions or remaining concern, please let us know during the discussion.

Reviewer_weTNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TARO, a test-time adversarial purification framework that treats diffusion timesteps as scale-dependent denoising experts. Instead of using a single diffusion noise level per optimization update, TARO aggregates multiple denoising estimates into an affine coarse-to-fine target. In their framework, high-noise experts provide globally smoothed structure that suppresses fragile adversarial components, while low-noise experts preserve image-specific and class-relevant details. The paper also provides a bias-variance risk interpretation and a local Gaussian interpretation, and combines TARO with adversary-aware residual likelihood modeling in a TARO+AA variant. This paper also proposes an optimization-aware evaluation protocol emphasizing that optimization-based purifiers require graph-consistent higher-order gradient tracking through the inner purification loop; otherwise, robustness can be severely overestimated.

This paper conducts thorough experiments across various datasets against adversarial training, diffusion purification, and test-time optimization baselines. The experimental results show strong clean accuracy and improved robust accuracy under several adaptive attacks, especially for l∞l_{\infty}, and show that TARO is complementary to AAOpt-style residual modeling.

Critical strengths
  1. The paper identifies a real limitation of existing diffusion-based purifiers: single-noise-level updates collapse multi-scale diffusion information, and adversary-aware methods depend on auxiliary perturbation models. The proposed temporal-expert view is intuitive and well aligned with the robustness-fidelity tradeoff.

  2. TARO can operate purely with a pretrained clean-image diffusion model, without an auxiliary adversarial residual model. The TARO+AA hybrid is also a sensible combination of an internal temporal prior and an external learned residual likelihood.

  3. The paper goes beyond standard PGD evaluation and includes PGD-EOT, BPDA-EOT, fully unrolled AutoAttack-EOT, and DiffBreak low-frequency attacks. The emphasis on attacking the full purifier-classifier composition, using EOT, and preserving higher-order gradients through the inner optimization loop is important and timely.

Critical weaknesses
  1. The core idea, aggregating diffusion denoisers at multiple noise levels and using a guidance-style discrepancy, is closely related to existing multi-scale diffusion purification, test-time ensembling, and guidance methods. The paper distinguishes itself through the affine extrapolation and the temporal PoE interpretation, but the methodological novelty is moderate. The negative-weight affine aggregation is essentially a ratio-of-experts/contrastive prior, and the local Gaussian analysis is conditional and intuitive rather than a rigorous justification.

  2. The risk proposition is a standard bias-variance decomposition. The local Gaussian interpretation requires positive definiteness of the signed precision, which is not guaranteed under negative weights. The paper acknowledges this, but does not provide conditions under which affine extrapolation is stable in practice, nor does it analyze how negative weights affect optimization dynamics. The continuous-time PoE with negative weights is not a standard normalized product of experts, so the probabilistic interpretation is somewhat heuristic.

  3. TARO is strong under PGD-EOT l∞l_{\infty} , but the picture is less uniform elsewhere. In BPDA-EOT on CIFAR-10, pure TARO-2 is below AAOpt (88.90% vs. 91.36%), and only TARO-2+AA surpasses AAOpt. Under DiffBreak, TARO-2+AA is worse than pure TARO-2 ( 77.08% vs. 80.72). Under fully unrolled AutoAttack with stop-gradient, TARO-2 is substantially below AAOpt (71.09% vs. 85.54%). The paper reports averaged results, but the mixed behavior suggests that the advantage of TARO is threat-model and configuration dependent. This should be discussed more honestly and analyzed rather than mostly emphasizing the favorable averages.

  4. The most concerning issue is the stop-gradient treatment. Algorithm 1 uses sg(⋅) in the purifier update, but the attack is sometimes run without sg(⋅). Table 5 reports that TARO-2 achieves 76.73% robust accuracy without stop-gradient and 71.09% with stop-gradient, while AAOpt achieves 57.42% without and 85.54% with. It is unusual that removing stop-gradient increases TARO’s robust accuracy. This may indicate that the “with sg” and “without sg” attacks are not directly comparable, or that the attack configuration changes the effective threat model. The paper needs to clarify exactly which computation is deployed and which computation is attacked.

Questions for authors
  1. Does the deployed TARO purifier use sg(⋅) or not? If the defense uses sg(⋅), why is the attack sometimes run without it? If the defense does not use it, why does Algorithm 1 include it? Please clarify the exact computation used at inference and the exact computation attacked.

  2. In Table 5, removing stop-gradient increases TARO-2 robust accuracy from 71.09% to 76.73%, while AAOpt drops from 85.54
    % to 57.42%. Usually, exposing more gradients should make the attack stronger and reduce robust accuracy. Can the authors explain this behavior? Are the “with sg” and “without sg” attacks using the same number of iterations, EOT samples, and attack objective? Could the “with sg” attack be accidentally stronger due to a different optimization trajectory?

  3. The paper compares against AAOpt and Score-Opt but does not include several obvious baselines: simple averaging or ensembling of multiple diffusion denoisers at different noise levels, random-timestep multi-call purification, test-time augmentation over noise samples, and single-scale purification with matched total denoiser evaluations or matched wall-clock time. Table 11 compares TARO-1 with additional EOT samples, but it does not compare TARO-2 against a single-scale method with twice the number of iterations or twice the number of denoiser calls. Can the authors show such experimental results?

1 comment
AuthorsOpenReview ↗

Response to Reviewer

We thank the reviewer for the review and the detailed assessment in recognizing the temporal-expert construction, its use of a pretrained clean-image model, and the breadth of adaptive evaluation. We have strengthened the revision around your concerns.

1. Clarifying and unifying the stop-gradient formulation across deployment and attack

  • Why the original "with sg" vs. "without sg" difference occurred: In our original submission, Algorithm 1 used sg(⋅)\text{sg}(\cdot) during purification for efficiency, while our PGD-EOT attack removed sg(⋅)\text{sg}(\cdot) during the backward pass to ensure we were not hiding gradients behind the diffusion denoiser. Under fully unrolled AutoAttack, we removed sg(⋅)\text{sg}(\cdot) from both the attack and defense which altered the purifier's inner optimization trajectory itself: removing sg(⋅)\text{sg}(\cdot) exposed the denoiser Jacobian ∂xtarget∂xi\frac{\partial x_{\text{target}}}{\partial x_i}, which actually helped TARO-2's inner restoration trajectory (raising TARO-2 from 71.09% to 76.73%). To avoid such issues while still allowing denoiser gradients to be visible to both attacker and purifier, we have updated all experiments to use the same setting in both attack and defense.

  • Unified formulation in the revised manuscript (Eq. 4 & Algorithm 1): We completely agree that the deployed defense and the attacked defense must be identical. In the revised manuscript, we introduce a single unified purification objective (Section 3.2, Equation 4): x~target=(1−β)sg(xtarget)+βxtarget,ℓi=∥xi−x~target∥22.\tilde{x}_{\text{target}} = (1-\beta)\text{sg}(x_{\text{target}}) + \beta x_{\text{target}}, \quad \ell_i = \lVert x_i - \tilde{x}_{\text{target}} \rVert_2^2.

    and use the exact same loss and mixing coefficient (β=0.5\beta = 0.5) during both deployment and attack generation across every experiment in the paper.

  • Matched stop-gradient ablation (Section 4.5, Table 8) and unified results: In addition, we also add ablation about this in Table 8 where our default β=0.5\beta = 0.5 achieves the best overall balance (91.91% clean, 86.56% ℓ∞\ell_\infty, 87.30% ℓ2\ell_2, 86.93% average). Moreover, with this unified β=0.5\beta = 0.5 formulation, pure TARO-2 now consistently outperforms all other baselines across every threat model.

2. Additional aggregation and computation baselines.

The original Table 11 tested increased defense EOT which increased the number of denoiser evaluations during defense. Following your suggestion, we have expanded that comparison with doubled single-scale iterations and random-timestep averaging. Revised Table 12 reports are also shown below for reference:

ConfigurationRobust accuracy (%)
TARO-184.82
Doubled single-scale iterations79.29
Random-timestep averaging53.12
Doubled defense EOT84.96
Tripled defense EOT84.70
TARO-286.65

Table 7 also reports equal-weight averaging which obtains 70.04% / 71.42% under ℓ∞/ℓ2\ell_\infty/\ell_2, compared with 86.56% / 87.30% for TARO-2. These comparisons support the importance of how temporal information is combined and how TARO-2 achieves improved results.

3. Contribution and theoretical interpretation.

We clarify in Section 3.1 and Appendix A.1 that the effective expert precisions are Λ~k=Λσtk2Σtk−1\tilde{\Lambda}_k=\Lambda\sigma_{t_k}^2\Sigma_{t_k}^{-1}, with Λ>0\Lambda > 0. For γ>1\gamma > 1, the combined precision Λaff=γΛ~f−(γ−1)Λ~c\Lambda_{\mathrm{aff}}=\gamma\tilde{\Lambda}_f-(\gamma-1)\tilde{\Lambda}_c is positive definite exactly when Λ~f≻γ−1γΛ~c\tilde{\Lambda}_f\succ\frac{\gamma-1}{\gamma}\tilde{\Lambda}_c. The stronger condition Λ~f⪰Λ~c\tilde{\Lambda}_f\succeq\tilde{\Lambda}_c, with Λ~f≻0\tilde{\Lambda}_f\succ 0, also guarantees sharpening relative to the fine expert.

Proposition 1 complements this conditional interpretation: its variance term is quadratic in γ\gamma, identifying a potential cost of excessive extrapolation. Consistently, Table 7 shows peak robustness at γ=1.2\gamma=1.2 and deterioration at larger correction strengths.

4. Performance across threat models.

Your observation about the original results is accurate. For the revised mixed configuration, pure TARO-2 obtains 93.27% under CIFAR-10 BPDA-EOT, compared with the original 88.90% and AAOpt’s 91.36%. The updated fully unrolled AutoAttack-EOT result is 83.07%, compared with 76.73% for the original non-stopped TARO variant. Under DiffBreak LF, TARO-2 obtains 86.97%, compared with its original 80.72% and Score-Opt-O’s 80.20%. These results consistently outperform different baselines in different settings.

We thank the reviewer for the suggestions. We hope this new evidence addresses the main reservations behind your initial recommendation, and we would appreciate your reconsideration in light of the revised manuscript. If a specific concern remains, we would welcome the opportunity to address it during the discussion.

Reviewer_1YczOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TARO builds an affine target from a diffusion model's denoising predictions at several noise scales, then optimizes the input towards that target at test time. The goal of TARO is to purify the adversarial perturbations while ensuring that the semantic content is not lost. TARO does not require separate training of the perturbation model, but can be used in combination with the adversarial perturbation likelihood learned by AAOpt. The paper explains the TARO method through theories such as the sequential product expert model (PoE), local Gaussian model, and bias-variance decomposition. The paper also conducted a large number of experiments in multiple adaptive attack scenarios and multiple datasets.

Critical strengths
  1. The TARO method integrates denoising predictions at different noise scales, which is an interesting approach. Additionally, TARO can be easily combined with existing diffusion-based adversarial purification methods. Furthermore, TARO has an optional likelihood inspired by AAOpt, which further enhances its flexibility.
  2. This paper compares the use of only coarse-grained and only fine-grained objectives, as well as interpolation and extrapolation strategies through experiments. The experimental results show that moderate extrapolation can simultaneously improve both clean and robust accuracy.
  3. This paper conducted various types of ablation experiments, including adaptive attacks, fully unrolled evaluations, and stop-gradient comparisons.
Critical weaknesses
  1. The results of the original defense method may not be accurate. For Algorithm 1, when calculating the inner gradient, the temporal target is fixed. However, removing the stop-gradient strategy leads to the introduction of the target Jacobian matrix. Usually, this changes the way of adversarial purification update. Looking at Section 4.3.1 again, in this section, neither defense nor attack uses the stop gradient strategy. Does this mean that the experiments in Table 4 are conducted on another variant?
  2. Algorithm 1 uses independently sampled Gaussian noise as the input. This does not seem to guarantee a correspondence with Equation (3).
  3. This paper does not provide the complete experimental settings for Table 4 and Table 5.
Questions for authors
  1. Can you provide the original numerical update and the target Jacobian matrix, especially for the " With sg" case in Table 5?
  2. Can you associate Equation (3) with Algorithm 1? Can you explain the relationship between the construction and implementation of the prior distribution as described in Appendix A.1?
  3. Can you provide the complete experimental settings?
1 comment
AuthorsOpenReview ↗

Response to Reviewer

We thank the reviewer for the review and for recognizing the value of temporal aggregation and the need to clarify the stop-gradient formulation and its evaluation. We address your concerns below.

1. Stop-gradient and consistency between attack and defense.

The initial implementation had a mismatch in the attack and defense settings for the PGD evaluation, while having a matched setting but one with no stop gradient for the full unrolled evaluation. We agree that removing stop-gradient from the inner loss can change the purification update itself. We have revised this in Equation 4 and Algorithm 1 to specify one mixed formulation, using β=0.5\beta=0.5, for both attack generation and defense. The revision adopts this mixed formulation throughout all the experiments using the same loss and setting during attack generation and defense. The stopped contribution holds part of the target fixed during the inner update, while the differentiable contribution allows the target and denoiser to influence that update. We also replace the original Table 5 with the new matched-target ablation in Section 4.5, Table 8, evaluated under PGD-EOT for both norms. The mixed formulation substantially improves ℓ2\ell_2 robustness over the stopped target while retaining strong ℓ∞\ell_\infty robustness. In addition, it shows improved performance across different settings.

  1. Exact numerical update and target Jacobian:

Taking the gradient of ℓi(xi)\ell_i(x_i) with respect to xix_i yields: ∇xiℓi(xi)=2(I−βJtarget(xi)⊤)(xi−xtarget),\nabla_{x_i} \ell_i(x_i) = 2\left(I - \beta J_{\text{target}}(x_i)^\top\right)(x_i - x_{\text{target}}), where Jtarget(xi)=∂xtarget∂xi=∑j=1Kαj∂Dθ(xi+σtjϵj,σtj)∂xiJ_{\text{target}}(x_i) = \frac{\partial x_{\text{target}}}{\partial x_i} = \sum_{j=1}^K \alpha_j \frac{\partial D_\theta(x_i + \sigma_{t_j}\epsilon_j, \sigma_{t_j})}{\partial x_i} is the multi-scale denoiser Jacobian.

  • When β=0\beta = 0 ("With sg"), the Jacobian term vanishes (I−βJtarget⊤=II - \beta J_{\text{target}}^\top = I)
  • When β=1\beta = 1 ("No sg"), the full denoiser Jacobian Jtarget(xi)⊤J_{\text{target}}(x_i)^\top is included in every inner step.
  • When β=0.5\beta = 0.5 (our default across all main experiments in the revision), the inner gradient retains the direct target-residual contribution and includes half of the Jacobian contribution present in the fully differentiable loss, preserving a gradient path through the diffusion denoiser.

3. Equation (3), Algorithm 1, and the prior interpretation.

Thank you for this question. Our earlier presentation left the distinction between the candidate image x\mathbf{x} and the noisy expert input xt=x+σtϵ\mathbf{x}_t=\mathbf{x}+\sigma_t\boldsymbol{\epsilon} implicit. We have made this distinction and the expectation over the injected noise explicit in Section 3.1.

The connection is also explicit algebraically: rearranging Algorithm 1’s weighted telescoping residual sum gives xtarget=∑j=1Kαjujx_{\mathrm{target}}=\sum_{j=1}^{K}\alpha_j u_j, with ∑jαj=1\sum_j\alpha_j=1. For example, with two experts, the target is γuf+(1−γ)uc\gamma u_f+(1-\gamma)u_c. Conditional on the selected timesteps, independently sampling the expert perturbations therefore produces a single-sample Monte Carlo realization of the expected temporal target Dˉα(xi)\bar D_\alpha(\mathbf{x}_i) in Equation (3). Equation (4) then specifies the loss used with this target.

Appendix A.1 provides a complementary Gaussian interpretation of this aggregation. We have clarified how noise averaging and the variance factors enter the effective experts, making their relationship to Section 3.1 explicit. The precision analysis is conditional on the resulting precision matrix being positive definite; it does not assume that ordering the noise levels alone guarantees this condition. We hope these revisions make the connection between the prior, temporal target, and implementation easier to follow.

4. Experimental settings.

We tried to provide all the settings we used in our experiments. The revised Sections 4.1–4.3 and Appendix A.12 provide additional settings that we added. PGD-EOT uses 20 attack iterations, budgets 8/2558/255 and 0.50.5 for ℓ∞\ell_\infty and ℓ2\ell_2, and 20 samples for attack EOT and defense averaging. BPDA-EOT uses 50 attack iterations and EOT=20. Fully unrolled AutoAttack-EOT uses ℓ∞\ell_\infty, ϵ=8/255\epsilon=8/255 with EOT=5. DiffBreak LF uses 100 attack steps, EOT=20, and we take the default settings for the rest. Purification learning rates and schedules are specified in Appendix A.12. We have also attached the code in the supplementary materials.

We appreciate your constructive suggestions, which led to a more clearly specified method and a direct comparison of the target formulations. If these revisions resolve your concerns, we would be grateful if you could reflect the strengthened evidence in your updated assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Main Comment

Dear AC and Reviewers,

We sincerely thank you for your thoughtful reviews and constructive suggestions. We have updated the main paper and appendix, with changes marked in blue, to address the concerns about the formulation, evaluation, and comparisons.

  1. Unified Defense and Attack Formulation: We introduced a partially stopped target, x~target=(1−β)sg(xtarget)+βxtarget\tilde{x}_{\text{target}}=(1-\beta)\text{sg}(x_{\text{target}})+\beta x_{\text{target}}, using the same purification loss and mixing coefficient β=0.5\beta=0.5 during defense and attack generation. The matched-target ablation shows that this formulation substantially improves robustness and achieves the best results without an auxiliary residual model in different settings.

  2. Additional Computation Controls and Recent Baselines: We expanded Appendix A.7, Table 12, with doubled single-scale iterations (2×2\times), increased defense EOT (2×2\times–3×3\times), and random-timestep averaging. Table 7 also compares against equal-weight averaging (γ=0.5\gamma=0.5). These controls support the importance of how temporal information is combined. We additionally included 2026 baselines MultiDAP (Bai et al., ICML 2026) and CMAP (Zhang et al., TPAMI 2026) in the revised comparisons.

  3. Updated Zero-Shot Results and Clearer Presentation: The revised TARO-2 formulation improves results under PGD-EOT, BPDA-EOT, fully unrolled AutoAttack-EOT, and DiffBreak LF, with additional evidence from CIFAR-100 and Imagenette. These results demonstrate the benefit of temporal correction without training an auxiliary adversarial residual model. We moved the TARO+AA results to Appendix A.3 to keep the main presentation focused on pure zero-shot TARO.

Below, we provide point-by-point responses to each reviewer. We hope the additional evidence and clarifications address your concerns, and we respectfully invite you to reconsider your assessment in light of the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.