acceptodds
Under review as a conference paper at ICLR 2027

TALA: Surrogate-Free Query-Based Black-Box Attacks on Image Forgery Localization

Abstract

Cross-detector adversarial transfer for image forgery localization varies across surrogate–target pairs. Our analysis identifies substantial source-to-target effectiveness gaps on initially well-localized images, while attribution analysis suggests that the forensic evidence used for localization is both model- and input-dependent. These observations motivate us to explore whether victim localization feedback alone can guide effective attacks, without relying on cross-detector transfer. To this end, we propose Target-Adaptive Localization Attack (), to the best of our knowledge, query-based attack specifically designed for image forgery localization that requires neither a surrogate detector nor a surrogate-trained attack policy. TALA iteratively updates the image through local perturbations, using victim feedback to adapt their location, spatial extent, and type. It combines victim-guided patch-center sampling with failure-aware spatial penalties and determines patch sizes from the entropy-based effective support of the guided distribution. A shared rejection-feedback mechanism coordinates spatial search and switching between complementary smooth and high-frequency perturbation engines. Across three heterogeneous detectors, five datasets, and four evaluation conditions, TALA achieves the highest mean attack success rate in settings and ties for the highest in one additional setting against four baselines. It simultaneously achieves higher attack success, lower LPIPS, and higher PSNR than the strongest baseline in attack success rate in 51 settings. These findings establish surrogate-free target-side adaptation as an effective complementary approach to black-box robustness evaluation and expose the susceptibility of forgery localization detectors to local perturbations guided directly by their own feedback.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Black-Box Attack, Adversarial Robustness, Image Forgery Localization, Image Forensics

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Jd2YOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TALA, a surrogate-free query-based attack on image forgery localization. It uses victim feedback to adapt perturbation location, size, and type. Experiments across three detectors and five datasets report higher attack success rates than existing baselines in most evaluated settings.

Critical strengths

Dense localization feedback provides spatial information that TALA uses to guide its search without training a surrogate. The results across multiple detectors and datasets support the value of this approach when target feedback is available.

Critical weaknesses

TALA requires repeated access to the deployed detector’s pixel-level probability maps, which may be difficult to obtain in practice. Transfer-based attacks can prepare adversarial images offline without target feedback. Without evidence on realistic access and generation time, the practical threat posed by TALA is not well established.

Questions for authors
  1. In practice, attackers may not know which forgery detector is deployed. TALA nevertheless depends on feedback from the deployed detector to optimize its perturbations. This access assumption limits its applicability, and the setting is not clearly more realistic than generating adversarial images on surrogate models and transferring them to an unknown target.

  2. The paper explicitly notes that attack transferability depends strongly on the detector and input image, but does not evaluate the cross-detector transferability of TALA. For example, it remains unclear whether adversarial images generated against TruFor are effective against FOCAL or IFOSN.

  3. The Integrated Gradients visualizations lack implementation details, including the output being attributed and the quantity with respect to which gradients are computed. This makes the actual meaning of the heatmaps difficult to assess.

  4. Transfer-based attacks can generate adversarial images offline, whereas TALA requires repeated victim queries during attack generation. Its computational complexity and actual generation time are therefore important to assessing its practical threat. The paper does not compare these costs, leaving the time overhead of its online search unclear.

  5. Perturbation budgets are selected separately across experimental settings, and TALA is allowed larger perturbations in some cases to improve ASR. The rule for balancing attack success and visual quality is not quantified, making the source of the reported advantage difficult to determine.

  6. The method relies on several hand-designed rules and numerous hyperparameters, but the intuition and motivation behind these choices are not sufficiently explained.

13 comments
AuthorsOpenReview ↗

Response to Reviewer Jd2Y (Part 1/8)

We truly appreciate your careful reading and constructive feedback. Your comments are well taken and they have helped us make the practical threat and the method itself much clearer. We address each point in detail below, with new experiments and quantitative evidence, and we hope these responses resolve your concerns.

We would be glad to clarify anything further during the discussion. The revised PDF now incorporates the new experimental results and the corresponding changes in wording discussed below, highlighted in blue for ease of checking.

W1 & Q1 & Q4: Threat-model positioning and realistic access.

W1: TALA requires repeated access to the deployed detector’s pixel-level probability maps, which may be difficult to obtain in practice. Transfer-based attacks can prepare adversarial images offline without target feedback. Without evidence on realistic access and generation time, the practical threat posed by TALA is not well established.

Q1: In practice, attackers may not know which forgery detector is deployed. TALA nevertheless depends on feedback from the deployed detector to optimize its perturbations. This access assumption limits its applicability, and the setting is not clearly more realistic than generating adversarial images on surrogate models and transferring them to an unknown target.

Q4: Transfer-based attacks can generate adversarial images offline, whereas TALA requires repeated victim queries during attack generation. Its computational complexity and actual generation time are therefore important to assessing its practical threat. The paper does not compare these costs, leaving the time overhead of its online search unclear.

1. Positioning: two standard, complementary threat models.

  • We aim to complement transfer attacks, not compete with them. They have a real practical advantage: adversarial images can be prepared offline without victim feedback.
  • Both are standard black-box threat models, long studied in parallel, each with its own assumption, and neither subsumes the other. By the criterion that query access makes a threat model unrealistic, the entire query-based literature in image classification (e.g., Square Attack, ECCV'20) would be dismissed as well—yet it is studied precisely because both families are needed to chart a model's security boundary.
  • The gap we close. In forgery localization, surrogate-free query attacks remain almost unexplored. We will revise the manuscript to state this positioning and the access assumption explicitly.

2. The access assumption is weaker than it may appear. TALA observes only the victim’s standard localization output (predicted map or confidence score): no weights, no gradients, no internal state, and no knowledge of which detector is deployed. Not knowing the deployed model (Q1) therefore poses no obstacle—TALA adapts to whatever responds, which is exactly the setting it targets. Transfer attacks, generated offline, have no such mechanism when a perturbation fails on the deployed model.

3. Such access exists in real deployments. Baidu's image-tampering-detection API returns a tampering heatmap (return_heatmap=true) and an image-level confidence score (detect_proportion=true) [Baidu API]; Copyleaks' AI-image-detection API returns spatial detection masks [Copyleaks API]. Spatially-resolved detection feedback is therefore not hypothetical.

4. Research value beyond immediate deployability. As in classification, output-only query attacks chart the security boundary of detectors when only their outputs are exposed, and form a natural progression toward harder hard-label settings. We hope this axis proves worth exploring in its own right, beyond the specifics of any one deployment scenario.

AuthorsOpenReview ↗

Response to Reviewer Jd2Y (Part 2/8)

W1 & Q1 & Q4: Generation time (continued).

5. Generation time (Table R3-1). TALA generates adversarial images 3–9× faster per image than the transfer baselines PGN and MEF [1] (TIFS'26; shown to transfer better than PGN and the CVPR'25 method OPS [2] in image classification tasks [1]), while—under exactly the same configurations—achieving comparable ASR and LPIPS in the cross-detector transfer setting (response to Q2, Table R3-4 below) and significantly higher ASR at lower LPIPS in the query setting (Table R3-3).

Protocol. All methods are timed on the same single GPU (NVIDIA RTX 6000 Ada) over identical images, measuring only adversarial-example generation (no I/O or data loading); each call is wrapped with torch.cuda.synchronize() around a wall-clock timer, and the first image is discarded as warm-up. We report the mean and median over the 15 detector–dataset settings. Timed regions and time-critical parameters: for PGN and MEF, the full optimization loop at their officially recommended T=20T=20 steps, including all NN augmented samples per step, as implemented in their official code; for TALA, the full query loop until success or query budget exhaustion at Q=50Q=50.

Table R3-1. Per-image generation time.

MethodAccessMean (s)Median (s)vs. TALA
TALA (Ours)query (online)12.88.41×
MEFtransfer (offline)39.222.43.1× slower
PGNtransfer (offline)112.465.48.8× slower

The gap has a structural cause: TALA terminates as soon as an image succeeds, whereas transfer attacks always run their full fixed schedule of T×NT \times N surrogate forward/backward passes, where TT is the number of optimization steps and NN the number of augmented input samples drawn per step (both set to 20 by their official configurations). The medians show the gap is not driven by the large-image IMD2020 outlier that inflates all methods.

Two further costs also favor TALA:

  • peak GPU memory is low, as TALA performs no back-propagation and stores no activation graphs;
  • transfer attacks additionally carry the one-time offline cost of training/aligning a surrogate, which TALA avoids entirely. The online search thus introduces no time penalty—per image, it is in fact cheaper than offline transfer generation.

[1] Qiu C, Li A, Duan Y, et al. Boosting adversarial transferability with low-cost optimization via maximin expected flatness. IEEE TIFS, 2026.

[2] Guo Y, et al. Boosting adversarial transferability through augmentation in hypothesis space. CVPR 2025, pp. 19175–19185.

AuthorsOpenReview ↗

Response to Reviewer Jd2Y (Part 3/8)

W1 & Q1 & Q4: Generation computational complexity (continued).

6. Computational complexity (Table R3-2). We characterize cost hardware-independently by the number of network evaluations, reporting forward and backward passes separately. TALA performs no back-propagation: at most Q=50Q=50 victim forward passes per image (average ≈20\approx 20) and zero backward passes. MEF runs T×N=20×20=400T\times N = 20\times20 = 400 surrogate forward and 400 backward passes; PGN also uses T=20T=20 and additionally computes a gradient-norm penalty (an extra gradient per sample), i.e., a comparable-or-larger fixed schedule, consistent with its ≈3×\approx 3\times wall-clock over MEF.

Table R3-2. Per-image network evaluations.

MethodVictim forwardsSurrogate forwardsSurrogate backwards
TALA (Ours)≤50\le 50 (avg ≈20\approx 20)00
MEF0400400
PGN0≥400\ge 400≥400\ge 400

The two families spend entirely different resources: TALA, a small number of victim queries with no gradient computation; transfer attacks, hundreds of surrogate forward and backward passes with no victim access.

Summary. Feedback that is the detector's standard output, no knowledge of the deployed model required, at most 50 queries, 3–9× faster and lower-memory generation, and no surrogate-training cost—TALA constitutes a realistic, practically relevant threat model, complementary to transfer-based attacks.

Table R3-3. Query-setting attack effectiveness and perceptual quality on the large-scale evaluation — images with initial localization F1 >0.5> 0.5 under a maximum query budget Q=50Q=50 (up to 1000 per detector–dataset pair). Each entry reports ASR (%) ↑\uparrow / LPIPS ↓\downarrow.

VictimMethodCASIAv1COCOGlideCOVERAGEIMD2020FaceApp
TruForPGN18.1 / 0.0455‾\underline{0.0455}67.0‾\underline{67.0} / 0.177456.1‾\underline{56.1} / 0.194940.5‾\underline{40.5} / 0.258094.7 / 0.1817‾\underline{0.1817}
MEF19.3‾\underline{19.3} / 0.047745.9 / 0.1190‾\underline{0.1190}25.5 / 0.1353‾\underline{0.1353}36.6 / 0.2252‾\underline{0.2252}95.0‾\underline{95.0} / 0.1871
Ours61.4 / 0.035096.4 / 0.096985.5 / 0.127474.5 / 0.190598.1 / 0.1493
IFOSNPGN71.7‾\underline{71.7} / 0.1632‾\underline{0.1632}80.6‾\underline{80.6} / 0.1271‾\underline{0.1271}73.7‾\underline{73.7} / 0.0873‾\underline{0.0873}50.0‾\underline{50.0} / 0.1820‾\underline{0.1820}89.9‾\underline{89.9} / 0.2546
MEF65.3 / 0.164072.9 / 0.129463.2 / 0.094638.8 / 0.199288.8 / 0.2446‾\underline{0.2446}
Ours82.4 / 0.154995.8 / 0.1072100.0 / 0.078958.9 / 0.179393.7 / 0.2282
FOCALPGN37.1 / 0.1295‾\underline{0.1295}60.1‾\underline{60.1} / 0.167843.9‾\underline{43.9} / 0.189023.6 / 0.1858‾\underline{0.1858}89.1 / 0.1300‾\underline{0.1300}
MEF40.7‾\underline{40.7} / 0.136649.8 / 0.1450‾\underline{0.1450}28.1 / 0.1289‾\underline{0.1289}24.6‾\underline{24.6} / 0.186789.6‾\underline{89.6} / 0.1334
Ours59.3 / 0.096989.2 / 0.095690.2 / 0.115532.2 / 0.136497.4 / 0.0763
Reviewer_J77BOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work explores a surrogate-free, query-based adversarial attack TALA for image forgery localization. TALA adaptively chooses the patch-center and sizes, and generates perturbations using two complementary engines. Extensive experiments on various victim models and datasets have been done to show TALA's performance compared to several query-based methods and one diffusion-based attack.

Critical strengths

The work conducted experiments on 5 datasets and 3 victim detectors and provided a detailed ablation table. Also, those additional supplementary results in the appendix provide more information if readers want to gain a deeper understanding.

Critical weaknesses
  1. Although the experiment covers a wide range of victim models and datasets, the evaluation sample size is relatively small, with at most 50 or 100 images per detector–dataset pair. This may limit the statistical reliability of the results.

  2. The claimed contributions of TALA mainly lie in the adaptation for the image forgery localization task, including sampling the patch center, determining patch sizes, and choosing the perturbation types. However, the ablation results suggest that the distribution-adaptive patch size helps only LPIPS and PSNR, but hurts attack performance and other metrics (especially by increasing query counts). Is this module really necessary? Apart from this, there are many designs for complementary engines without explaining why they are designed that. For example, why does Engine 0 have a fixed Rademacher-perturbed reference image with a bicubically upsampled low-resolution Rademacher, instead of only a bicubically upsampled low-resolution Rademacher? Also, the work manually sets several important parameters, such as the sequence of spatial evidence during center searching and the engine-switching threshold K, and so on. There could be a hyperparameter sensitivity study to show why these values are chosen and clarify how robust the method is. Overall, the contribution looks a bit heuristic.

  3. The motivation of this work is the limitations in cross-detector transferability, but PGN (transfer-based attack) achieves fairly strong transfer results in some settings, even without querying the victim model. It would help to clarify the contribution if adding one or two recent transfer-based baselines (at least PGN) to the main results table. Transfer-based methods and TALA should be evaluated under same perturbation budgets, sample-selection criteria, and metrics.

Questions for authors
  1. Do the five random seeds correspond to repeated attacks on the same evaluation images? It would be helpful to report the exact number of evaluated images for each setting and, if possible, include results on larger subsets.

  2. The paper motivates TALA by the limitations of transfer-based attacks, whose transferability analysis mainly relies on PGN published in 2023. Moreover, while there is no comparison in the main results, PGN results in Appendix are evaluated under different perturbation budget, evaluation images and even metrics. Have you analyzed more recent and stronger transfer-based attacks?To better support your contribution, can you include more comprehensive comparisons with transfer-based attacks under consistent experimental settings?

13 comments
AuthorsOpenReview ↗

Response to Reviewer J77B (Part 1/7)

We truly appreciate your careful and constructive review, and the detailed questions that helped us clarify and strengthen the paper. We have taken every point seriously and, where helpful, added new experiments; below we respond to each of your weaknesses and questions point by point.

Please let us know if any concerns remain, and we would be glad to discuss them further. A revised PDF has been uploaded; it includes the additional results reported here as well as several clarifications in the text, all highlighted in blue.

W1: Larger-sample evaluation and statistical reliability (Table R2-2).

Although the experiment covers a wide range of victim models and datasets, the evaluation sample size is relatively small, with at most 50 or 100 images per detector–dataset pair. This may limit the statistical reliability of the results.

We agree that sample size affects statistical reliability. We therefore enlarged the evaluation to up to 1000 eligible images per cell (Table R2-1). On these larger samples, TALA still achieves both higher ASR and lower LPIPS than the runner-up baseline in all 15 detector–dataset settings (Table R2-2), and its advantage holds consistently as the sample size grows.

We note that DiffCFG and AdvFor occasionally reach a lower LPIPS than TALA, but only because their ASR is far lower, and because TALA’s operating point here is chosen against the ASR runner-up rather than against them. Beyond these tabulated budgets, the ASR-vs-LPIPS scatter plots in Figs. 3 and 5 (evaluated on 50 images per setting) show that TALA’s points lie to the upper-left of all four black-box baselines across the detector–dataset settings, attaining a higher ASR at a lower LPIPS, with only occasional local points where its ASR is marginally lower. They also include operating points where TALA reaches a lower LPIPS and higher PSNR than DiffCFG and AdvFor while still surpassing them in ASR.

Table R2-1. Number of eligible images (initial localization F1 >> 0.5, capped at 1000) per victim–dataset setting; the last row gives each dataset's total size.

VictimCASIAv1COCOGlideCOVERAGEIMD2020FaceApp
FOCAL727213821000193
TruFor68919655423316
IFOSN499144191000616
Dataset size92051210020104501

Note: TruFor–IMD2020 is limited to 423 images because RLGC ran out of memory on large inputs and time constraints prevented reruns; all other settings use their full eligible sets.

Table R2-2. Large-scale evaluation (all F1 >> 0.5 images, up to 1000 per setting; sample sizes in Table R2-1). Each entry is ASR (%) ↑\uparrow / LPIPS ↓\downarrow.

VictimMethodCASIAv1COCOGlideCOVERAGEIMD2020FaceApp
FOCALDiffCFG13.1 / 0.115‾\underline{0.115}22.5 / 0.117‾\underline{0.117}18.3 / 0.116‾\underline{0.116}5.8 / 0.15816.1 / 0.154
AdvFor11.1 / 0.15130.0 / 0.1846.1 / 0.2452.9 / 0.17936.3 / 0.165
RLGC18.4 / 0.19849.3 / 0.23219.5 / 0.3668.9 / 0.41549.2 / 0.217
Square30.0‾\underline{30.0} / 0.12659.6‾\underline{59.6} / 0.12145.4‾\underline{45.4} / 0.12029.8‾\underline{29.8} / 0.13989.6‾\underline{89.6} / 0.104‾\underline{0.104}
Ours59.3 / 0.09789.2 / 0.09690.2 / 0.11632.2 / 0.13697.4 / 0.076
TruForDiffCFG11.1 / 0.11216.2 / 0.117‾\underline{0.117}21.1 / 0.11415.2 / 0.1612.8 / 0.158‾\underline{0.158}
AdvFor28.9 / 0.12630.6 / 0.22316.4 / 0.1783.5 / 0.189‾\underline{0.189}32.4 / 0.190
RLGC26.7 / 0.18270.9‾\underline{70.9} / 0.28938.2 / 0.32233.1‾\underline{33.1} / 0.42770.8 / 0.330
Square33.7‾\underline{33.7} / 0.097‾\underline{0.097}43.4 / 0.17047.3‾\underline{47.3} / 0.13725.3 / 0.21087.7‾\underline{87.7} / 0.182
Ours61.4 / 0.03596.4 / 0.09785.5 / 0.127‾\underline{0.127}74.5 / 0.19198.1 / 0.149
IFOSNDiffCFG11.2 / 0.1052.8 / 0.131‾\underline{0.131}10.5 / 0.1013.7 / 0.1410.8 / 0.177‾\underline{0.177}
AdvFor9.8 / 0.16846.5 / 0.18836.8 / 0.1573.0 / 0.18724.5 / 0.175
RLGC35.7‾\underline{35.7} / 0.29767.4‾\underline{67.4} / 0.29789.5‾\underline{89.5} / 0.30950.8‾\underline{50.8} / 0.42326.8 / 0.419
Square30.1 / 0.18561.8 / 0.22631.6 / 0.092‾\underline{0.092}8.0 / 0.22569.8‾\underline{69.8} / 0.269
Ours82.4 / 0.155‾\underline{0.155}95.8 / 0.107100.0 / 0.07958.9 / 0.179‾\underline{0.179}93.7 / 0.228
AuthorsOpenReview ↗

Response to Reviewer J77B (Part 2/7)

W2: Design motivation, necessity, and hyperparameter robustness

The claimed contributions of TALA mainly lie in the adaptation for the image forgery localization task, including sampling the patch center, determining patch sizes, and choosing the perturbation types. However, the ablation results suggest that the distribution-adaptive patch size helps only LPIPS and PSNR, but hurts attack performance and other metrics (especially by increasing query counts). Is this module really necessary? Apart from this, there are many designs for complementary engines without explaining why they are designed that. For example, why does Engine 0 have a fixed Rademacher-perturbed reference image with a bicubically upsampled low-resolution Rademacher, instead of only a bicubically upsampled low-resolution Rademacher? Also, the work manually sets several important parameters, such as the sequence of spatial evidence during center searching and the engine-switching threshold K, and so on. There could be a hyperparameter sensitivity study to show why these values are chosen and clarify how robust the method is. Overall, the contribution looks a bit heuristic.

We first give the overall picture before the per-point details. The design rules and hyperparameter values you raise were chosen a priori from preliminary experiments on a subset of settings, then fixed and reused in every experiment—we did not tune them per setting. The ablation we now add (all 15 detector–dataset settings) demonstrates that the defaults are robust—no setting collapses—and sit at a strong overall ASR–quality trade-off among the alternatives we tried. The configuration is therefore a pre-committed design, not per-setting tuning and not a budget picked to flatter the results. We detail each choice below.

W2.1: Necessity of the distribution-adaptive patch size. This module is necessary. As discussed in Sec. 5.3, ASR increases as the patch-size interval grows, so we generally adopt a larger patch-size interval. At this large patch-size, the adaptive patch size incurs only a negligible ASR cost (0.1%) compared with the Linear one, yet it yields a substantial perceptual-quality gain—PSNR improves by 3.5 dB and LPIPS decreases by 0.0447. Trading a 0.1% drop in ASR for such a large reduction in perceptual distortion is clearly worthwhile, which is exactly the high-success / low-distortion trade-off our method is designed for.

AuthorsOpenReview ↗

Response to Reviewer J77B (Part 3/7)

W2.2: On the design choices (Rademacher Engine 0, spatial-evidence order, stage-switch thresholds, and engine-switching threshold K).

W2.2.1: The Rademacher reference in Engine 0 (Table R2-3).

Engine 0 mixes two kinds of perturbation, controlled by α\alpha in Eq. (21): a fixed, full-resolution Rademacher-perturbed reference xB\mathbf{x}^B, and a bicubically upsampled low-resolution Rademacher δsmooth\mathbf{\delta}^{\mathrm{smooth}}. The motivation is a simple balance: the δsmooth\mathbf{\delta}^{\mathrm{smooth}} is spatially smooth and barely hurts image quality, while the pure full-resolution Rademacher is sharper and perturbs the detector more strongly. Mixing them lets α\alpha trade a stronger attack (sharp Rademacher) against better perceptual quality (smooth Rademacher):

x^t,Rt(0)=ΠRt(α xRtB+(1−α) δtsmooth).\hat{\mathbf{x}}^{(0)}_{t,\mathcal{R}_t} = \Pi_{\mathcal{R}_t}\left(\alpha\,\mathbf{x}^{B}_{\mathcal{R}_t} + (1-\alpha)\,\delta^{\mathrm{smooth}}_t\right).

The reviewer's suggested variant—using only the upsampled low-resolution Rademacher—is the α=0\alpha = 0 case. Averaged over 3 victim models ×\times 5 datasets (Table R2-3), α=0\alpha = 0 lowers ASR by 5.1 points relative to our α=0.8\alpha = 0.8, while LPIPS and PSNR stay comparable, so we keep the mixed design.

More generally, α\alpha trades attack success against distortion: a small α\alpha sacrifices ASR, while α=1\alpha = 1 (pure Rademacher) maximizes ASR but worsens both LPIPS and PSNR. α=0.8\alpha = 0.8 keeps ASR high while retaining low distortion, giving the highest Balance score (0.66), so we set α=0.8\alpha = 0.8.

Balance score: to summarize the three-way trade-off with a single number, we min–max normalize ASR, LPIPS, and PSNR to [0,1][0,1] (higher is better) and report their equally-weighted mean.

Table R2-3. Effect of the mixing coefficient α in Engine 0 (mean ± std over 3 victim models × 5 datasets). α = 0 is the reviewer's upsampled-Rademacher-only variant; α = 1 is pure Rademacher.

αASR (%) ↑LPIPS ↓PSNR ↑Balance ↑
0 (only upsampled Rademacher)64.74 ± 0.200.0433 ± 0.011436.12 ± 0.620.58
0.249.78 ± 0.210.0229 ± 0.008533.60 ± 0.800.31
0.553.52 ± 0.210.0213 ± 0.008834.17 ± 0.880.46
0.8 (mixed Rademacher, ours)69.80 ± 0.200.0350 ± 0.010335.39 ± 0.780.66
1.074.13 ± 0.200.0462 ± 0.009934.05 ± 0.520.39

W2.2.2: Spatial-evidence order (Table R2-4). The main text motivates the order by attack strategy; here we confirm it also balances perceptual cost. All three states share the same per-pixel budget ϵ\epsilon, so image quality is set by the perturbed area, i.e., by how concentrated each state's sampling is.

Median and Max both confine perturbations to the true-positive region and incur nearly the same cost (single-state PSNR 32.61 vs. 32.84 dB), whereas Boundary targets p≈0.5p \approx 0.5 pixels across the entire image (details in Eq. (11))—the most diffuse and most costly pattern (28.90 dB, LPIPS 0.162). Our default therefore uses the two localized, low-distortion states first and defers the diffuse Boundary search to a last resort.

Among the three strong-balance orders (Balance ≥0.68\geq 0.68), our default attains the highest ASR (86.1%) at 31.11 dB PSNR; the orders that reach a marginally higher ASR (e.g., Boundary→\toMax→\toMedian, 86.8%) invoke Boundary early and fall to 29.54 dB PSNR and 0.47 Balance. Performance is also not brittle to the order: the top three orders differ by ≤0.02\leq 0.02 in Balance, and both leading alternatives likewise begin with a localized state rather than Boundary.

Table R2-4. Ablation on the spatial-evidence order under Q=50Q{=}50 (mean ±\pm std over the 15 detector--dataset settings), sorted by Balance score.

Spatial-evidence orderASR (%) ↑LPIPS ↓PSNR (dB) ↑Balance ↑
Strong-balance orders (Balance ≥ 0.68)
max → boundary → median84.5 ± 11.70.102 ± 0.03831.53 ± 1.500.70
max only80.4 ± 15.20.084 ± 0.03632.84 ± 2.130.70
median → max → boundary (Ours)86.1 ± 11.00.118 ± 0.04431.11 ± 2.190.68
Remaining orders
median only79.7 ± 15.00.087 ± 0.03832.61 ± 2.110.63
median → boundary → max81.9 ± 14.50.103 ± 0.03531.48 ± 1.440.57
boundary → max → median86.8 ± 10.10.142 ± 0.05229.54 ± 1.330.47
max → median → boundary82.4 ± 12.70.122 ± 0.05030.87 ± 2.060.46
boundary → median → max84.0 ± 13.00.140 ± 0.05229.67 ± 1.280.36
boundary only85.9 ± 12.90.162 ± 0.05928.90 ± 1.330.29
Reviewer_EpdNOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TALA, a query-based attack against image forgery localization models that requires no surrogate detector or attack-policy training. It uses the victim’s probability maps to choose where to perturb, adjusts patch sizes, and switches between two perturbation types based on unsuccessful attempts. Experiments cover three detectors and five datasets, with comparisons against existing attacks and common image degradations.

Critical strengths
  1. The method makes effective use of spatial feedback from localization models without requiring a surrogate or additional training

  2. The improvements are substantial in several settings and extend across different detectors and manipulation types. For example, on FOCAL-COVERAGE, TALA raises attack success from 28.0% for the strongest baseline to 91.2% under a 50-query budget. Although some comparisons are much closer, the overall results provide convincing evidence that the approach is effective

  3. The ablations provide useful support for the design choices. Guided patch selection improves attack success over uniform sampling, while adaptive patch sizing reduces image distortion with little loss in success rate. The experiments also show that the two perturbation types succeed on partially different images, supporting their use within a single search

  4. The evaluation goes beyond comparisons with existing attacks. The authors test 11 global image degradations, including compression, filtering, noise, resizing, and diffusion reconstruction

Critical weaknesses
  1. Perturbation budgets are selected separately for different experimental settings using a loosely specified balance between attack success and image quality. The paper does not describe a separate validation procedure for this selection. The trade-off curves help, but cover only part of the evaluation

  2. LPIPS and PSNR are averaged only over successful attacks. Since different methods succeed on different images, these averages do not directly establish which method introduces less distortion on the same inputs. Comparisons on shared successful examples, or success rates under fixed distortion limits, would be more convincing

  3. The evaluation uses at most 50 eligible images per detector-dataset pair under the initial F1 > 0.5 setting. Repeating the attacks with five seeds measures search variability, but does not establish how well the results generalize to a larger image sample

Questions for authors

I have no questions beside the points stated in the weaknesses section

6 comments
AuthorsOpenReview ↗

Response to Reviewer EpdN (Part 1/3)

We sincerely thank you for the positive assessment of our method design and experimental results. Below we address your raised weaknesses point by point.

If anything remains unclear, we would be glad to address it in the discussion. We have uploaded a revised PDF containing the new experiments and the wording adjustments described below, with all changes highlighted in blue.

W1: Perturbation budget selection and the ASR–quality trade-off.

Perturbation budgets are selected separately for different experimental settings using a loosely specified balance between attack success and image quality. The paper does not describe a separate validation procedure for this selection. The trade-off curves help, but cover only part of the evaluation.

1. Full coverage. The ASR–LPIPS curves for all 5 datasets × 3 victim detectors are now added as Fig. 3 (CASIAv1) and Appendix Fig. 5 (the remaining four datasets). The conclusion is unchanged: across datasets and victims, TALA simultaneously achieves higher ASR and lower LPIPS than the other black-box attacks at almost all LPIPS values.

2. The selection procedure is described in Appendix C.2. In more detail:

  • Baselines whose original algorithm has no adjustable budget (RLGC, DiffCFG, AdvFor) keep their default configuration and appear as a single point in Figs. 3 and 5.
  • Methods with a tunable budget — Square (its ℓ2\ell_2 range) and our TALA (its ℓ∞\ell_\infty range) — are line-searched over the budget to obtain full ASR–LPIPS curves (Figs. 3 and 5).

3. How the Table 1 operating point is chosen. The Table 1 point is simply selected from the curves in Figs. 3 or 5 by a consistent rule:

  • (i) we take a TALA point that beats the strongest competitor (the ASR runner-up, typically Square or RLGC) on both axes — lower LPIPS and higher ASR;
  • (ii) for the adjustable baseline Square we deliberately pick a point with slightly larger LPIPS than TALA (i.e., we grant Square more budget), so it is not understated;
  • (iii) where possible, we also keep TALA's LPIPS below the non-adjustable low-distortion baselines (AdvFor, DiffCFG).

In the few cases where (i) and (iii) do not hold at the same operating point, Appendix Table 20 reports a comparison in which TALA’s LPIPS is strictly lower than both AdvFor’s and DiffCFG’s while its ASR remains far higher. The Table 1 comparison is therefore favorable to the baselines, not to us, and the operating point is a deterministic selection from the curve — not a tuned, loosely specified balance.

4. On the absence of a separate validation procedure. The budget is a single parameter — the norm bound — not a learned hyperparameter, so it needs no held-out validation set: the full ASR-LPIPS curve is reported directly in Figs. 3 and 5, and Table 1 just takes one LPIPS-matched point from it. The two views are complementary — Figs. 3 and 5 show how ASR and LPIPS change as the perturbation budget is varied, with one run per budget, while Table 1 repeats the selected point over five random seeds (mean ± std) for a statistically reliable comparison.

AuthorsOpenReview ↗

Response to Reviewer EpdN (Part 2/3)

W2: Fair distortion comparison — shared successes and fixed-distortion success rates.

1. Shared successful examples (Table R1-2).

We thank you for this suggestion. We now report distortion on the images that all three methods — the two strongest baselines (Square, RLGC) and TALA — successfully attack. We exclude AdvFor and DiffCFG here, as their low success rates would leave too few shared images for a meaningful comparison. For each dataset we consider up to 1000 images with initial localization F1 >0.5>0.5 (eligible counts in Table R1-1).

Table R1-1. Number of eligible images (initial localization F1 >> 0.5, capped at 1000) per victim–dataset setting; the last row gives each dataset's total size.

VictimCASIAv1COCOGlideCOVERAGEIMD2020FaceApp
FOCAL727213821000193
TruFor68919655423316
IFOSN499144191000616
Dataset size92051210020104501

Note: TruFor–IMD2020 is limited to 423 images because RLGC ran out of memory on large inputs and time constraints prevented reruns; all other settings use their full eligible sets.

On these identical inputs, TALA attains the lowest LPIPS in 14 of 15 detector–dataset cells. For PSNR — a global ℓ2\ell_2 measure — TALA is highest in 9/15 cells, and in the remaining cells it is very close to the best PSNR meanwhile achieving much lower LPIPS and significantly higher ASR.

Table R1-2. Shared successful examples (count in the Shared N row): ASR (%) ↑\uparrow / LPIPS ↓\downarrow / PSNR ↑\uparrow, where ASR is the overall success rate on the eligible set.

VictimMethodCASIAv1COCOGlideCOVERAGEIMD2020FaceApp
FOCALShared N9574104191
RLGC18.4 / 0.182 / 34.149.3 / 0.203 / 33.119.5 / 0.321 / 31.38.9 / 0.360 / 32.449.2 / 0.210 / 36.7
Square30.0 / 0.121 / 31.959.6 / 0.120 / 33.545.4 / 0.154 / 27.829.8 / 0.153 / 25.189.6 / 0.146 / 34.6
Ours59.3 / 0.054 / 33.089.2 / 0.056 / 35.190.2 / 0.047 / 34.132.2 / 0.096 / 34.497.4 / 0.063 / 32.5
TruForShared N116691542169
RLGC26.7 / 0.145 / 37.470.9 / 0.219 / 33.838.2 / 0.299 / 32.433.1 / 0.392 / 30.870.8 / 0.351 / 29.5
Square33.7 / 0.095 / 35.043.4 / 0.173 / 28.747.3 / 0.147 / 25.625.3 / 0.193 / 28.587.7 / 0.174 / 30.5
Ours61.4 / 0.021 / 37.696.4 / 0.064 / 33.585.5 / 0.085 / 33.674.5 / 0.177 / 29.198.1 / 0.143 / 28.6
IFOSNShared N592865439
RLGC35.7 / 0.284 / 27.067.4 / 0.248 / 29.089.5 / 0.231 / 32.450.8 / 0.298 / 33.626.8 / 0.382 / 27.1
Square30.1 / 0.129 / 28.061.8 / 0.139 / 26.931.6 / 0.092 / 29.98.0 / 0.229 / 23.469.8 / 0.163 / 28.0
Ours82.4 / 0.098 / 30.595.8 / 0.064 / 32.4100.0 / 0.055 / 33.458.9 / 0.098 / 33.793.7 / 0.175 / 27.3

2. Success rates under fixed distortion limits (Table R1-3).

We further compare TALA against the four baselines (Square, RLGC, AdvFor, DiffCFG) by attack success rate under fixed distortion limits. TALA keeps a large and consistent margin at every threshold: its average lead stays around +28 points for limits from 0.08 to 0.25 and remains +20 even at the very tight 0.05. So our advantage holds consistently across the whole range of thresholds we examined, rather than resting on a particular choice.

Table R1-3. Average success rate over the 15 victim–dataset settings under fixed distortion limits (LPIPS ≤t\le t), computed on the large-scale evaluation (up to 1000 images per setting, Table R1-1), Q=50Q=50, initial F1 >> 0.5. "Best baseline" is the strongest of Square, RLGC, AdvFor, DiffCFG in each setting, averaged; Δ = Ours − best baseline.

Distortion limit (LPIPS)OursSquareRLGCAdvForDiffCFGBest baselineΔ (Ours − best)
≤ 0.0527.2%3.0%5.2%1.1%0.2%6.7%+20.5
≤ 0.0837.5%8.1%6.7%2.6%1.1%11.2%+26.3
≤ 0.1042.3%12.1%7.9%4.4%3.1%14.9%+27.4
≤ 0.1246.5%17.3%8.7%5.8%5.8%18.1%+28.4
≤ 0.1553.4%25.5%10.2%9.0%8.9%25.5%+27.9
≤ 0.2062.5%33.8%13.1%13.6%10.9%34.1%+28.4
≤ 0.2569.3%38.0%16.2%17.6%11.3%38.6%+30.7
AuthorsOpenReview ↗

Response to Reviewer EpdN (Part 3/3)

W3: Generalization to a larger image sample (Table R1-4).

The evaluation uses at most 50 eligible images per detector–dataset pair under the initial F1 > 0.5 setting. Repeating the attacks with five seeds measures search variability, but does not establish how well the results generalize to a larger image sample.

As shown in Table R1-1, we re-evaluated on a much larger sample — all images with initial localization F1 >0.5>0.5, up to 1000 per detector–dataset pair (vs. 50/100 before). On this order-of-magnitude larger sample, TALA attains both the highest ASR and a lower LPIPS than the ASR runner-up in all 15 detector–dataset settings, confirming that the conclusions generalize well beyond the 50-image setting rather than being a small-sample artifact.

Table R1-4. Large-scale evaluation (all F1 >> 0.5 images, up to 1000 per setting; sample sizes in Table R1-1). Each entry is ASR (%) ↑\uparrow / LPIPS ↓\downarrow.

VictimMethodCASIAv1COCOGlideCOVERAGEIMD2020FaceApp
FOCALDiffCFG13.1 / 0.115‾\underline{0.115}22.5 / 0.117‾\underline{0.117}18.3 / 0.116‾\underline{0.116}5.8 / 0.15816.1 / 0.154
AdvFor11.1 / 0.15130.0 / 0.1846.1 / 0.2452.9 / 0.17936.3 / 0.165
RLGC18.4 / 0.19849.3 / 0.23219.5 / 0.3668.9 / 0.41549.2 / 0.217
Square30.0‾\underline{30.0} / 0.12659.6‾\underline{59.6} / 0.12145.4‾\underline{45.4} / 0.12029.8‾\underline{29.8} / 0.13989.6‾\underline{89.6} / 0.104‾\underline{0.104}
Ours59.3 / 0.09789.2 / 0.09690.2 / 0.11632.2 / 0.13697.4 / 0.076
TruForDiffCFG11.1 / 0.11216.2 / 0.117‾\underline{0.117}21.1 / 0.11415.2 / 0.1612.8 / 0.158‾\underline{0.158}
AdvFor28.9 / 0.12630.6 / 0.22316.4 / 0.1783.5 / 0.189‾\underline{0.189}32.4 / 0.190
RLGC26.7 / 0.18270.9‾\underline{70.9} / 0.28938.2 / 0.32233.1‾\underline{33.1} / 0.42770.8 / 0.330
Square33.7‾\underline{33.7} / 0.097‾\underline{0.097}43.4 / 0.17047.3‾\underline{47.3} / 0.13725.3 / 0.21087.7‾\underline{87.7} / 0.182
Ours61.4 / 0.03596.4 / 0.09785.5 / 0.127‾\underline{0.127}74.5 / 0.19198.1 / 0.149
IFOSNDiffCFG11.2 / 0.1052.8 / 0.131‾\underline{0.131}10.5 / 0.1013.7 / 0.1410.8 / 0.177‾\underline{0.177}
AdvFor9.8 / 0.16846.5 / 0.18836.8 / 0.1573.0 / 0.18724.5 / 0.175
RLGC35.7‾\underline{35.7} / 0.29767.4‾\underline{67.4} / 0.29789.5‾\underline{89.5} / 0.30950.8‾\underline{50.8} / 0.42326.8 / 0.419
Square30.1 / 0.18561.8 / 0.22631.6 / 0.092‾\underline{0.092}8.0 / 0.22569.8‾\underline{69.8} / 0.269
Ours82.4 / 0.155‾\underline{0.155}95.8 / 0.107100.0 / 0.07958.9 / 0.179‾\underline{0.179}93.7 / 0.228

Comments on the paper

5 comments

Thanks for the detailed response and the additional experiments. The comparison at fixed LPIPS thresholds goes a long way toward addressing my concern about perturbation budgets. The ablations also show that using the detector’s spatial feedback helps guide the search.

What I’m still less convinced about is how much the specific search order and engine-switching rule contribute. Some simpler search choices give a similar trade-off, and the two-engine results show that the perturbations work well together but leave the benefit of the switching rule itself less clear. The new results make me more confident that TALA works well, but my main concern about the size of the methodological contribution remains. I’ll keep my original score.

Thank you for engaging so carefully with our responses, and for acknowledging the additional experiments.

  1. On the engine-switching rule, we would like to point to a result that directly isolates its benefit, which may be easy to miss as it sits in Appendix C.6 and Table 7 (Table is copied below). On FOCAL–CASIAv1, E0 and E1 individually reach 66.1% and 70.3% ASR. Taking the union of their success sets from two independent runs reaches 75.4%, but this is an oracle statistic requiring the queries of two full attack runs. Our adaptive switching, within a single trajectory and at half that query cost, reaches 80.2% — above the oracle union. The gain therefore does not come from merely having two perturbation types available, but from switching between them according to rejection feedback.

Table. Diagnostic analysis of the dual-engine interaction on FOCAL–CASIAv1 under approximately matched PSNR.

StrategyASR (%) ↑Unique Success (%)
E0 only66.15.1
E1 only70.39.3
Independent-run union75.4–
Adaptive E0+E180.2–
  1. On the search order, we agree the Balance scores of several variants are close. We would note that Balance weights ASR, LPIPS, and PSNR equally, which by construction favors conservative, low-distortion variants. In ASR, the primary attack objective, the gap is larger: our order reaches 86.1% versus 80.4% for Max only (the Top-1 Balance score), and no single-state variant attains both high ASR and low distortion. Therefore, we believe the specific search order contributes a balance that favors attack success while keeping distortion moderate.

We appreciate your time and your constructive engagement throughout the discussion.

Thanks for pointing me to the comparison in Appendix C.6. I still think it’s hard to separate the benefit of the specific switching rule from that of using both engines in a single run, but the fact that adaptive switching outperforms even the combined results of two independent runs makes the case stronger. The search-order comparison also gives me a clearer reason to prefer the proposed order when attack success is the priority. I still have some reservations about the size of the methodological contribution, but the additional evidence is enough for me to raise my score to 3.

One small suggestion for the revision: the method section would be easier to follow if it explained the intuition behind the search order and engine switching before getting into the details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.