acceptodds
Under review as a conference paper at ICLR 2027

Targeted Transfer Attacks on MLLMs: What a Spectral Constraint Buys

Abstract

Targeted adversarial images crafted on open surrogate encoders can transfer to multimodal large language models (MLLMs) that the attacker never queries. The strongest such attacks spend the full budget, which leaves open how much perturbation transfer requires and what spending less buys. We study this question through spectral constraints on the perturbation. We insert three rules into one surrogate-only attack skeleton: hard rank truncation, uniform singular-value shrinkage and a gradient-informed threshold. We evaluate them in two regimes. In a high-transfer regime we read held-out encoder success, perceptual distortion and transfer to six generative MLLMs. In a stealth regime at a smaller step we read detectability under a residual-based detector calibrated without any constrained attack. The results separate the skeleton from the rule. Across five attack objectives, the skeleton with no rule reproduces the success gain of the complete configuration over its released baseline, so the rules we ran do not buy success. What a rule shifts is the trade-off at comparable success. In the stealth regime the soft rules lower LPIPS by about one third and reduce detector flagging from near saturation to roughly –, and the gradient-informed rule still reaches four open-weight MLLMs with every paired difference from the full-budget baseline covering . Controls show that neither scaling a perturbation down to the rule's amplitude nor lowering its rank alone reproduces this, and recalibrating the detector on reduced-amplitude copies of the attack it already knows recovers most of the detection, so the evasion depends on detector coverage. Detection and generative transfer are read on different victims. Spectral constraints therefore do not make transfer attacks more successful by themselves. They make transferable perturbations more efficient and less detectable under the studied detector coverage. [Code](https://anonymous.4open.science/r/TASS-0C8E)

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multimodal Large Language Models, adversarial attack, Spectral Constraints

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_SqyoOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the role of spectral constraints in targeted transfer attacks against multimodal large language models. The authors incorporate three spectral shrinkage strategies—Hard Spectral Shrinkage, Uniform Spectral Shrinkage, and Target-Aware Spectral Shrinkage—into an existing surrogate-based attack framework, and evaluate their effects on attack success rate, perceptual distortion, and detectability.

The experimental results show that the improvement in transfer attack success achieved by the complete spectral configurations over the original attack configurations is primarily attributable to changes in the optimization framework rather than the spectral constraints themselves. Under specific experimental conditions, spectral shrinkage can reduce the perceptual distortion of adversarial perturbations and significantly lower the detection rates of a particular detector. The authors further investigate the factors influencing detection performance through amplitude-matched controls, effective-rank controls, and analyses of detector training coverage.

Critical strengths

S1. Relatively Comprehensive Ablation Studies.

The authors compare attack performance with and without spectral constraints under the same optimization framework, thereby distinguishing the effects of the optimization configuration from those of the spectral operations. The experimental results show that the improvement in transfer success rates primarily comes from changes in the optimization configuration rather than from HSS, USS, or TASS themselves. This is a valuable negative result that also helps avoid incorrectly attributing performance improvements to the spectral operations.

S2. Evaluation Beyond Attack Success Rates.

The paper evaluates transfer success rates, LPIPS, RMS, effective rank, and detection rates, providing a more comprehensive understanding of the trade-offs among different evaluation metrics in adversarial attacks.

S3. Meaningful Control Experiments.

The amplitude-matched experiments, effective-rank controls, and analyses of detector training coverage provide useful empirical evidence. In particular, the analysis of detector training coverage demonstrates that low detection rates do not necessarily imply that adversarial attacks are universally undetectable.

Critical weaknesses

W1. Lack of a Clear Connection Between Spectral Constraints and Targeted Transfer Attacks, with Insufficient Motivation

The paper focuses on targeted transfer attacks against MLLMs, but the spectral constraint operations employed do not adequately reflect the specific characteristics of this task.

Specifically, HSS and USS operate solely on the singular values of the perturbation matrix, without explicitly considering the target class or the relationship between surrogate and victim models. Although TASS incorporates gradients of the attack loss computed on surrogate models, it remains a generic gradient-projection-based weighted shrinkage strategy that does not explicitly model transferable attack directions shared across models or target-specific semantic features.

The paper does not first investigate the singular-value spectral characteristics of transferable adversarial perturbations, nor does it establish a relationship between singular directions and target-class semantics, cross-model gradient alignment, or transfer success rates.

W2. The Main Experimental Findings Focus on Detectability, but the Mechanistic Connection Among Spectral Structure, Transferability, and Detectability Remains Unclear

The most significant experimental finding is that spectral shrinkage can substantially reduce the detection rate of a particular detector while maintaining comparable attack success rates.

However, the paper does not provide a sufficient mechanistic explanation for this phenomenon.

Through amplitude-matched experiments, the authors show that simply scaling down existing perturbations cannot reproduce the effects of spectral shrinkage. Similarly, effective-rank control experiments suggest that reducing effective rank alone is insufficient to explain the observed changes in detection rates.

Although these results rule out some simple explanations, several important questions remain unanswered:

  • Which singular values or singular directions are associated with cross-model transferability?
  • What spatial statistical properties of adversarial perturbations are actually altered by spectral shrinkage?
  • Why do these changes affect the detector's responses?
  • Do the spectral structures that reduce detectability also help preserve the information necessary for targeted transfer?

W3. Lack of Fair Comparisons with Existing Structured-Constrained Attack Methods, Leaving the Necessity of SVD Unsubstantiated

The spectral shrinkage operations employed in this paper are closely related to established techniques. HSS corresponds to truncated SVD, USS corresponds to standard singular-value soft-thresholding, and TASS is a gradient-weighted thresholding variant.

The main detection experiments compare the spectral configurations against the original Omni-Attack, a no-rule configuration, and an L2-constrained configuration. However, the L2 baseline uses a different parameterization and optimization setup from the spectral configurations, making it difficult to isolate the contribution of spectral constraints.

Furthermore, the amplitude-matched experiments mainly construct control perturbations by scaling down previously optimized unconstrained perturbations, rather than re-optimizing the attacks under smaller energy budgets.

In addition, the DCT-based compaction experiments in the appendix achieve detection performance comparable to singular-value-based compaction, further suggesting that SVD may not be necessary to obtain low detection rates.

Questions for authors

Q1. Research Motivation and Task Specificity

Why does this paper specifically focus on targeted transfer attacks against MLLMs? Beyond adopting an existing surrogate-based attack framework, is there any theoretical or empirical evidence suggesting a particular connection between singular-value spectral structures and targeted transferability?

Q2. Spectral Structure and Transferability

Have the authors investigated how different singular directions affect the targeted attack loss, gradient alignment across surrogate models, and transfer success rates on unseen models? If spectral constraints do not improve transfer success rates, can they at least preserve attack-relevant information shared across models more effectively?

Q3. The Unique Contribution of Spectral Constraints

Can the authors provide comparisons with re-optimized L2-constrained, DCT-based, low-pass, and other structured-constrained attacks under exactly the same optimization framework? If these alternatives achieve similar distortion and detection rates, how should the core contribution of spectral constraints in this paper be understood?

Q4. Detection Mechanism

How exactly does spectral shrinkage affect the residual statistics used by the detector? Are there more direct and informative statistical measures than RMS and effective rank that could explain the differences in detection rates across different constraint methods?

Q5. Joint Evaluation of Generative MLLM Transfer and Detectability

Can the authors report both targeted attack success rates on generative MLLMs and detector true-positive rates using the same attack configurations and image samples, to demonstrate that the main findings hold under a unified experimental setting?

Q6. Generalizability Across Detectors

If different detector architectures are used, or if the detector is trained on attack distributions with varying perturbation magnitudes, does spectral shrinkage still provide a detection advantage over other structured constraint methods?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Sqyo (Part 1/2)

We thank Reviewer Sqyo for the thoughtful comments. Our point-by-point responses follow.

W1 / Q1. The rules are generic and not tied to targeted transfer; why this task, and is there evidence that spectral structure relates to transferability?

Generic rules, and why this setting. The rules are generic, and this is by design. The paper uses them to measure what a spectral constraint changes in this task once its effect is separated from the optimisation configuration and from the detector; a rule tailored to the task would not answer that question. Targeted transfer to MLLMs is a setting where that question is open: the strongest attacks spend the full ℓ∞\ell_\infty budget, the attacker cannot query the victim, so input screening matters, and prior work on structured perturbations reports success and low distortion jointly.

Spectral structure and transferability. The paper answers this at two levels.

  • Rules. On five objectives, none of the three rules adds held-out transfer over the no-rule skeleton (Table 2).
  • Components. The relation exists at this level: the leading singular components carry most of the transfer (ablation below). They are also where the victims' gradients concentrate. The leading-32 subspace of every perturbation holds three to four times the gradient energy of a random subspace of the same rank, for the held-out encoders as much as for the surrogates (Table A10, an exploratory diagnostic). The target's semantics sit there too: the leading components alone still make the held-out encoders return the target.

W2 / Q2 / Q4. The mechanism is unclear: which singular components carry transfer, what the shrinkage changes in the perturbation, and why the detector responds differently.

The submitted Section 5.5 left this open. We ran two new analyses.

Component ablation. For each of the 50 scored images we rebuild the final perturbation from its top kk singular components only (keep kk), or from all the others (drop kk), and read success and detection again.

Perturbation, variantRMSTop-1TPR@5%
USS, full7.360.930.12
USS, keep 647.080.910.08
USS, drop 85.930.340.10
No rule, full10.460.920.94
No rule, keep 648.830.810.28
No rule, drop 645.530.000.56

(All kk and all four configurations: Table A9 of the revised paper.)

  • Transfer sits in the leading components. The top 64 components retain most of the success (USS 0.910.91 against 0.930.93), and removing the top 8 cuts USS from 0.930.93 to 0.340.34.
  • Much of the detector's signal sits in the trailing components. The no-rule perturbation without its top 64 components carries no transfer and has about half the RMS, yet it is still flagged on 0.560.56.

What the detector reads. Across the 57 configurations with a zero-day rate, the RMS of the perturbation after the detector's own residual filters tracks TPR@5% with Spearman ρ=0.945\rho = 0.945, against 0.8650.865 for plain RMS and 0.6700.670 for effective rank. The detector's rate therefore follows high-frequency residual energy. Shrinkage lowers that energy by removing trailing components and keeps the leading ones that carry success. The rules thus preserve the shared attack information at lower cost, not more of it: the same success at about a third less LPIPS (Table 3).

This is a descriptive account of final images, not a causal test of the optimisation. The revised Section 5.5 and App. D state it this way.

AuthorsOpenReview ↗

Response to Reviewer Sqyo (Part 2/2)

W3 / Q3. The rules are close to established techniques, the L2L_2 baseline is not in the same framework, and DCT compaction does as well as SVD; if alternatives perform similarly, what is the contribution?

Relation to established techniques. We agree, and Section 2 says so: singular-value thresholding "is exactly our uniform rule", and TASS is its weighted form with weights taken from the attack gradient. The paper's claims rest on the controlled comparison, not on the novelty of the operators.

L2L_2 in the same framework, DCT, and the role of SVD. We re-optimised both constraints inside the same skeleton at the rules' own step, on all 500 pairs. CNN is a second, learned detector trained under the same protocol.

Configuration (step 0.5, same skeleton)Top-1LPIPSTPR@5%, published detectorTPR@5%, CNN
USS0.8640.1860.0860.13
TASS0.8490.1830.0860.12
L2L_2 ball 7.330.8750.1750.0940.33
L2L_2 ball 8.20.8840.2040.1500.63
DCT low-pass R=160R{=}1600.7180.2270.0500.07
DCT low-pass R=192R{=}1920.8060.2380.0540.07

(Intervals and paired differences: Tables A6 to A8 and App. D of the revised paper.)

  • L2L_2 ball. Under the published detector the 7.33 ball matches USS in success and in detection, and the 8.2 ball is flagged more often. The learned detector flags the balls at the rules' energy about 2.52.5 to 55 times as often as the rules.
  • DCT low-pass. It is flagged no more often than the rules, but it loses 66 to 1515 points of success at a larger distortion.
  • SVD is not necessary. The basis controls of App. D already reach this conclusion, and the new controls confirm it.

How to understand the contribution. It is the attribution: success gains belong to the optimisation skeleton (Table 2), distortion and detection move with the constraint (Tables 3 and 4), and the detection rate depends on the detector's training coverage (Table A5). The revised paper states the detection finding for energy-reducing structured constraints in general, not for the singular decomposition.

Q5. Can generative-victim success and detector TPR be reported on the same configurations and images?

We now report it (Table A11 of the revised paper), on the 200 images sent to the generative victims. Each victim cell gives targeted ASR (%) / Joint, where Joint is the fraction of the 200 images that name the target and are not flagged at 5% FPR.

ConfigurationTPR@5%Qwen2.5Qwen3.8DeepSeekGLM-5.3Gemini-3.8GPT-5.6
Omni-Attack (unconstrained)0.98536.5 / 0.00043.5 / 0.00540.0 / 0.00035.0 / 0.00032.0 / 0.00026.5 / 0.000
No rule (step 2.5)0.91541.5 / 0.02542.0 / 0.02544.5 / 0.03041.5 / 0.03035.0 / 0.01531.0 / 0.020
HSS (step 2.5)0.28537.0 / 0.30047.5 / 0.33545.0 / 0.33540.0 / 0.30039.5 / 0.29529.0 / 0.205
USS (step 2.5)0.42042.0 / 0.25045.0 / 0.25041.5 / 0.24539.0 / 0.26037.0 / 0.21527.5 / 0.150
TASS (step 2.5)0.53542.5 / 0.20046.5 / 0.19547.5 / 0.22544.0 / 0.22035.0 / 0.16531.0 / 0.145
TASS (step 0.5)0.10032.0 / 0.29043.0 / 0.40041.0 / 0.38535.0 / 0.315not sentnot sent

ASR is similar across configurations; Joint differs because of detection. Every rule's Joint rate exceeds that of the unconstrained attack on every victim, with paired intervals excluding zero. The revised Section 5.6 reports this in place of the submitted statement that no setting was measured as both effective and unflagged on the same victims.

Q6. Does the detection advantage survive other detector architectures, or a detector trained on other perturbation magnitudes?

  • Magnitudes. Table A5 trains the detector, at the same budget of attacked images, on the known attack at reduced amplitude. Adding it at quarter amplitude raises detection of the compacted configuration from 0.260.26 to 0.880.88 at a nominal 10% FPR, at almost no cost on the original attack. Detection is recovered, and the revised Section 5.6 states this explicitly.
  • Architectures. A learned residual CNN trained under the same zero-day protocol (Table A6) reproduces the low rates of the soft rules (0.100.10 to 0.130.13) and the ordering among constraints. Both detectors read the luminance plane (Section 5.1), and the claim is made for that input.
  • Against other structured constraints. Under the learned detector the rules are flagged less often than the same-skeleton L2L_2 ball. DCT low-pass is flagged slightly less often than the rules but loses 66 to 1515 points of success (W3 / Q3).

Thank you for the detailed rebuttal and substantial additional experiments. I appreciate the authors' efforts and consider most of my concerns regarding experimental completeness to have been adequately addressed.

However, my core concern regarding scientific novelty remains unresolved. The new analyses do not establish a convincing, task-specific connection between SVD and targeted transferability. Similar observations hold for unconstrained attacks, and other structured constraints achieve comparable results. It therefore remains unclear what new scientific insight SVD provides in this setting.

Overall, while the experimental evidence is now comprehensive, my fundamental concerns about motivation, mechanistic understanding, and scientific contribution remain. This is why I am willing to raise my rating from 1 to 2 in recognition of the substantial improvements, but not to 3, as the central scientific concerns remain unresolved.

AuthorsOpenReview ↗

Response to Reviewer Sqyo

We thank Reviewer Sqyo for the follow-up, for raising the rating, and for recognising the added experiments.

In short, what SVD provides is not more transfer, but a way to make a transferable perturbation cheaper and harder to detect, together with an account of why this costs no success. The gain in success that accompanies such constraints belongs to the optimisation skeleton (Table 2).

We answer the three remaining points in turn, with two further analyses under the first.

1. Connection between SVD and targeted transfer. Beyond the component and gradient analyses of the revision, we ran two further analyses on the 50 scored images that ask what the leading singular directions encode. They are image-specific, not target-specific.

We re-attacked each image toward another image's target and compared the leading-k singular subspaces of pairs of perturbations (1 = identical):

Pair of perturbationsk=8k=16k=64
Same image, different target0.1670.2290.375
Different image, same target0.1390.2090.364
Different image, different target0.1400.2100.364
Chance0.0270.0540.214

Sharing the target adds nothing across images (−0.001 [−0.006, +0.003] at k=8), and a perturbation added to a different clean image reaches its target on 0 of 50 images.

The connection is therefore specific. Transfer is carried by a few leading components of each perturbation: removing the top 8 lowers success from 93 to 34 points. That is why the spectrum can be shrunk without losing it (Table A9 of the revised paper).

2. Unconstrained attacks. They show the same leading structure, and this is what makes it a property of targeted transfer and not of our rules. The difference is in the trailing components. Without its top 64, the no-rule perturbation has no success and is still flagged on 0.56 (Table A9). Shrinkage removes that part, and the flagged rate falls from 0.94 to 0.12 at the same success (Table 3).

3. Other structured constraints. Under the published detector an energy ball is comparable, and the revision says so. The constraints are not equivalent otherwise:

  • a DCT low-pass loses 6 to 15 points of success at the same low rate (Table A8 of the revised paper);
  • a learned detector flags the balls at the rules' energy on 0.33 to 0.63, against 0.12 to 0.13 for the soft rules, on 500 pairs (Tables A6 and A7 of the revised paper).

We will add the two analyses to App. D. We hope this makes the role of SVD clear.

Reviewer_ECFZOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper examines the actual role of spectral constraints in targeted transfer attacks against MLLMs. Within a unified attack pipeline that relies solely on surrogate models, the authors compare three spectral constraints: hard rank truncation, uniform singular-value shrinkage, and gradient-informed shrinkage. To disentangle the effects of the spectral constraints from those of the optimization settings, they further introduce no-rule baselines, amplitude-matched and energy-constrained controls, and detector recalibration. The evaluation spans five attack objectives, two datasets, and six generative MLLMs. According to the results, the gains in attack success stem mainly from changes to the optimization pipeline rather than from the spectral constraints themselves. In terms of stealthiness, soft shrinkage lowers LPIPS by about one third and, under the initial detector calibration, reduces the flagging rate to roughly 0.12–0.14. However, recalibrating the detector restores much of its detection capability, suggesting that this stealth advantage depends on the detector’s training coverage.

Critical strengths
  1. The experimental evaluation is extensive and supported by accessible code and records. It covers five attack objectives, two datasets, held-out encoders, and six generative MLLMs, with no-rule baselines, amplitude-matched and energy-constrained controls, and detector-calibration experiments. Paired confidence intervals, parameter analyses, and supplementary experiments strengthen the evidence. The anonymized code and per-image records also facilitate verification and follow-up research.

  2. The motivation is clear, and the experiments systematically address the central question. Starting from the observation that existing attacks typically exhaust the perturbation budget, the paper asks what reducing perturbation expenditure actually buys. It examines attack success, perceptual distortion, and detectability separately, progressively testing explanations involving optimization settings, perturbation amplitude, rank, and detector training coverage. This creates a clear connection between the research question, experimental design, and conclusions.

Critical weaknesses
  1. The detectability findings may not generalize. All detection experiments use the SRM–FLD detector family, leaving untested whether the low flagging rates of soft spectral constraints, and the ranking among constraints, hold for learned-residual or end-to-end detectors.
  2. It remains unclear whether spectral constraints offer a better transferability–detectability trade-off than constraining energy during optimization. Amplitude-matched controls only rescale perturbations post hoc, and the re-optimized L2 baseline differs in parameterization and step size; an energy-constrained baseline in the same direct-pixel Adam pipeline is missing.
Questions for authors
  1. Could the authors evaluate another detector family under the same training coverage, data splits, and target false-positive rate, to test whether the low flagging rates of soft spectral constraints and the ordering among constraints persist?
  2. Could the authors re-optimize an L2-constrained baseline within the same direct-pixel Adam pipeline and at the same step size as USS/TASS, then compare perceptual distortion and detection at similar realized RMS or attack success?
4 comments
AuthorsOpenReview ↗

Response to Reviewer ECFZ (Part 1/2)

We thank Reviewer ECFZ for the thoughtful comments. Our point-by-point responses follow.

W1 / Q1. All detection results use one detector family; do the low rates of the soft rules and the ordering among constraints hold for a learned detector under the same coverage, splits and target FPR?

The published detector is the rich-model steganalyser of Section 5.1 (SRM residual features, FLD ensemble), the family the reviewer refers to. We trained a second family under the training coverage, splits and target FPR the reviewer specifies, with thresholds set by the same clean-image calibration procedure: an end-to-end learned residual CNN (learned 5×55{\times}5 high-pass front end with a truncated linear unit, three convolutional blocks, global mean and standard-deviation pooling, linear head), trained from scratch per fold on the same clean pool and the unconstrained-attack versions of the out-of-fold ids, scored out of fold, thresholds from clean images only, scores averaged over three detector seeds. Nothing from any constrained configuration enters training. We ran it on the 50-image cohort of Table 3a and on all 500 pairs, with the same luminance input as the published detector.

configuration (step 0.5 unless stated)steganalyser, 500CNN, 500steganalyser, 50CNN, 50
Unconstrained (Omni-Attack, εtanh⁡\varepsilon\tanh, step 0.05)0.991.000.960.98
No rule (τ=0\tau{=}0)0.971.000.940.96
HSS (r=128r{=}128)0.660.930.840.70
USS (τ=0.5\tau{=}0.5)0.090.130.120.10
TASS (τ=1\tau{=}1)0.090.120.140.10
L2L_2 ball 6.4 / 7.33 / 8.20.06 / 0.09 / 0.150.16 / 0.33 / 0.630.08 / 0.24 / 0.480.08 / 0.12 / 0.22
Low-pass R=160R{=}160 / 1921920.05 / 0.050.07 / 0.070.06 / 0.060.06 / 0.06
Omni-Attack rules, step 2.5 (HSS / USS / TASS)0.32 / 0.41 / 0.560.63 / 0.76 / 0.850.64 / 0.78 / 0.820.48 / 0.58 / 0.70

(TPR at the 5% clean threshold; realised FPR of the CNN 0.030.03 to 0.060.06. The 50-image steganalyser entries for HSS and the step-2.5 rules are the paper's Table A2. The unconstrained configuration here is the 500-pair launch of App. D, flagged on 0.96, against 0.98 for the 100-image launch of Table 3a.) Both parts of the question have the same answer. The low flagging rates of the soft rules persist: USS and TASS stay at 0.100.10 to 0.130.13 under the learned detector. The ordering among constraints persists: no rule, then HSS at or above the step-2.5 rules, then USS and TASS, with the L2L_2 balls rising with energy. The learned detector is in fact the more discriminating one for the question of W2: at the rules' energy the same-skeleton L2L_2 ball is flagged on 0.330.33 to 0.630.63 against 0.120.12 to 0.130.13 for the rules, where the published detector separates them by a few hundredths. The revised paper adds this detector to App. D and reports both families in Table 3.

AuthorsOpenReview ↗

Response to Reviewer ECFZ (Part 2/2)

W2 / Q2. An energy-constrained baseline in the same direct-pixel Adam pipeline and at the same step as USS/TASS is missing, so the comparison with constraining energy is open.

The submitted App. D stated that this control was not run. We have now run it. The L2L_2 baseline uses the same direct-pixel Adam skeleton, surrogates, augmentations, step 0.50.5 and iteration count as USS/TASS, with the spectral rule removed; after each update the iterate is clipped to the ℓ∞\ell_\infty budget and then shrunk onto a per-image L2L_2 ball. We ran three radii: 7.337.33 and 8.28.2 bracket the rules' realised RMS, and 6.46.4 lies below it.

The table scores all 500 NIPS-2017 pairs under the frozen zero-day protocol, with both detectors calibrated on the unconstrained attack only. CNN is the learned detector of our response to W1/Q1. Rows are ordered by RMS, and brackets are 95% bootstrap intervals.

Configuration (step 0.5, same skeleton)RMSTop-1LPIPSTPR@5%, published detectorTPR@5%, CNN
No rule10.450.8830.2840.970 [0.95, 0.98]1.00
L2L_2 ball 8.27.680.8840.2040.150 [0.12, 0.18]0.63
USS7.400.8640.1860.086 [0.06, 0.11]0.13
TASS7.350.8490.1830.086 [0.06, 0.11]0.12
L2L_2 ball 7.336.810.8750.1750.094 [0.07, 0.12]0.33
L2L_2 ball 6.45.870.8380.1410.064 [0.04, 0.09]0.16

(Table A7 of the revised paper gives this table, its 50-image version and the paper's original L2L_2 control; paired differences are in App. D.)

Three things follow.

  1. An energy constraint alone lowers the detection rate. The rate falls monotonically with the ball's RMS, while the effective rank stays near 200200.
  2. Under the published detector, the rules do not dominate the balls. The 7.337.33 ball is not distinguished from USS in top-1 or in detection (paired difference −0.008-0.008 [−0.020,+0.002][-0.020, +0.002]) and has a lower LPIPS. The rules and the same-skeleton balls lie on one success-detection curve.
  3. The learned detector separates them. It flags the two bracketing balls on 0.330.33 and 0.630.63, against 0.130.13 and 0.120.12 for USS and TASS. A ball scales every singular component, while a rule removes the trailing ones, which carry most of the high-frequency residual that this detector reads.

The revised Section 5.4 and App. D report this reading. The statement that the energy control "is flagged more often than both" soft rules now holds for the paper's original control and, under the learned detector, for the same-skeleton ball. Under the published detector the margin over the same-skeleton ball is small.

Reviewer_ECFZOpenReview ↗

Respopnse to the authors

Thank you for your response. You have addressed my concerns, and I will maintain my recommendation to accept the paper.

AuthorsOpenReview ↗

Response to Reviewer ECFZ

We thank Reviewer ECFZ for taking the time to review our responses and additional experiments. We are glad that the evaluations with the learned residual CNN detector and the re-optimised same-skeleton L2L_2 baseline have addressed your concerns. Your suggestions have helped us clarify the empirical comparisons and strengthen the paper.

Reviewer_u17JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether spectral constraints improve targeted transfer attacks and whether any apparent reduction in detectability is attributable to the constraint itself. It builds a shared surrogate-only attack pipeline with hard spectral truncation, uniform spectral shrinkage, and gradient-informed shrinkage, then compares these rules against no-rule, amplitude-matched, rank-reduction, and energy-constrained controls. Experiments cover five attack objectives, 500 image-target pairs, two held-out encoders, six generative multimodal models, two supervised classifiers, an ImageNetV2 replication, and a second attack pipeline. The central result is that the spectral rules do not consistently improve attack success; they can reduce distortion and lower detection by a specific steganalysis detector under its stated calibration, but this advantage weakens when the detector sees related attacks or the input is resized. Overall, the controlled evaluation is useful, but the attack-success evidence is not clearly stronger than the no-rule control, and the main detectability advantage is tied to a narrow detector calibration.

Critical strengths
  1. The paper isolates the effect of the spectral rules with useful controls, including a shared optimization skeleton, amplitude-matched perturbations, and a rank-reduction control. This makes the finding that the rules do not buy consistent success gains more informative than a comparison of bundled attack configurations.
  2. It evaluates several attack objectives and target types and includes a second dataset and attack pipeline. The authors also state the detector-calibration and resizing limitations, which keeps the interpretation appropriately bounded.
Critical weaknesses
  1. The strongest detectability result relies on one zero-day steganalysis detector calibrated without the constrained attack family. The paper itself shows that adding related attack examples to calibration or changing input resolution can substantially alter the detector’s verdict. This limits the practical significance of the claimed evasion advantage.
  2. The proposed rules do not yield a consistent targeted-transfer success improvement over the no-rule skeleton. The paper is therefore most convincing as a controlled evaluation of spectral constraints, while the evidence for a consequential new attack method is limited.
Questions for authors

Could you report the detector’s training and threshold-calibration composition in enough detail to establish which attack families and input transformations it has and has not seen? This would help assess the scope of the reported detection gap.

2 comments
AuthorsOpenReview ↗

Response to Reviewer u17J

We thank Reviewer u17J for the thoughtful comments. Our point-by-point responses follow.

W1. The detection result rests on one zero-day detector calibrated without the constrained attack family, and its verdict changes with calibration and input resolution, which limits its practical significance.

The detection result is indeed conditional on the detector's coverage, and this bounds its practical significance as an evasion result. We address the three parts of the concern in turn.

One detector. We trained a second detector family under the same coverage, a learned residual CNN (response to ECFZ W1). It gives the same low rates for the soft rules and the same ordering among constraints, so the result is not specific to one detector.

Calibration without the constrained family. This remains the scope of the result: once related examples enter calibration, detection is recovered (Table A5). We therefore present it as a measurement of what a detector calibrated only on the unconstrained attack misses, which tells a defender what calibration coverage is needed. We have revised the abstract and Section 5.6 so that it is not read as a general evasion claim.

Input resolution. The same scope applies. The detector is not re-fitted on resized images, and the paper states the result for the stated preprocessing only (Section 5.6).

W2. The rules give no consistent success gain over the no-rule skeleton, so the paper is a controlled evaluation and not a new attack method.

We share this reading: the rules add no consistent success over the no-rule skeleton, and the paper's value lies in the controlled evaluation. What that evaluation delivers is an attribution. Without the no-rule control, the success gain of the complete configuration would have been attributed to the spectral rule; Table 2 shows, on all five objectives, that it comes from the optimisation skeleton. What the rules themselves change is distortion: at comparable success they lower LPIPS by about a third (0.190.19 against 0.300.30 for the no-rule skeleton; Table 3). For work on structured perturbations this has three practical consequences:

  • compare a complete attack configuration against a no-rule skeleton before attributing its gain to a structural constraint;
  • do not treat post-hoc shrinkage as a substitute for an energy constraint applied during optimisation;
  • report a low detection rate together with the detector's input and training coverage.

The revised introduction makes this framing more prominent.

Q. What exactly the detector saw in training and threshold calibration, by attack family and input transformation.

The revised App. D includes the following table. All entries refer to the zero-day detector of Table 3a unless stated.

Composition
FeaturesAbout 700 rich-model residual features computed from the image alone, with no clean reference, on the luminance plane (mean of the three channels)
ClassifierEnsemble of Fisher linear discriminants on random feature subspaces
Attacked training imagesUnconstrained Omni-Attack only (ε·tanh parameterisation, Adam step 0.05, 299 px, ε = 16/255), drawn from a 100-image launch of that configuration; the coverage controls use 40 attacked images per fold from its 500-pair launch
Clean training images300-image clean training pool, plus the clean counterparts of the attacked ids outside the fold
ThresholdSet on a separate 300-image clean development pool that enters no training fold
Realised FPRRead on a further 300 clean images that no detector trained on
ScoringOut of fold; each attacked image receives one score from a detector that never saw it
ImageNetV2 (Table 3b)Own frozen splits and five folds; 900-image clean pool disjoint from the 500 attacked images

Never seen in calibration: any spectrally constrained configuration (HSS, USS, TASS); the no-rule direct-pixel skeleton; the L2L_2 control; the four other objectives (FTM, FOA-Attack, M-Attack, MF-ii); the MPCAttack pipeline; any 224-px or resized input.

Seen only in the controls of Table A5: the unconstrained attack rescaled to half and quarter amplitude, and the compacted family itself, each at the same total budget of attacked training images.

AuthorsOpenReview ↗

Follow-up to Reviewer u17J

Dear Reviewer u17J,

We would like to follow up on our response of 12 Nov, with the tables it referred to included below.

W1. One detector, calibrated without the constrained family.

A second detector family. We trained a learned residual CNN under the same coverage. TPR at the 5% clean threshold (Table A6 of the revised paper):

Configuration (step 0.5 unless stated)Steganalyser, 500CNN, 500Steganalyser, 50CNN, 50
Unconstrained (Omni-Attack, ε tanh, step 0.05)0.991.000.960.98
No rule (τ=0)0.971.000.940.96
HSS (r=128)0.660.930.840.70
USS (τ=0.5)0.090.130.120.10
TASS (τ=1)0.090.120.140.10
L2 ball 6.4 / 7.33 / 8.20.06 / 0.09 / 0.150.16 / 0.33 / 0.630.08 / 0.24 / 0.480.08 / 0.12 / 0.22
Low-pass R=160 / 1920.05 / 0.050.07 / 0.070.06 / 0.060.06 / 0.06
Rules at step 2.5 (HSS / USS / TASS)0.32 / 0.41 / 0.560.63 / 0.76 / 0.850.64 / 0.78 / 0.820.48 / 0.58 / 0.70

The soft rules stay at 0.09 to 0.13 under both families.

Generative victims. The gap also holds on six MLLMs (Table A11 of the revised paper). Cells are ASR (%) / Joint, the fraction of 200 images that name the target unflagged.

ConfigurationTPR@5%Qwen2.5Qwen3.8DeepSeekGLM-5.3Gemini-3.8GPT-5.6
Omni-Attack (unconstrained)0.98536.5 / 0.00043.5 / 0.00540.0 / 0.00035.0 / 0.00032.0 / 0.00026.5 / 0.000
No rule (step 2.5)0.91541.5 / 0.02542.0 / 0.02544.5 / 0.03041.5 / 0.03035.0 / 0.01531.0 / 0.020
HSS (step 2.5)0.28537.0 / 0.30047.5 / 0.33545.0 / 0.33540.0 / 0.30039.5 / 0.29529.0 / 0.205
USS (step 2.5)0.42042.0 / 0.25045.0 / 0.25041.5 / 0.24539.0 / 0.26037.0 / 0.21527.5 / 0.150
TASS (step 2.5)0.53542.5 / 0.20046.5 / 0.19547.5 / 0.22544.0 / 0.22035.0 / 0.16531.0 / 0.145
TASS (step 0.5)0.10032.0 / 0.29043.0 / 0.40041.0 / 0.38535.0 / 0.315not sentnot sent

Scope. We agree that the result depends on calibration. Once reduced-amplitude copies of the known attack enter calibration, detection rises from 0.26 to 0.88 (Table A5), and the revised Section 5.6 states that the low rate is not a general evasion claim.

W2. Controlled evaluation.

We share this reading, and the revised introduction frames the paper that way: the rules are the vehicle of the measurement and not a proposed method. The success gain belongs to the optimisation skeleton (Table 2), and at comparable success the rules lower LPIPS by about a third (0.19 against 0.30 for the no-rule skeleton; Table 3).

Q. Detector composition.

The full table is in our response above and is Table A3 of the revised paper.

As the discussion period is closing, we would be grateful to know whether these changes address the concerns, and we are glad to clarify anything that remains.

Comments on the paper

2 comments

Dear Reviewers,

Please take a moment to review the authors' rebuttal and assess whether your questions and concerns have been adequately addressed. If necessary, please engage in further discussion with the authors.

Kindly provide your final recommendation as soon as possible.

Thank you for your time and efforts!

AC

AuthorsOpenReview ↗

Thanks to all reviewers, and a final check before the discussion closes

Dear Reviewers,

As the author–reviewer discussion closes in less than 24 hours, we would like to thank all three reviewers for the time and care they have given to our submission.

We are especially grateful for the recognition the paper has received: Reviewer ECFZ recommended acceptance from the first round and maintained it after our response, Reviewer Sqyo raised the rating and considered the experimental evidence comprehensive, and Reviewer u17J found the controlled evaluation useful and its interpretation appropriately bounded.

We have revised the paper carefully in response to every comment, and uploaded the revision with all changes marked in blue. Following the reviewers' suggestions, the abstract, the introduction and Section 5.6 now state the detection result as conditional on detector coverage, and Sections 5.4 and 5.5 report the new controls next to the rules. App. D adds the detector's calibration composition (Table A3), a second, learned detector family (Table A6), energy and DCT controls re-optimised in the same skeleton (Tables A7 and A8), the component and gradient analyses (Tables A9 and A10), and the joint evaluation on six generative victims (Table A11). We believe the paper is clearer and more complete as a result, and we thank the reviewers for that.

If any question or concern remains, we would be glad to answer it in the time left. If the responses and the revision have addressed your concerns, we would appreciate it if you could let us know or update your assessment.

Thank you again for your efforts.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.