acceptodds
Under review as a conference paper at ICLR 2027

FactorizePatch: Factorizing the Printer-Camera Pipeline for Robust Physical Adversarial Patches

Abstract

Adversarial patches optimized in the digital domain often lose effectiveness once printed and recaptured, because printing and camera imaging distort their colors, textures, and high-frequency structures. Prior work improves physical robustness through generic geometric and photometric transformations, printer-aware modeling, camera-aware simulation, or a single monolithic digital-to-physical mapping. Single-stage methods, however, cover only part of the print-capture path, while monolithic mappings obscure where degradation originates and offer little control over difficult imaging conditions. We present FactorizePatch, which factorizes the print-and-capture process into sequential yet independent differentiable stages (print appearance, scene deployment, and camera formation) instead of collapsing it into one such mapping, and places the full pipeline inside targeted universal patch optimization. Printing and imaging therefore remain explicit and independently controllable. To support attack-relevant printer modeling, we build a dedicated print-scan dataset covering adversarial and print-sensitive patterns; on held-out pairs our print proxy reconstructs printed appearance more accurately than prior proxies, reducing MAE by up to . Within the camera stage, FactorizePatch samples multiple degradation strengths across optics/motion, sensor, ISP, and full-camera branches and adaptively reweights the branches by their current attack losses, directing optimization toward the patch's most vulnerable imaging condition while preserving coverage of compound degradations. Across CLIP and ResNet victims under simulated and real print-and-capture settings, FactorizePatch narrows the digital-to-physical gap to as little as % and achieves state-of-the-art targeted physical robustness. Code and data will be publicly available.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Physical Adversarial Patch, Printer-Camera Pipeline, Camera-Aware Robustness, Image Formation, Adversarial Machine Learning, Vision Security

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HyCxOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FactorizePatch to improve the physical robustness of adversarial patches. The method separates the printing and camera imaging processes into differentiable stages and adaptively weights different camera degradations during optimization. The authors also construct a print-scan dataset for training a printer proxy. Experiments on CLIP and ResNet-50 demonstrate improved targeted attack performance across simulated and real-world settings, including different cameras and illumination conditions.

Major Comment

The paper presents a practical framework and achieves strong physical attack results. However, the current comparisons do not fully establish how much of the improvement comes from factorization rather than additional calibration data and simulation components. The limited evaluation across printing conditions also raises concerns about generalization. I lean toward rejection but would reconsider if these issues are adequately addressed.

Critical strengths

1. Practical and well-motivated design. Separating printing and camera imaging is a reasonable approach to physical adversarial attacks. The factorized pipeline allows different degradation stages to be modeled independently, while adaptive weighting improves robustness under challenging imaging conditions.

2. Strong physical attack results. FactorizePatch achieves 79.06% and 99.06% physical attack success rates on CLIP and ResNet-50, respectively. The evaluations across three cameras, different illumination conditions, and independent print-placement sessions provide useful evidence of physical robustness.

Critical weaknesses

1. The contribution of factorization is not fully isolated.

Although the D2P baseline follows its original training protocol, FactorizePatch uses substantially more calibration data. A data-matched comparison would help clarify whether the improvements come from the proposed factorization or the additional calibration data.

Specifically, FactorizePatch uses 2,000 print-scan pairs, while D2P uses only 200 print-camera pairs. Moreover, the two methods differ in their simulation components. The current experiments therefore do not fully isolate the benefit of the factorized design itself.

2. Limited generalization across printing conditions.

The printer proxy is trained using a single printer and scanner, while physical evaluations mainly vary the capture devices. Although the paper includes independent printing and placement tests, its effectiveness across different printers, paper types, and printing settings remains unclear. This limits the generality of the proposed physical modeling framework.

Questions for authors

Q1. Can the authors compare FactorizePatch with a monolithic D2P model using comparable calibration data and computational resources? This would help isolate the benefit of factorization.

Q2. How well does FactorizePatch transfer to different printers or printing settings? How much additional calibration data would be required?

Q3. What is the computational overhead of the four camera branches and adaptive weighting? Can the authors provide training-time comparisons with the baselines?

7 comments
AuthorsOpenReview ↗

weakness 1-2

We thank the reviewer for their constructive comments. We have carefully addressed all concerns and detailed questions raised in the review through point-by-point responses, additional experiments, and analyses. We hope these clarifications adequately resolve your concerns and kindly ask you to reconsider the paper's score. We would greatly appreciate an increase in the score if deemed appropriate.

1. The contribution of factorization is not fully isolated.

Even when reducing the printer proxy training data of FactorizePatch to only 200 pairs, its physical attack performance still significantly outperforms the second-best method, D2P Adaptation, trained with the same amount of paired data.

To eliminate the influence of training data volume, we further reduced the number of print-scan pairs used to train the FactorizePatch printer proxy from 1,600 to 200, and compared it with D2P Adaptation trained using the same 200 paired samples. The results are shown below:

ModelMethodDigital ASRPhysical ASR
CLIPD2P Adaptation76.35%63.00%
CLIPOurs (200 pairs)89.20%73.33%
ResNet-50D2P Adaptation95.64%71.33%
ResNet-50Ours (200 pairs)99.94%83.33%

Even with only 200 print-scan pairs, FactorizePatch achieves improvements of 10.33 and 12.00 percentage points on CLIP and ResNet-50, respectively. These results demonstrate that the performance advantages of our method do not merely come from using more calibration data, but rather from full physical pipeline modeling and joint optimization.

2. Limited generalization across printing conditions.

Through transfer experiments on two additional printers, we demonstrate that FactorizePatch can effectively adapt to new printers using only 80–120 additional calibration pairs, while maintaining strong physical attack performance in complex real-world scenarios.

We acknowledge that the original manuscript primarily trained the printer proxy using a single printing device, providing insufficient validation of cross-printer generalization. To address this limitation, we conducted additional experiments on two different printers, using 80 and 120 additional calibration pairs, respectively, for full-network fine-tuning.

The results show that the fine-tuned models achieve average ΔE00 values of 2.0679 and 2.0062, respectively, both outperforming training from scratch with the same amount of data, which yields 2.2133 and 2.1536. Notably, after fine-tuning with 120 pairs, the reconstruction performance approaches the value of 1.9992 achieved by training on the original printer with 2,000 pairs.

Furthermore, in physical attack experiments using the additional printers, the average ASRs on CLIP and ResNet-50 reach 70.83% and 63.33%, respectively, representing an average improvement of 14.46 percentage points over D2P, the second-best method across different printers.

We also conducted additional tests in complex real-world scenarios, including piled clothing, stacked books, and a bathroom sink. With only 200 calibration pairs, FactorizePatch achieves an ASR of 94.00%, outperforming D2P (85.33%) by 8.67 percentage points (a relative improvement of 10.16%).

These results further validate the practical applicability of our method in terms of both cross-printer adaptation and robustness to complex physical environments.

AuthorsOpenReview ↗

Question 1-3

Q1. Can the authors compare FactorizePatch with a monolithic D2P model using comparable calibration data and computational resources?

With the same calibration data volume, FactorizePatch improves physical attack success rates by 5.73 and 15.73 percentage points on CLIP and ResNet-50, respectively.

We further trained a monolithic D2P model using the same 2,000 calibration pairs. The experimental results are shown below:

ModelDigital ASRPhysical ASR
CLIP89.20%73.33%
ResNet-5099.94%83.33%

In comparison, FactorizePatch achieves Physical ASRs of 79.06% and 99.06% on CLIP and ResNet-50, respectively, corresponding to improvements of 5.73 and 15.73 percentage points.

These results demonstrate that the factorized framework retains significant advantages even after controlling for differences in calibration data volume.

Q2. How well does FactorizePatch transfer to different printers or printing settings? How much additional calibration data would be required?

FactorizePatch requires only 80–120 calibration pairs per new printer for full-network fine-tuning to achieve color reconstruction performance close to that on the original printer, while maintaining strong attack performance on new printing devices.

We conducted experiments on two additional printers, comparing full-network fine-tuning with training from scratch. The results are shown below:

PrinterCalibration DataFull-Network Fine-Tuning ΔE00 ↓Training from Scratch ΔE00 ↓
Printer 180 pairs2.06792.2133
Printer 2120 pairs2.00622.1536
Original Printer2,000 pairs1.9992—

Fine-tuning on both new printers outperforms training from scratch with the same amount of data, demonstrating that the pretrained printer proxy can effectively transfer to new printing devices.

Further physical attack experiments show that the average ASRs on CLIP and ResNet-50 reach 70.83% and 63.33%, respectively, representing an average improvement of 14.46 percentage points over D2P, the second-best method across different printers.

Additionally, in complex real-world environments, including piled clothing, stacked books, and a bathroom sink, FactorizePatch achieves an ASR of 94.00% using only 200 calibration pairs, outperforming D2P (85.33%) by 8.67 percentage points.

These results demonstrate that FactorizePatch possesses cross-device adaptation capabilities and maintains strong physical attack performance under limited calibration data and complex background interference.

Q3. What is the computational overhead of the four camera branches and adaptive weighting? Can the authors provide training-time comparisons with the baselines?

The primary computational overhead of the four-branch architecture comes from additional forward and backward passes, whereas the EMA-based adaptive weighting mechanism itself introduces almost negligible computational cost.

We further measured the average iteration time (ms/iteration) of different methods on ResNet-50 and CLIP. The results are shown below:

MethodResNet-50CLIP
EOT-only137.2592.9
P&S148.1603.4
D2P Adaptation325.1777.6
Single P+C1,243.901,901.60
4-Branch Uniform4,931.408,021.30
4-Branch Adaptive4,881.008,066.30

First, compared with Single P+C, 4-Branch Uniform increases the average iteration time by 3.96× and 4.22× on ResNet-50 and CLIP, respectively. This overhead primarily comes from the additional forward and backward passes introduced by the four camera branches.

Second, compared with 4-Branch Uniform, the average iteration time of 4-Branch Adaptive changes by only -1.02% and +0.56% on ResNet-50 and CLIP, respectively. This demonstrates that EMA-driven adaptive weight computation introduces almost no additional computational overhead.

Overall, compared with D2P Adaptation, 4-Branch Adaptive requires approximately 15.01× and 10.37× the per-iteration time on ResNet-50 and CLIP, respectively.

Thank you for the additional experiments and detailed responses. While I appreciate the efforts to address my concerns, several issues remain unresolved:

1. Factorization is not sufficiently validated.

The results reported for the monolithic D2P model in Q1 exactly match those of Ours (200 pairs) in W1, making the claimed data-matched comparison difficult to verify. Thus, the specific contribution of factorization remains unclear.

2. Substantial computational overhead.

The four-branch adaptive method requires approximately 15.0× and 10.4× the per-iteration time of D2P Adaptation on ResNet-50 and CLIP, respectively. Its performance advantage under comparable computational budgets remains unverified.

3. Limited cross-printer generalization.

The additional experiments require 80–120 calibration pairs and full-network fine-tuning. They demonstrate adaptation to new printers rather than broad generalization across printing conditions. Variations in paper types and printing settings remain unexplored.

Overall, although the additional experiments partially address my concerns, they do not sufficiently resolve the above issues. Therefore, I maintain my original rating of 2 (Weak Reject).

Reviewer_1yt4OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies targeted universal adversarial patches and aims to improve their attack success rate after printing and camera capture. The authors propose FactorizePatch, which models the physical process in three differentiable stages: print appearance, scene deployment, and camera formation. Training starts with digital pretraining, followed by patch refinement through the full physical pipeline. Adaptive branch weighting is used to handle different camera degradations. The authors evaluate the method on two victim models and three capture devices, and use component ablations and repeated experiments to assess its effectiveness and stability.

Critical strengths
  1. Physical tests cover two models and three capture devices. The method clearly outperforms the compared baselines under the tested conditions.
  2. The method maintains its advantage across repeated printing, placement, and capture sessions. Separate repeated-scan tests also support the stability of the print–scan data.
  3. The authors build an attack-oriented print–scan dataset that could be useful for other studies once released.
  4. Ablation results show that modeling printing and camera formation together works better than modeling either one alone.
Critical weaknesses
  1. The authors train the print-appearance proxy on 1,600 paired samples, while P&S uses a pretrained model without retraining and D2P Adaptation uses 200 paired samples. This makes it difficult to tell whether the gains come from the method itself or from differences in training data and calibration.
  2. The method mainly combines existing printing, camera-modeling, and augmentation techniques in physical order. Its methodological novelty is limited.
  3. The adaptive multi-branch scheme does not always help. On ResNet-50, it performs worse than the single-path baseline under strong and medium illumination.
  4. The very small digital-to-physical gap is achieved only on ResNet-50. The gap on CLIP is still 11.65 percentage points.
  5. The evaluation does not include cross-printer tests and is limited to indoor scenes at distances of 20–40 cm. Performance with other printers and in more complex scenes remains untested.
Questions for authors
  1. Would the method still have a clear advantage if the learned-mapping baselines were trained on the same amount of paired data collected with the same printer?
  2. Why does the adaptive multi-branch scheme perform worse than P+C on ResNet-50 under strong and medium illumination?
  3. How well does the method work with a different printer without recalibration? Does it remain effective at greater distances or against more complex backgrounds?
4 comments
AuthorsOpenReview ↗

Weakness 1-3

We thank the reviewer for their constructive comments. We have carefully addressed all concerns and detailed questions raised in the review through point-by-point responses, additional experiments, and analyses. We hope these clarifications adequately resolve your concerns and kindly ask you to reconsider the paper's score. We would greatly appreciate an increase in the score if deemed appropriate.

W1. Fairness Concerns Due to Differences in Training Data Volume

Even when reducing the printer proxy training data of FactorizePatch to only 200 pairs, its physical attack performance still significantly outperforms the second-best method, D2P Adaptation, trained with the same amount of paired data.

To eliminate the influence of training data volume, we further reduced the number of print-scan pairs used to train the FactorizePatch printer proxy from 1,600 to 200, and compared it with D2P Adaptation trained using the same 200 paired samples. The results are shown below:

ModelMethodDigital ASRPhysical ASR
CLIPD2P Adaptation76.35%63.00%
CLIPOurs (200 pairs)89.20%73.33%
ResNet-50D2P Adaptation95.64%71.33%
ResNet-50Ours (200 pairs)99.94%83.33%

Even with only 200 print-scan pairs, FactorizePatch achieves improvements of 10.33 and 12.00 percentage points on CLIP and ResNet-50, respectively. These results demonstrate that the performance gains of our method do not merely come from using more calibration data, but rather from full physical pipeline modeling and joint optimization.

W2. Insufficient Methodological Novelty

The core contribution of FactorizePatch lies in formulating the complete digital-to-physical imaging process as an explicitly factorized, differentiable, and jointly optimizable framework, rather than simply combining existing modules.

Specifically, our contributions are threefold:

  1. Complete Physical Pipeline Factorization: We decompose the physical process into three stages: Print Appearance, Scene Deployment, and Camera Formation. This enables physical degradations at different stages to be independently modeled, controlled, and jointly optimized, rather than being represented by a single black-box mapping.

  2. Attack-Oriented Printer Proxy Calibration: We construct a dedicated print-scan dataset covering adversarial patterns, gamut-boundary patterns, and high-frequency structures, improving the fidelity of the printer proxy in simulating adversarial textures. Its MAE is reduced by 36.8% and 63.5% compared with P&S and D2P Adaptation, respectively.

  3. Adaptive Camera-Branch Optimization: We decompose camera imaging into four branches: Optical, Sensor, ISP, and Full, and introduce an EMA-based adaptive weighting mechanism to achieve coordinated optimization across different imaging conditions.

Therefore, FactorizePatch is not merely a straightforward combination of independent modules. Instead, it establishes a physical adversarial attack framework that takes the complete physical process as its modeling target, stage-wise factorization as its structural foundation, and adaptive optimization as its core mechanism. Ablation studies and cross-device experiments further validate the effectiveness of its individual components.

W3. Suboptimal Performance of the Adaptive Multi-Branch Strategy Under Certain Illumination Conditions on ResNet-50

This phenomenon primarily arises from differences in gradient responses to camera degradation branches across victim models, reflecting the trade-off between multi-condition joint optimization and single-path specialized optimization.

Specifically, on ResNet-50, Adaptive performs worse than P+C under both Strong and Medium illumination, with a decrease of 6.42 percentage points under Strong illumination. However, it achieves an improvement of 15.25 percentage points under Weak illumination.

To further investigate this phenomenon, we fixed the respective Adaptive Patches obtained after training and conducted gradient diagnostics across camera branches. On ResNet-50, the gradient magnitude proportions of the O/S/I branches are 7.20%, 25.93%, and 35.50%, respectively. In contrast, the corresponding proportions on CLIP reach 59.90%, 67.39%, and 73.56%.

These results indicate that ResNet-50 exhibits relatively weaker optimization responses to the O/S/I branches, with the optimization process still predominantly driven by the Full branch. The additional optimization benefits provided by multiple branches are mainly reflected under Weak illumination. In contrast, CLIP can obtain broader optimization signals from different branches and therefore consistently outperforms P+C under all three illumination conditions. This also explains why the same adaptive strategy leads to different performance outcomes across victim models.

AuthorsOpenReview ↗

Weakness 4-5

W4. Persistent Digital-to-Physical Performance Gap on CLIP

Although a D2P Gap of 11.65 percentage points remains on CLIP, FactorizePatch achieves the best physical attack performance on both victim models.

On CLIP, FactorizePatch achieves an average physical Target Top-1 of 79.06%, outperforming the second-best method, D2P Adaptation (63.00%), by 16.06 percentage points. Meanwhile, it reduces the D2P Gap from 13.35 to 11.65 percentage points.

On ResNet-50, FactorizePatch achieves an average physical Target Top-1 of 99.06%, outperforming D2P Adaptation (71.33%) by 27.73 percentage points, with a D2P Gap of only 0.87 percentage points.

Therefore, despite the differences in digital-to-physical performance gaps across victim models, FactorizePatch consistently achieves the best physical attack performance on both models.

W5. Generalization Across Printers, Shooting Distances, and Complex Backgrounds

Additional experiments demonstrate that FactorizePatch maintains strong physical attack performance across different printers and in complex real-world backgrounds.

To evaluate cross-printer generalization, we conducted experiments using different printers. The average physical attack success rates reach 70.83% and 63.33% on CLIP and ResNet-50, respectively, demonstrating the method's ability to transfer across different printers.

To evaluate generalization to complex backgrounds, we further conducted physical experiments in real-world scenarios, including piled clothing, stacked books, and a bathroom sink. These scenarios involve complex background textures and visual interference. FactorizePatch, trained using only 200 print-scan pairs, achieves the following results:

FactorizePatch achieves a physical attack success rate of 94.00%, outperforming the second-best method, D2P (85.33%), by 8.67 percentage points, and the third-best method, EOT (73.94%), by 20.06 percentage points. Notably, this result is achieved using only 200 print-scan pairs for training, further demonstrating the strong adaptability of our method to complex backgrounds under limited calibration data.

AuthorsOpenReview ↗

Question 1-3

Q1. Does Our Method Still Have a Clear Advantage When Learning-Based Mapping Baselines Are Trained Using the Same Printer and the Same Amount of Paired Data?

Yes. Even when reducing the printer proxy training data of FactorizePatch to only 200 pairs, its performance still significantly outperforms the second-best method, D2P Adaptation.

As shown in W1, with the same amount of training data, FactorizePatch achieves physical attack success rates of 73.33% and 83.33% on CLIP and ResNet-50, respectively, outperforming D2P Adaptation by 10.33 and 12.00 percentage points. These results demonstrate that the performance advantages of our method do not merely stem from using more print-scan paired data.

Q2. Why Does the Adaptive Multi-Branch Strategy Underperform P+C Under Strong and Medium Illumination on ResNet-50?

This phenomenon primarily arises from differences in gradient responses to camera degradation branches across victim models, reflecting the trade-off between multi-condition joint optimization and single-path specialized optimization.

Specifically, on ResNet-50, Adaptive performs worse than P+C under both Strong and Medium illumination, with a decrease of 6.42 percentage points under Strong illumination. However, it achieves an improvement of 15.25 percentage points under Weak illumination.

To further investigate this phenomenon, we fixed the respective Adaptive Patches obtained after training and conducted gradient diagnostics across camera branches. On ResNet-50, the gradient magnitude proportions of the O/S/I branches are 7.20%, 25.93%, and 35.50%, respectively. In contrast, the corresponding proportions on CLIP reach 59.90%, 67.39%, and 73.56%.

These results indicate that ResNet-50 exhibits relatively weaker optimization responses to the O/S/I branches, with the optimization process still predominantly driven by the Full branch. The additional optimization benefits provided by multiple branches are mainly reflected under Weak illumination. In contrast, CLIP can obtain broader optimization signals from different branches and therefore consistently outperforms P+C under all three illumination conditions. This also explains why the same adaptive strategy leads to different performance outcomes across victim models.

Q3. How Does the Method Perform on Different Printers Without Recalibration? Can It Adapt to Longer Shooting Distances or Complex Backgrounds?

To evaluate cross-printer generalization, we conducted experiments using different printers. The average physical attack success rates reach 70.83% and 63.33% on CLIP and ResNet-50, respectively, demonstrating the method's ability to transfer across different printers.

To evaluate generalization to complex backgrounds, we further conducted physical experiments in real-world scenarios, including piled clothing, stacked books, and a bathroom sink. These scenarios involve complex background textures and visual interference. FactorizePatch, trained using only 200 print-scan pairs, achieves the following results:

FactorizePatch achieves a physical attack success rate of 94.00%, outperforming the second-best method, D2P (85.33%), by 8.67 percentage points, and the third-best method, EOT (73.94%), by 20.06 percentage points. Notably, this result is achieved using only 200 print-scan pairs for training, further demonstrating the strong adaptability of our method to complex backgrounds under limited calibration data.

Thank you for your careful review of our work. We have also addressed each of the other concerns you raised and provided corresponding analyses and additional experiments.

Furthermore, during the discussion period, we provided detailed responses and supplementary experiments addressing several key concerns shared by you and another reviewer. Specifically, to distinguish the contribution of Factorization itself from the influence of calibration data volume, we conducted fair comparisons using the same amount of calibration data (200 pairs). To further investigate the performance of Adaptive Weighting across different models and illumination conditions, we conducted additional gradient analyses and comparisons with alternative weighting strategies. Moreover, to address concerns regarding the reliability of the Digital-to-Physical Gap evaluation, we supplemented our experiments with Clean Accuracy measurements and Matched-Scene Evaluation. To further assess the robustness and generalization of our method under physical conditions, we conducted additional physical experiments across different printers and in complex real-world backgrounds. Following the submission of these responses and additional experiments, the other reviewer subsequently increased their overall rating of the paper.

We hope these additional results will help you gain a more comprehensive understanding of the technical contributions of FactorizePatch and further address your concerns.

We sincerely look forward to your response and would greatly appreciate your feedback on our additional experiments and responses. If you consider that our supplementary experiments and responses have adequately addressed your concerns, we would be grateful if you would consider raising your overall rating of the paper accordingly.

Reviewer_bAAxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FactorizePatch optimizes physical adversarial patches through a learned print proxy and a differentiable camera pipeline. It also introduces adaptive weighting across camera-degradation branches. Experiments cover CLIP and ResNet-50, with physical evaluation on three capture devices.

Critical strengths
  1. The physical evaluation is a useful part of the paper. Cross-device tests and independent print–placement sessions provide evidence beyond simulation.
  2. The component analysis shows a clear benefit from combining printer and camera modeling.
  3. The attack-oriented print–scan dataset is a useful contribution, and its acquisition and registration procedures are described in detail.
Critical weaknesses
  1. The comparisons do not isolate the benefit of factorization. The proposed print proxy uses 1,600 training pairs from the authors’ setup, whereas P&S uses released weights without recalibration and D2P is trained on 200 pairs. Moreover, Table 3 evaluates D2P against scanned targets even though it is trained on camera captures. Matched-data comparisons are needed to separate architecture gains from calibration-data and acquisition differences.
  2. The weighting rule does not necessarily emphasize the weakest branch. Equation 7 measures each branch’s loss relative to its own history. A consistently difficult branch can receive less weight than an easier branch whose loss has recently increased. The weights are also applied to the next block, when degradation configurations are resampled. Please justify this design and compare it with weighting based on current absolute losses.
  3. The reported D2P gap is not a paired measure of transfer degradation. Digital T-ASR is measured on initially correct dataset images, while physical Target Top-1 uses videos of tabletop objects. Their difference mixes changes in the evaluation population with print-and-capture effects. The 0.87 percentage-point gap should therefore be interpreted cautiously. A matched-scene evaluation and clean physical accuracy would be more informative.
  4. The printer–camera separation is approximate. The print proxy learns R ◦ Π, including the scanner response, rather than printer appearance alone. The repeatability experiment measures random acquisition variation but does not rule out systematic scanner color or blur effects. This matters when interpreting the modules as separate physical stages or attributing degradation to a particular stage.
  5. Physical uncertainty is only partly characterized. The main results average densely sampled video frames, and the independent print–placement experiment covers one source–target setting. Appendix E also fixes the initial patch and evaluates one target. Repeated physical sessions across several targets, with session-level uncertainty, would better establish the consistency of the gains.
Questions for authors
  1. Does adaptive weighting still outperform uniform weighting under matched victim forward/backward computation?
  2. In Table 4, adaptive weighting gives lower worst-illumination performance than the single-path baseline on ResNet-50: 50.58% versus 57.00%. What explains this result?
  3. How well do simulated branch losses predict failures under real capture conditions?
3 comments
AuthorsOpenReview ↗

Weaknesses 1-2

We thank the reviewer for their constructive comments. We have carefully addressed all concerns and detailed questions raised in the review through point-by-point responses, additional experiments, and analyses. We hope these clarifications adequately resolve your concerns and kindly ask you to reconsider the paper's score. We would greatly appreciate an increase in the score if deemed appropriate.

W1. Factorized Modeling and Calibration Data Volume

To address the fairness concern regarding differences in calibration data volume, we further trained Ours using only 200 pairs of data.

To isolate the effect of training data volume, we reduced the number of print-scan pairs used to train the FactorizePatch printer proxy from 1,600 to 200, and compared it with D2P Adaptation trained using the same 200 paired samples. The results are shown below:

ModelMethodDigital ASRPhysical ASR
CLIPD2P Adaptation76.35%63.00%
CLIPOurs (200 pairs)89.20%73.33%
ResNet-50D2P Adaptation95.64%71.33%
ResNet-50Ours (200 pairs)99.94%83.33%

Even with only 200 print-scan pairs, FactorizePatch achieves improvements of 10.33 and 12.00 percentage points on CLIP and ResNet-50, respectively. These results demonstrate that the performance gains of our method do not merely come from using more calibration data, but rather from full physical pipeline modeling and joint optimization.

W2. Adaptive Weighting vs. Absolute-Loss Weighting

Our experiments show that adaptive weighting outperforms absolute-loss weighting by 0.46 and 7.31 percentage points on CLIP and ResNet-50, respectively. Its advantage primarily stems from the effective gradient allocation adjustments enabled by historical normalization.

Our weighting mechanism is designed to measure the loss variation of each branch relative to its own historical state, rather than simply assigning higher weights to branches with larger absolute losses. This distinction is important because, except for the Full branch, the other three branches contain fixed components. If these fixed components lead to consistently higher losses, the corresponding branches may continuously receive larger weights, thereby overlooking the contributions of the varying components.

Further analysis shows that, on CLIP, the differences between the current branch losses are relatively small compared with the temperature parameter T=1T=1. Consequently, absolute-loss weighting behaves almost identically to uniform weighting, with an average deviation of its maximum weight from 0.25 of only 0.00293, whereas adaptive weighting achieves 0.12463. Compared with the uniformly weighted gradient under the same state, the average angle of the gradient induced by adaptive weighting reaches 9.27∘9.27^\circ, while that of absolute-loss weighting is only 0.24∘0.24^\circ. This demonstrates that adaptive weighting can effectively adjust branch-wise gradient allocation even when the differences in raw losses are small. Its advantage comes from consistently effective gradient direction scheduling.

On ResNet-50, the average deviation of the maximum weight from 0.25 under absolute-loss weighting is only 0.000983, whereas adaptive weighting achieves 0.05250. The average angle between the adaptively weighted gradient and the uniformly weighted gradient is 1.55∘1.55^\circ, compared with only 0.028∘0.028^\circ for absolute-loss weighting. This indicates that the performance gains arise from localized gradient adjustments in certain states, which accumulate through continuous optimization to produce measurable improvements in attack performance.

Furthermore, we provide the following clarification regarding the update and application schedule of adaptive weights across different Blocks.

Specifically, all KK batches within the kk-th Block use the weights available at the beginning of that Block, w(k)\boldsymbol{w}^{(k)}, together with fixed degradation configurations ξb(k)\xi_b^{(k)}. At the end of the Block, the branch-wise average losses are used to compute the weights for the next Block, w(k+1)\boldsymbol{w}^{(k+1)}, while the EMA historical statistics are updated accordingly.

Upon entering the (k+1)(k+1)-th Block, each branch independently samples a new degradation configuration ξb(k+1)\xi_b^{(k+1)}, while the updated weights w(k+1)\boldsymbol{w}^{(k+1)} are adopted to compute the weighted attack objective. Both the degradation configurations and branch weights remain fixed throughout the KK batches within that Block, until they are updated again at its end.

AuthorsOpenReview ↗

Weakness 3-5

W3. Matched-Scene Evaluation of Digital-to-Physical Transferability

To address the inconsistency in evaluation samples in the original D2P Gap assessment, we supplemented our experiments with Clean Accuracy measurements and matched-scene evaluations to re-examine digital-to-physical transferability under identical scene conditions.

First, we evaluated the Clean Accuracy on the original desktop videos without adversarial patches. Subsequently, we overlaid digital patches onto the same desktop scenes and conducted matched-scene evaluations of EOT, D2P, P&S, and our method, comparing the results with the original physical attack results.

The experimental results show that the Clean Accuracy of CLIP and ResNet-50 reaches 100.00% and 97.78%, respectively. Moreover, neither model naturally predicts the target class in any of the evaluated samples, with a target-class prediction rate of 0% for both models.

In the matched-scene evaluation, our method achieves physical attack success rates of 79.06% and 99.06% on CLIP and ResNet-50, respectively, outperforming the second-best method, D2P (63.00% and 71.33%), by 16.06 and 27.73 percentage points.

Meanwhile, our method also achieves the smallest absolute gap between digital and physical attack performance: only 2.61 percentage points on CLIP, compared with 8.67 percentage points for the second-best method, D2P; and only 0.76 percentage points on ResNet-50, compared with 2.27 percentage points for the second-best method, P&S.

W4. Separation of the Printing and Camera Imaging Stages

We acknowledge that the printer proxy learns an effective Print-Scan mapping that incorporates scanner responses, rather than an isolated printer response. The purpose of adopting a fixed scanning protocol is to reduce random acquisition variations, rather than to claim that the systematic effects of the scanner are completely eliminated.

First, to model the printing process separately from subsequent camera imaging, the physically printed patches must be converted into digital images, thereby establishing an approximate mapping between digital inputs and physical printing outputs. Since physical printing results cannot be directly used to train a digital proxy model, an acquisition device is required to perform this conversion. We therefore employ a scanner to construct the Print-Scan paired dataset, approximating the physical variations introduced during the printing stage.

Second, flatbed scanners employ contact-based two-dimensional imaging under fixed illumination conditions. Their optical sensors capture reflected light from the printed surface, effectively reducing interference caused by environmental illumination and variations in camera pose. Kirillova et al. (2017) experimentally evaluated two flatbed scanners and showed that, after color calibration, 96.9%–97.5% of the test samples achieved a color difference of ΔEab∗<3\Delta E_{ab}^{*} < 3. Kemp (2014) also reported a relative standard deviation of below 0.17% for repeated scans using flatbed scanners.

These studies support the high acquisition accuracy and repeatability of flatbed scanners under controlled conditions. Therefore, when using fixed devices and acquisition parameters, the scanner response can be regarded as a shared and reproducible system transformation. Its effects can consequently be consistently captured by the proxy model, reducing random acquisition variations across samples.

W5. Physical Experiment Uncertainty and Generalization to Complex Backgrounds

Additional experiments in complex backgrounds show that Ours achieves an attack success rate of 94.00%, even when trained with only 200 calibration pairs.

To further evaluate physical attack performance in complex environments, we conducted additional experiments in three real-world scenarios: piled clothing, stacked books, and a bathroom sink. These scenarios introduce substantial background clutter and visual interference.

FactorizePatch achieves a physical attack success rate of 94.00%, outperforming the second-best method, D2P (85.33%), by 8.67 percentage points, and the third-best method, EOT (73.94%), by 20.06 percentage points. Notably, these results are obtained using only 200 print-scan pairs for training, further demonstrating the strong adaptability of our method to complex backgrounds under limited calibration data.

AuthorsOpenReview ↗

Question 1-3

Q1. Does Adaptive Weighting Still Outperform Uniform Weighting Under a Matched Computational Budget?

Yes. Adaptive Weighting and Uniform Weighting use exactly the same computational budget for forward and backward passes through the four camera branches, and differ only in their weighting strategies.

Both Uniform Weighting and Adaptive Weighting perform forward and backward passes through the same four victim-model camera branches at every iteration, following an identical batch training schedule.

The only difference lies in their weighting strategies: Uniform Weighting uses fixed weights, whereas Adaptive Weighting dynamically adjusts the weights based on the already computed branch losses. This process involves only lightweight scalar operations and requires no additional victim-model forward or backward passes. Therefore, the performance improvement brought by adaptive weighting comes from the dynamic weight adjustment mechanism rather than additional victim-model computational overhead.

Q2. Why Does ResNet-50 Perform Worse Under Strong Illumination?

Adaptive Weighting does not outperform the single-path baseline under all illumination conditions. Instead, on ResNet-50, it trades off some performance under strong and medium illumination for a 15.25-percentage-point improvement under weak illumination.

We attribute this result to differences in how the victim model responds to gradients from different camera degradation branches, reflecting the trade-off between multi-condition joint optimization and single-path specialized optimization.

To investigate this phenomenon, we fixed the respective Adaptive Patches and performed gradient diagnostics under three perturbation strengths: Strong, Medium, and Weak. For ResNet-50, the gradient magnitude shares of the O/S/I branches are 7.20%, 25.93%, and 35.50%, respectively, indicating that its optimization is still predominantly driven by the F branch, with relatively limited additional optimization contributions from the other branches. In contrast, the corresponding gradient magnitude shares for CLIP reach 59.90%, 67.39%, and 73.56%, indicating that its optimization benefits from broader contributions across different branches.

This difference is consistent with the results in Table 4: on ResNet-50, Adaptive performs 6.42 percentage points worse than P+C under Strong illumination, but achieves a 15.25-percentage-point improvement under Weak illumination. In contrast, CLIP consistently outperforms P+C under all three illumination conditions.

Therefore, the same multi-branch optimization strategy can lead to different performance trade-offs depending on the gradient response characteristics of the victim model. For ResNet-50, jointly optimizing across multiple degradation conditions may sacrifice some specialized performance under Strong illumination, whereas CLIP benefits more broadly from multi-branch optimization.

Q3. To What Extent Can Simulated Branch Losses Predict Attack Failures Under Real Acquisition Conditions?

Simulated branch losses provide a physically grounded proxy signal for physical robustness optimization.

Real-world cameras follow a continuous imaging pipeline involving optical imaging, sensor response, and ISP processing. Our simulator follows the same processing order and constructs a complete imaging pipeline, using the O/S/I/F branches to simulate degradations at different stages and their combined effects.

When the degradation intensity space is divided into 1,000 bins for each branch, the grid occupancy rate reaches 74.675%. In the 20 × 20 joint space of depth and motion angle, the occupancy rates of the O and F branches reach 75.25% and 73.50%, respectively. These results demonstrate that the simulator covers a broad range of degradation conditions, providing a basis for computing branch losses under different imaging states.

Real-world physical experiments further validate the effectiveness of optimizing based on branch losses. Compared with Uniform Weighting, Adaptive Weighting improves the average physical Target Top-1 by 9.72 and 14.75 percentage points on CLIP and ResNet-50, respectively. It also improves performance under the worst illumination condition by 27.33 and 24.66 percentage points, respectively.

These results support the effectiveness of simulated branch losses as an optimization signal for physical robustness.

Comments on the paper

1 comment

We sincerely thank all reviewers and the Area Chair for their time, thoughtful evaluations, and constructive feedback.

We have carefully addressed all questions and concerns raised by the reviewers through detailed point-by-point responses, additional experiments, and comprehensive analyses, further strengthening the technical justification and experimental validation of our work.

Specifically, to address concerns regarding the contribution of factorized modeling, we conducted fair comparisons using matched calibration data. To further validate the multi-branch adaptive optimization mechanism, we provided comparisons with alternative weighting strategies, gradient allocation analyses, and computational cost evaluations. To address concerns regarding the consistency between digital and physical evaluations, we supplemented our experiments with Clean Accuracy measurements and Matched-Scene Evaluation. Furthermore, we conducted additional experiments on cross-printer transfer, cross-printer-system evaluation without target-specific calibration, few-shot adaptation, and physical attack robustness in complex real-world environments. In response to concerns about computational overhead, we also conducted additional compute-matched experiments under strictly controlled GPU budgets, demonstrating the performance advantages of FactorizePatch under equal computational resources.

Importantly, one of the reviewers has acknowledged and positively recognized our additional experiments and analyses addressing their major concerns, particularly regarding calibration data fairness, adaptive weighting, digital-to-physical evaluation consistency, printer-camera pipeline modeling, and robustness in complex physical environments. Following these clarifications and additional experiments, the reviewer raised their overall rating from Weak Reject to Weak Accept.

We sincerely appreciate the constructive feedback from all reviewers, which has helped us further strengthen the methodological justification and experimental evaluation of our work. We also hope that the newly added compute-matched experiments, together with our other supplementary results, will adequately address the remaining concerns and encourage the other reviewers to reconsider their assessments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.