- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
PHASE studies targeted steering of retrieval-augmented diffusion models by replacing the four retrieved conditioning embeddings supplied to a frozen generator. It builds a pool of candidate images, scores them in the retrieval space, assembles several candidate packets, and chooses one by checking the generated outputs. The evaluation uses one frozen RDM and 80 source–target attack instances, with additional analyses across seeds, samplers, evaluators, and packet configurations.
Critical strengths
- Checking the actual generated outputs is a sensible way to study packet selection. The observation that retrieval similarity does not reliably predict the generator’s response is useful.
- The held-out seed/sampler evaluation and native-retrieval diagnostics help clarify the scope of the results. I also appreciate that the authors report negative findings and acknowledge the uncertainty in some of the statistical comparisons.
Critical weaknesses
-
My main concern is how the attacker obtains the assumed access. Section 3.1 does state that the attacker can replace the conditioning packet and query the frozen generator, so the issue is not simply a missing threat-model section. What is missing is a concrete system in which those permissions are available. The introduction mentions external services, middleware, and caches, while Appendix A mentions user-controlled repositories, but controlling a repository is different from replacing all four embeddings after retrieval. Can the attacker submit arbitrary packets to the victim generator and observe its outputs? What access do they have to the native encoder and sampling settings? Appendix H makes this distinction consequential: direct insertion gives only 2.50% Hit@4 and a 1.25-point ASR gain. The main result therefore depends heavily on direct control of the conditioning interface, whose trust boundary is never clearly established.
-
I do not understand the separation between the main baselines and the “target-prior stress tests.” TopTargetSim reaches 47.50% primary ASR, compared with 41.67% for PHASE, yet it is set aside as requiring stronger assumptions. Under the stated threat model, the attacker already constructs the whole packet, and there is no explicit minimum query similarity or limit on target-class members. RandomTarget also uses four target-class neighbors and is counted as a matched baseline. More target-dominated conditioning is a property of the resulting packet, not by itself a different attacker capability. If PHASE is solving a constrained problem, those constraints need to be stated and applied to every baseline. Otherwise, the stronger result from this simpler selection rule leaves the benefit of PHASE unclear.
-
The ablations do not establish the value of most of the proposed construction strategy. Tables 20 and 21 measure similarities of selected packets, rather than attack success on generated images. This is a weak basis for conclusions about which scoring terms or candidate families improve steering, especially when the paper’s premise is that retrieval-space scores are unreliable proxies for generation. StaticOnly does test the benefit of probing, but it removes generator queries altogether. I would like to see a simple packet search given the same candidate pool and generator-query budget. The advantage over JOB-style is also uncertain: the primary gain is 5.00 points with a 95% interval of [-5.42, 15.42] in Table 6. The results suggest that probing helps, but do not yet make a convincing case for the full hierarchy.
-
The semantic evaluation is less robust than the main claims suggest. Table 2 shows clear gains within the CLIP family, but almost no gain under SigLIP. The explanation based on the recognizability threshold is incomplete: SigLIP target top-5 changes only from 24.58% to 26.25%, target-beats-source from 47.08% to 49.17%, and the target–source margin stays near zero. These measures do not depend on that threshold. Since the method uses CLIP-based retrieval scoring and an evaluator-based selection objective, an independent check of the generated content matters here. The selected visual examples are helpful, but they do not establish how often the target is actually recognizable across the benchmark. A blinded human check on a representative subset would help resolve this concern.
Questions for authors
- In a concrete deployment, who controls the retrieved packet, and how can the attacker replace it? Please distinguish repository access from direct interface access, and specify encoder access, generator-query access, output visibility, and control over sampling settings.
- What explicit constraint makes TopTargetSim a stronger-threat baseline while RandomTarget is threat-matched? How do the methods compare under the same packet constraints, candidate pool, and generator-query budget?
- How much of the measured success is confirmed by an independent assessment of the generated images? Also, does Table 2 report SigLIP ASR with or without the recognizability gate? Appendix F says the gated ASR is zero.