- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
In this work, authors introduce D-Scope, a framework for retrieving and steering SAE features in diffusion transformers without per-feature text annotations or target-generation activations. Authors construct a visual centroid for each indexed feature by aggregating SigLIP 2 embeddings of its highly activating image patches. A target text query is then matched against these centroids, either directly or contrastively relative to a reference query, to select a decoder direction for spatially masked intervention. Authors study 150 SAEs across two one-step generators, different SAE families, input settings, and layers. On a 1000 sample steering benchmark, they compare direct and contrastive retrieval in terms of regional target alignment and outside-region preservation under per-case best-of-sweep strength selection. Authors additionally demonstrate that reconstruction fidelity alone does not characterize dictionary utilization or the availability of visual evidence.
Critical strengths
- Overall, I find the proposed approach interesting and useful. Matching text queries against visual evidence removes the need to annotate individual SAE features with text descriptions or collect target-generation activations for feature selection. I believe this is a useful contribution for applying SAE dictionaries to fine-grained generation control.
- Experiments and ablation studies are comprehensive, covering different SAE families, input settings, layers, etc. The finding that high reconstruction accuracy can coexist with low dictionary utilization and limited visual-evidence coverage is interesting. I appreciate that authors evaluate target alignment and outside-region preservation separately, rather than treating successful steering as sufficient evidence of a localized edit.
Critical weaknesses
- My primary concern is with the evaluation of target alignment. SigLIP 2 is used for both feature retrieval and target scoring, and intervention strength is selected separately for each case using the same score. Authors clearly disclose this. Nevertheless, these results characterize best-of-sweep performance, and it is not clear how much of the gain would remain with a strength chosen independently of the test outcome. An independent evaluator and a fixed or validation-selected strength would help clarify this.
- The practical scope of contrastive retrieval could be discussed more clearly. The benchmark provides the reference query, which describes either the generic object or the competing source attribute. It is not clear how this reference should be chosen for complex edits. Comparisons with description-based SAE retrieval would also help quantify the benefit of avoiding per-feature text annotations.
- Please see the questions below as well.
Questions for authors
Questions:
- Could the authors report steering results with an evaluator independent of SigLIP 2 and a strength selected on a held-out validation set? This would help assess whether the gains persist beyond the current best-of-sweep protocol.
- For contrastive retrieval, how is the reference query obtained in practice? Appendix D.1 specifies a generic object query for under-specified contexts and the competing source attribute for explicit-conflict contexts. How would this choice extend to more complex scenes or edits that do not have a simple attribute-object structure? Could the reference be derived automatically from the source prompt or a caption of the target region?
- For the SANA teacher, the student-trained dictionaries are reused and the same direction is injected at every denoising step. Could the authors comment on why is it reasonable to reuse the same student-trained SAE dictionary across all denoising steps, given that the representations may change with the timestep? Did the authors examine at which timesteps the intervention is most effective? In particular, could restricting the intervention to a subset of timesteps improve outside-region preservation?
- For outside-region preservation, could the authors additionally report masked MSE or PSNR? Since color changes are an important part of the benchmark, a complementary pixel-wise metric would help characterize changes that may not be fully reflected by LPIPS.
- In Section 4.2, the example of EV = 0.9997 with only 1.5% of features utilized is interesting. In particular I'm surprised that the SAE reconstruction error is so small. What should be the takeaway here? That the superposition hypothesis is pretty accurate for Nitro Layer 2?
Suggestions:
- Figure 3(c) reports the change in utilization due to centering, rather than absolute utilization. Although this is stated in the caption, I would recommend emphasizing it in the discussion so that the small changes do not read as poor absolute utilization. The same distinction applies to the target-gain changes in Figure 4(c).
- A comparison against description-based SAE retrieval would be particularly informative, since avoiding per-feature annotations is a main motivation. Comparisons with non-oracle editing methods would further clarify the trade-off between target alignment and preservation.
- Could the authors comment on how their technique would extend to SAEs trained directly on multi-step diffusion models, where features may change across denoising timesteps?
9 comments
The first table below compares the two protocols on the same test cases for mean-centered TopK. Here, "C − D" denotes contrastive minus direct retrieval, paired by case and seed. We report the mean ± sample standard deviation (SD) across the three seed means. Regional gains, LPIPS-out, and their SDs are scaled by 10³ in the tables.
| Strength selection | Independent target scorer | Direct | Contrastive | Paired C − D |
|---|---|---|---|---|
| Per-case best-of-sweep | DFN | 21.04 ± 0.85 | 35.17 ± 1.18 | +14.13 ± 0.56 |
| Calibration–test | DFN | 12.32 ± 0.58 | 27.23 ± 0.68 | +14.91 ± 0.29 |
| Per-case best-of-sweep | CLIP-H | 23.71 ± 0.73 | 43.23 ± 0.96 | +19.52 ± 0.55 |
| Calibration–test | CLIP-H | 12.74 ± 0.48 | 33.88 ± 0.70 | +21.14 ± 0.78 |
Across these cases, contrastive retrieval achieves higher mean gains for all five dictionaries under both protocols and both scorers. These results are consistent with our results in the original paper.
Fixed strength and held-out concepts under the original metric. Following your suggestion, we also ran additional experiments to compare the strategies for mean-centered TopK using Region SigLIP Δ. At fixed ρ = 0.30, we compute "C − D" (i.e., contrastive minus direct retrieval) from the mean gains of the two strategies. Best-of-sweep maximizes Region SigLIP Δ for each case and seed. Two-fold calibration maximizes mean Region SigLIP Δ on 50 concepts, applies the selected strength to the other 50, and then swaps the groups. Both strategies use the same strength within each test fold (0.30 in one fold and 0.40 in the other).
| Strength selection | Direct | Contrastive | C − D |
|---|---|---|---|
| Fixed ρ = 0.30 | 11.45 ± 0.34 | 20.13 ± 0.17 | +8.67 ± 0.50 |
| Per-case best-of-sweep | 20.88 ± 0.22 | 29.92 ± 0.36 | +9.05 ± 0.23 |
| Two-fold calibration | 11.30 ± 0.27 | 19.82 ± 0.25 | +8.52 ± 0.48 |
As shown in the table, two-fold calibration reduces absolute target gains for both strategies but retains most (94%) of the contrastive advantage observed under per-case best-of-sweep (8.52 / 9.05). These results are consistent with our results in the original paper.
Independent scorers on the same edits. To evaluate the robustness of the results across different scorers, we selected 200 cases from 20 target concepts and use CLIP-L, CLIP-H, and DFN to evaluate the same edited images from these 200 cases. The table below shows results across all three generation seeds, using the two-fold SigLIP 2 calibration described above.
| Target scorer | Mean-centered TopK, C − D |
|---|---|
| CLIP-L | +5.65 ± 0.07 |
| CLIP-H | +4.10 ± 1.33 |
| DFN | +2.40 ± 0.62 |
For mean-centered TopK, all three scorers show higher mean regional target gains for contrastive retrieval than for direct retrieval, consistent with results in our original paper.
Editing success. We also ran additional experiments to evaluate cases using two VLM judges. Specifically, on the same 200 cases, we use two VLM judges to evaluate mean-centered TopK edits generated under the original SigLIP 2 best-of-sweep protocol and two-fold SigLIP 2 calibration. The judges were shown the source image, edited image, mask, and query, while the method and strength were hidden. A target score of 0 indicates an absent or incorrect change, 1 indicates a partial or ambiguous change, and 2 indicates a clearly visible change in the target region. Success is defined as the percentage of edits scored 2. The table below shows the results.
| SigLIP 2 strength selection | Judge | Direct success (%) | Contrastive success (%) |
|---|---|---|---|
| Per-case best-of-sweep | Claude Haiku 5.5 | 59.56 ± 2.55 | 66.84 ± 1.78 |
| Per-case best-of-sweep | GPT-6 Luna | 65.82 ± 3.06 | 70.73 ± 2.80 |
| Two-fold calibration | Claude Haiku 5.5 | 49.75 ± 3.66 | 56.68 ± 1.78 |
| Two-fold calibration | GPT-6 Luna | 57.53 ± 2.50 | 63.45 ± 2.54 |
Under both strength-selection protocols, both judges report higher success rates for contrastive retrieval than for direct retrieval, consistent with our results in the original paper.
We have described the additional evaluation protocol in Section 4.1 and summarized the independent encoder and VLM findings in Section 4.4, including the calibration–test comparison in Table 2. The fixed-strength and two-fold calibration results, additional independent encoder comparisons, and VLM evaluation are provided in Appendix D.3 (Tables 21–23) of the revised manuscript.
[1] Radford et al. Learning Transferable Visual Models From Natural Language Supervision. ICML, 2021.
[2] Fang et al. Data Filtering Networks. ICLR, 2024.
[3] Cherti et al. Reproducible Scaling Laws for Contrastive Language-Image Learning. CVPR, 2023.
W2.1. "It is not clear how this reference should be chosen for complex edits." & Q2. "For contrastive retrieval, how is the reference query obtained in practice?"
We are sorry for the confusion. Below we first clarify how reference text is used and then provide a concrete example of applying D-Scope to a complex editing request.
The reference describes the region before editing. Our benchmark provides this phrase, and users can specify it in the same way. For example, for the target red dress, we use dress when the source prompt leaves the color unspecified. We subtract the reference embedding from the target embedding and retrieve the feature whose visual centroid best matches the resulting direction.
We also illustrate multi-attribute and multi-object edits in Appendix D.2 of the revision (Complex edits with contrastive retrieval). For example, red dress and striped dress share the reference dress and the same mask, while green sofa and striped rug use separate masks with sofa and rug as their respective references.
In practice, a linguistic parser or a small language model can translate a source prompt and a free-form edit request into D-Scope commands specifying the region, target, and reference for each change. We illustrate this process using Qwen3.5-0.8B [4] in Appendix D.2 (Deriving edit commands from a free-form request). For example, given the source prompt "a woman with long hair wearing a dress walking in a park" and the request "Put her in a red dress and dye her hair yellow, but keep the park as it is," the model returned {"region": "dress", "target": "red dress", "reference": "dress"} and {"region": "hair", "target": "yellow hair", "reference": "hair"}.
[4] Qwen Team. Qwen3.5: Towards Native Multimodal Agents. Qwen blog, 2026.
W2.2. "Comparisons with description-based SAE retrieval would also help quantify the benefit of avoiding per-feature text annotations."
We compare D-Scope with offline feature descriptions generated by GPT-6 Luna and Claude Haiku 5.5. Each model describes a feature using its eight highest-activating patches. D-Scope instead retrieves features from activation-weighted visual centroids constructed from 64 patches.
The following table compares the three feature indices under direct and contrastive retrieval. Values are mean ± sample SD across three seeds, with regional gains, LPIPS-out, and their SDs multiplied by 10³. CLIP-L independently evaluates regional target gain.
| Retrieval | Feature index | Region SigLIP Δ ↑ | Region CLIP-L Δ ↑ | LPIPS-out ↓ |
|---|---|---|---|---|
| Direct | D-Scope visual | 17.95 ± 0.26 | 12.24 ± 0.65 | 142.29 ± 3.80 |
| Direct | GPT-6 Luna descriptions | 5.03 ± 0.41 | 5.11 ± 0.53 | 135.88 ± 1.79 |
| Direct | Claude Haiku 5.5 descriptions | 3.17 ± 0.31 | 4.43 ± 0.54 | 158.71 ± 3.71 |
| Contrastive | D-Scope visual | 19.16 ± 0.81 | 17.76 ± 0.65 | 143.44 ± 4.31 |
| Contrastive | GPT-6 Luna descriptions | 4.94 ± 0.99 | 2.32 ± 1.66 | 143.43 ± 2.64 |
| Contrastive | Claude Haiku 5.5 descriptions | 3.77 ± 0.57 | 4.79 ± 0.34 | 137.22 ± 3.34 |
We can see that D-Scope achieves higher target gains than both description-based baselines. The same ranking holds under direct retrieval: D-Scope achieves a CLIP-L gain of 12.24 × 10⁻³, compared with 5.11 × 10⁻³ and 4.43 × 10⁻³ for Luna and Haiku, respectively. These results further support retrieval directly from visual evidence without per-feature descriptions under the evaluated settings.
We have included this comparison and the accompanying discussion in Appendix D.2 (Table 13), with the target-gain finding summarized in Section 4.4 of the revision.
Thank you for your valuable comments. We appreciate your recognition of our approach as
"interesting and useful"and our experiments and ablation studies as"comprehensive". Following your suggestions, we conducted additional experiments using strengths calibrated on separate concepts and scorers independent of SigLIP 2. We also compared visual retrieval with description-based SAE retrieval and clarified how reference queries can be chosen for complex edits. Below, we address your concerns and questions point by point. We have also revised the manuscript accordingly, with all changes marked in blue.W1. "SigLIP 2 is used for both feature retrieval and target scoring, and intervention strength is selected separately for each case using the same score...An independent evaluator and a fixed or validation-selected strength would help clarify this." & Q1. "Could the authors report steering results with an evaluator independent of SigLIP 2 and a strength selected on a held-out validation set?"
This is a good point. Inspired by your comment, we ran additional experiments using a new evaluation protocol. Specifically, we calibrate the steering strength on one set of concepts and one image-text encoder (i.e., CLIP-L), and then use a different image–text encoder (e.g., CLIP-H) to evaluate different methods (e.g., comparing the regional target gains of direct and contrastive retrieval) on held-out concepts. To further assess editing success, we include a separate evaluation with two VLM judges who assess whether the requested change is visible in the target region.
Experimental setting. We use the 0.6B SANA-Sprint student with two denoising steps and five layer-14 SAE dictionaries (mean-centered L1, mean-centered TopK, mean-centered JumpReLU, TopK with the None input setting, and JumpReLU with RMS normalization). In the following calibration–test comparison, SigLIP 2 retrieves features, while CLIP-L [1] selects the steering strength. DFN [2] and CLIP-H [3] measure regional target gain as the change in crop–query cosine similarity relative to the unsteered source.
Strength selection. We split the 100 target concepts into 50 calibration and 50 test concepts, with ten concepts per attribute category in each set. Each concept has ten source contexts covering both under-specified and explicit-conflict prompts. The strength-selection protocols are as follows:
Per-case best-of-sweep. For each test case and seed, we select the strength with the highest regional CLIP-L gain among the ten candidates.
Calibration and test. For each dictionary and retrieval strategy, we average the regional CLIP-L gain over the calibration cases and three seeds at each strength. We then apply the strength with the highest mean gain to every test case and seed using image-text encoders different from SigLIP 2 and CLIP-L, i.e., DFN and CLIP-H. For mean-centered TopK, both strategies select ρ = 0.30.