acceptodds
Under review as a conference paper at ICLR 2027

D-Scope: Decomposing and Steering Diffusion Transformers with Sparse Autoencoders

33 views

Abstract

Sparse autoencoders (SAEs) reveal visual structure in diffusion transformers (DiTs), but interpreting a feature does not establish whether it can be used to control generation. We introduce D-Scope (Diffusion Scope), a framework that connects feature interpretation to generation control through shared visual evidence. D-Scope aggregates SigLIP 2 embeddings of highly activating image patches into visual centroids. Matching target text descriptions against these visual centroids in the shared image-text embedding space then enables retrieval of individual features without per-feature text annotations. The underlying patches provide evidence for inspecting each selection, while spatially masked interventions test the corresponding decoder direction at varying strengths under fixed generation conditions. We characterize 150 SAEs across two model families and five layers, and introduce a benchmark of 100 target concepts with ten contexts each spanning under-specified and explicit-conflict conditions. Our empirical results show that high reconstruction fidelity can coexist with low dictionary utilization and limited visual-evidence coverage. Under per-case best-of-sweep strength selection, contrastive retrieval yields larger mean regional SigLIP 2 gains than direct retrieval across the tested steering configurations, without consistently improving outside-region preservation. D-Scope provides an inspectable framework for evaluating sparse DiT features through their visual evidence and the effects of their decoder directions on generation.

Primary area:
interpretability and explainable AI
Keywords:
Sparse Autoencoder, Diffusion Transformers, Interpretability

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_42qiOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

In this work, authors introduce D-Scope, a framework for retrieving and steering SAE features in diffusion transformers without per-feature text annotations or target-generation activations. Authors construct a visual centroid for each indexed feature by aggregating SigLIP 2 embeddings of its highly activating image patches. A target text query is then matched against these centroids, either directly or contrastively relative to a reference query, to select a decoder direction for spatially masked intervention. Authors study 150 SAEs across two one-step generators, different SAE families, input settings, and layers. On a 1000 sample steering benchmark, they compare direct and contrastive retrieval in terms of regional target alignment and outside-region preservation under per-case best-of-sweep strength selection. Authors additionally demonstrate that reconstruction fidelity alone does not characterize dictionary utilization or the availability of visual evidence.

Critical strengths
  • Overall, I find the proposed approach interesting and useful. Matching text queries against visual evidence removes the need to annotate individual SAE features with text descriptions or collect target-generation activations for feature selection. I believe this is a useful contribution for applying SAE dictionaries to fine-grained generation control.
  • Experiments and ablation studies are comprehensive, covering different SAE families, input settings, layers, etc. The finding that high reconstruction accuracy can coexist with low dictionary utilization and limited visual-evidence coverage is interesting. I appreciate that authors evaluate target alignment and outside-region preservation separately, rather than treating successful steering as sufficient evidence of a localized edit.
Critical weaknesses
  • My primary concern is with the evaluation of target alignment. SigLIP 2 is used for both feature retrieval and target scoring, and intervention strength is selected separately for each case using the same score. Authors clearly disclose this. Nevertheless, these results characterize best-of-sweep performance, and it is not clear how much of the gain would remain with a strength chosen independently of the test outcome. An independent evaluator and a fixed or validation-selected strength would help clarify this.
  • The practical scope of contrastive retrieval could be discussed more clearly. The benchmark provides the reference query, which describes either the generic object or the competing source attribute. It is not clear how this reference should be chosen for complex edits. Comparisons with description-based SAE retrieval would also help quantify the benefit of avoiding per-feature text annotations.
  • Please see the questions below as well.
Questions for authors

Questions:

  • Could the authors report steering results with an evaluator independent of SigLIP 2 and a strength selected on a held-out validation set? This would help assess whether the gains persist beyond the current best-of-sweep protocol.
  • For contrastive retrieval, how is the reference query obtained in practice? Appendix D.1 specifies a generic object query for under-specified contexts and the competing source attribute for explicit-conflict contexts. How would this choice extend to more complex scenes or edits that do not have a simple attribute-object structure? Could the reference be derived automatically from the source prompt or a caption of the target region?
  • For the SANA teacher, the student-trained dictionaries are reused and the same direction is injected at every denoising step. Could the authors comment on why is it reasonable to reuse the same student-trained SAE dictionary across all denoising steps, given that the representations may change with the timestep? Did the authors examine at which timesteps the intervention is most effective? In particular, could restricting the intervention to a subset of timesteps improve outside-region preservation?
  • For outside-region preservation, could the authors additionally report masked MSE or PSNR? Since color changes are an important part of the benchmark, a complementary pixel-wise metric would help characterize changes that may not be fully reflected by LPIPS.
  • In Section 4.2, the example of EV = 0.9997 with only 1.5% of features utilized is interesting. In particular I'm surprised that the SAE reconstruction error is so small. What should be the takeaway here? That the superposition hypothesis is pretty accurate for Nitro Layer 2?

Suggestions:

  • Figure 3(c) reports the change in utilization due to centering, rather than absolute utilization. Although this is stated in the caption, I would recommend emphasizing it in the discussion so that the small changes do not read as poor absolute utilization. The same distinction applies to the target-gain changes in Figure 4(c).
  • A comparison against description-based SAE retrieval would be particularly informative, since avoiding per-feature annotations is a main motivation. Comparisons with non-oracle editing methods would further clarify the trade-off between target alignment and preservation.
  • Could the authors comment on how their technique would extend to SAEs trained directly on multi-step diffusion models, where features may change across denoising timesteps?
9 comments

Thank you for your valuable comments. We appreciate your recognition of our approach as "interesting and useful" and our experiments and ablation studies as "comprehensive". Following your suggestions, we conducted additional experiments using strengths calibrated on separate concepts and scorers independent of SigLIP 2. We also compared visual retrieval with description-based SAE retrieval and clarified how reference queries can be chosen for complex edits. Below, we address your concerns and questions point by point. We have also revised the manuscript accordingly, with all changes marked in blue.


W1. "SigLIP 2 is used for both feature retrieval and target scoring, and intervention strength is selected separately for each case using the same score...An independent evaluator and a fixed or validation-selected strength would help clarify this." & Q1. "Could the authors report steering results with an evaluator independent of SigLIP 2 and a strength selected on a held-out validation set?"

This is a good point. Inspired by your comment, we ran additional experiments using a new evaluation protocol. Specifically, we calibrate the steering strength on one set of concepts and one image-text encoder (i.e., CLIP-L), and then use a different image–text encoder (e.g., CLIP-H) to evaluate different methods (e.g., comparing the regional target gains of direct and contrastive retrieval) on held-out concepts. To further assess editing success, we include a separate evaluation with two VLM judges who assess whether the requested change is visible in the target region.

Experimental setting. We use the 0.6B SANA-Sprint student with two denoising steps and five layer-14 SAE dictionaries (mean-centered L1, mean-centered TopK, mean-centered JumpReLU, TopK with the None input setting, and JumpReLU with RMS normalization). In the following calibration–test comparison, SigLIP 2 retrieves features, while CLIP-L [1] selects the steering strength. DFN [2] and CLIP-H [3] measure regional target gain as the change in crop–query cosine similarity relative to the unsteered source.

Strength selection. We split the 100 target concepts into 50 calibration and 50 test concepts, with ten concepts per attribute category in each set. Each concept has ten source contexts covering both under-specified and explicit-conflict prompts. The strength-selection protocols are as follows:

  • Per-case best-of-sweep. For each test case and seed, we select the strength with the highest regional CLIP-L gain among the ten candidates.

  • Calibration and test. For each dictionary and retrieval strategy, we average the regional CLIP-L gain over the calibration cases and three seeds at each strength. We then apply the strength with the highest mean gain to every test case and seed using image-text encoders different from SigLIP 2 and CLIP-L, i.e., DFN and CLIP-H. For mean-centered TopK, both strategies select ρ = 0.30.

The first table below compares the two protocols on the same test cases for mean-centered TopK. Here, "C − D" denotes contrastive minus direct retrieval, paired by case and seed. We report the mean ± sample standard deviation (SD) across the three seed means. Regional gains, LPIPS-out, and their SDs are scaled by 10³ in the tables.

Strength selectionIndependent target scorerDirectContrastivePaired C − D
Per-case best-of-sweepDFN21.04 ± 0.8535.17 ± 1.18+14.13 ± 0.56
Calibration–testDFN12.32 ± 0.5827.23 ± 0.68+14.91 ± 0.29
Per-case best-of-sweepCLIP-H23.71 ± 0.7343.23 ± 0.96+19.52 ± 0.55
Calibration–testCLIP-H12.74 ± 0.4833.88 ± 0.70+21.14 ± 0.78

Across these cases, contrastive retrieval achieves higher mean gains for all five dictionaries under both protocols and both scorers. These results are consistent with our results in the original paper.

Fixed strength and held-out concepts under the original metric. Following your suggestion, we also ran additional experiments to compare the strategies for mean-centered TopK using Region SigLIP Δ. At fixed ρ = 0.30, we compute "C − D" (i.e., contrastive minus direct retrieval) from the mean gains of the two strategies. Best-of-sweep maximizes Region SigLIP Δ for each case and seed. Two-fold calibration maximizes mean Region SigLIP Δ on 50 concepts, applies the selected strength to the other 50, and then swaps the groups. Both strategies use the same strength within each test fold (0.30 in one fold and 0.40 in the other).

Strength selectionDirectContrastiveC − D
Fixed ρ = 0.3011.45 ± 0.3420.13 ± 0.17+8.67 ± 0.50
Per-case best-of-sweep20.88 ± 0.2229.92 ± 0.36+9.05 ± 0.23
Two-fold calibration11.30 ± 0.2719.82 ± 0.25+8.52 ± 0.48

As shown in the table, two-fold calibration reduces absolute target gains for both strategies but retains most (94%) of the contrastive advantage observed under per-case best-of-sweep (8.52 / 9.05). These results are consistent with our results in the original paper.

Independent scorers on the same edits. To evaluate the robustness of the results across different scorers, we selected 200 cases from 20 target concepts and use CLIP-L, CLIP-H, and DFN to evaluate the same edited images from these 200 cases. The table below shows results across all three generation seeds, using the two-fold SigLIP 2 calibration described above.

Target scorerMean-centered TopK, C − D
CLIP-L+5.65 ± 0.07
CLIP-H+4.10 ± 1.33
DFN+2.40 ± 0.62

For mean-centered TopK, all three scorers show higher mean regional target gains for contrastive retrieval than for direct retrieval, consistent with results in our original paper.

Editing success. We also ran additional experiments to evaluate cases using two VLM judges. Specifically, on the same 200 cases, we use two VLM judges to evaluate mean-centered TopK edits generated under the original SigLIP 2 best-of-sweep protocol and two-fold SigLIP 2 calibration. The judges were shown the source image, edited image, mask, and query, while the method and strength were hidden. A target score of 0 indicates an absent or incorrect change, 1 indicates a partial or ambiguous change, and 2 indicates a clearly visible change in the target region. Success is defined as the percentage of edits scored 2. The table below shows the results.

SigLIP 2 strength selectionJudgeDirect success (%)Contrastive success (%)
Per-case best-of-sweepClaude Haiku 5.559.56 ± 2.5566.84 ± 1.78
Per-case best-of-sweepGPT-6 Luna65.82 ± 3.0670.73 ± 2.80
Two-fold calibrationClaude Haiku 5.549.75 ± 3.6656.68 ± 1.78
Two-fold calibrationGPT-6 Luna57.53 ± 2.5063.45 ± 2.54

Under both strength-selection protocols, both judges report higher success rates for contrastive retrieval than for direct retrieval, consistent with our results in the original paper.

We have described the additional evaluation protocol in Section 4.1 and summarized the independent encoder and VLM findings in Section 4.4, including the calibration–test comparison in Table 2. The fixed-strength and two-fold calibration results, additional independent encoder comparisons, and VLM evaluation are provided in Appendix D.3 (Tables 21–23) of the revised manuscript.

[1] Radford et al. Learning Transferable Visual Models From Natural Language Supervision. ICML, 2021.

[2] Fang et al. Data Filtering Networks. ICLR, 2024.

[3] Cherti et al. Reproducible Scaling Laws for Contrastive Language-Image Learning. CVPR, 2023.

W2.1. "It is not clear how this reference should be chosen for complex edits." & Q2. "For contrastive retrieval, how is the reference query obtained in practice?"

We are sorry for the confusion. Below we first clarify how reference text is used and then provide a concrete example of applying D-Scope to a complex editing request.

The reference describes the region before editing. Our benchmark provides this phrase, and users can specify it in the same way. For example, for the target red dress, we use dress when the source prompt leaves the color unspecified. We subtract the reference embedding from the target embedding and retrieve the feature whose visual centroid best matches the resulting direction.

We also illustrate multi-attribute and multi-object edits in Appendix D.2 of the revision (Complex edits with contrastive retrieval). For example, red dress and striped dress share the reference dress and the same mask, while green sofa and striped rug use separate masks with sofa and rug as their respective references.

In practice, a linguistic parser or a small language model can translate a source prompt and a free-form edit request into D-Scope commands specifying the region, target, and reference for each change. We illustrate this process using Qwen3.5-0.8B [4] in Appendix D.2 (Deriving edit commands from a free-form request). For example, given the source prompt "a woman with long hair wearing a dress walking in a park" and the request "Put her in a red dress and dye her hair yellow, but keep the park as it is," the model returned {"region": "dress", "target": "red dress", "reference": "dress"} and {"region": "hair", "target": "yellow hair", "reference": "hair"}.

[4] Qwen Team. Qwen3.5: Towards Native Multimodal Agents. Qwen blog, 2026.

W2.2. "Comparisons with description-based SAE retrieval would also help quantify the benefit of avoiding per-feature text annotations."

We compare D-Scope with offline feature descriptions generated by GPT-6 Luna and Claude Haiku 5.5. Each model describes a feature using its eight highest-activating patches. D-Scope instead retrieves features from activation-weighted visual centroids constructed from 64 patches.

The following table compares the three feature indices under direct and contrastive retrieval. Values are mean ± sample SD across three seeds, with regional gains, LPIPS-out, and their SDs multiplied by 10³. CLIP-L independently evaluates regional target gain.

RetrievalFeature indexRegion SigLIP Δ ↑Region CLIP-L Δ ↑LPIPS-out ↓
DirectD-Scope visual17.95 ± 0.2612.24 ± 0.65142.29 ± 3.80
DirectGPT-6 Luna descriptions5.03 ± 0.415.11 ± 0.53135.88 ± 1.79
DirectClaude Haiku 5.5 descriptions3.17 ± 0.314.43 ± 0.54158.71 ± 3.71
ContrastiveD-Scope visual19.16 ± 0.8117.76 ± 0.65143.44 ± 4.31
ContrastiveGPT-6 Luna descriptions4.94 ± 0.992.32 ± 1.66143.43 ± 2.64
ContrastiveClaude Haiku 5.5 descriptions3.77 ± 0.574.79 ± 0.34137.22 ± 3.34

We can see that D-Scope achieves higher target gains than both description-based baselines. The same ranking holds under direct retrieval: D-Scope achieves a CLIP-L gain of 12.24 × 10⁻³, compared with 5.11 × 10⁻³ and 4.43 × 10⁻³ for Luna and Haiku, respectively. These results further support retrieval directly from visual evidence without per-feature descriptions under the evaluated settings.

We have included this comparison and the accompanying discussion in Appendix D.2 (Table 13), with the target-gain finding summarized in Section 4.4 of the revision.

Reviewer_oKJ1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a Sparse AutoEncoder (SAE) framework for feature interpretation and steering with text descriptions by matching the visual centroids with text in the shared image-text embedding space. It retrieves SAE decoder directions by matching target text queries to activation-weighted visual centroids of highly activating image patches. Experiments use 150 SAEs across two model families and five layers. The paper introduces a benchmark of 100 target concepts across five attribute categories with ten contexts. Experiments are shown on image generation/editing using SANA-Sprint, SANA teacher and Nitro-1-PixArt.

Critical strengths
  • The proposed method enables feature selection without needing any per-feature text descriptions and no target-generation activations. The selected features can be inspected through the patches that support it.

  • The positive target gains extending to the multi-step SANA teacher from the student-trained dictionaries is a plus.

  • The SAE training study is large and systematic. The practical recommendations in Table 2 are useful.

Critical weaknesses
  • The novelty is incremental as pooling the embeddings of top-activating regions for text-based retrieval is studied in SemanticLens (Dreyer et al., 2025). The observation that high reconstruction can coexist with many inactive features was reported for FLUX SAEs (Shabalin et al., 2025). Contrastive retrieval is mainly a text-embedding difference. The contribution seems mainly applying these to DiT SAE steering plus the dictionary study.

  • Evaluation uses SigLIP2 while it is still used for retrieving the feature and selecting the steering strength. It may hide the biases present in the evaluation. A separate evaluation using a vision-language judge is missing.

  • The method requires masks at inference. So, a fair comparison is with editing methods that use masks or inpainting methods.

  • The paper doesn’t compare against prior SAE feature-selection approaches, such as the paired-activation selection of Surkov et al. (2024).

  • The paper mentions that it needs only the text query, but it also needs a SAM 3 mask of the target and a reference query for contrastive retrieval. It doesn’t study the sensitivity of these reference queries to the performance.

  • The related work should discuss slider-based concept control methods such as Attribute Control (Baumann et al., CVPR 2025), Text Slider (Chiu et al., WACV 2026), Prompt Sliders (Sridhar et. al., ECCVW 2024), and All-in-One Slider (Ye et al., CVPR 2026), which also provide continuous, fine-grained attribute control from text. Concept Sliders and FluxSpace are only cited in the appendix, and comparing against at least one of these methods would better position D-Scope’s contribution.

  • Retrieval quality of the patches is not evaluated. There is no measure of whether the retrieved feature actually represents the queried concept (e.g., VLM judgment of the features).

  • Qualitative results are limited and selected. The steering figures (Fig. 1, 2f, 4a) show a few chosen examples, and there are no steering failure cases. The edits in Figure 4 are inconsistent and sometimes leak outside the mask.

Questions for authors
  • See weaknesses above.
  • What is the Region SigLIP Δ\Delta and LPIPS-out of a randomly chosen indexed feature (and of a mismatched retrieved feature) under the same best-of-sweep protocol?
  • Why does LPIPS-out remain 0.15-0.25 despite masking? Does the change come from attention propagating the edit, and could it be reduced?
  • How do results change with JJ, patch size, or pooling over all activating tokens instead of the maximum token per image?
11 comments

Thank you for your valuable comments. We appreciate your recognition of our "large and systematic" SAE training study and your observation that the selected features "can be inspected through the patches". Following your suggestions, we clarified the contribution relative to prior work, added independent evaluation and comparisons with prior methods, and conducted additional ablations. Below, we address your concerns and questions point by point. We have also revised the manuscript accordingly, with all changes marked in blue.

W1. "The novelty is incremental ... is studied in SemanticLens (Dreyer et al., 2025). ... was reported for FLUX SAEs (Shabalin et al., 2025). Contrastive retrieval is mainly a text-embedding difference. The contribution seems ..."

Thank you for mentioning this. We would like to clarify that implementation details like "pooling the embeddings of top-activating regions"in SemanticLens (Dreyer et al., 2025) [1] and observations like "high reconstruction can coexist with many inactive features" in FLUX SAEs (Shabalin et al., 2025) [2] are not the focus of our paper. Instead, D-Scope is intended to make three main contributions: (1) a general inspectable method for selecting an intervention direction, (2) a controlled benchmark for evaluating feature selection through generation, and (3) a systematic dictionary study. Specifically:

  1. An inspectable method for selecting an intervention direction. D-Scope uses the requested attribute change and visual evidence from a trained DiT SAE dictionary to select a single decoder direction. The selected feature provides both the image patches supporting the selection and the direction applied within the source object mask, making the intervention easier to interpret. Feature selection relies on an offline visual index and requires neither per-feature captions nor target-generation activations.

  2. A controlled benchmark for evaluating feature selection through generation. We introduce a new benchmark with 1,000 cases covering 100 target concepts, each evaluated in ten source contexts. The benchmark tests the same requested edit across scenes where the source attribute is either unspecified or conflicts with the target attribute. Under a single-feature intervention budget, it measures both the requested regional change and its effect on the surrounding scene. This provides a common evaluation setting for comparing feature selectors and SAE dictionaries.

  3. A systematic dictionary study. We appreciate the reviewer's recognition of its scale and systematic design. We analyze 150 SAEs and evaluate the middle-layer dictionaries to examine how training choices relate to feature availability and editing behavior.

SemanticLens (Dreyer et al., 2025) [1] retrieves components whose activating examples match a queried concept. D-Scope selects and evaluates a single SAE decoder direction for a specified local change to a source image. For example, a feature whose activating patches show red dresses may match "red dress". However, this match does not establish that adding its decoder direction will turn a green dress red. We evaluate the selected feature by applying its direction to the source dress and measuring the resulting attribute change and preservation of the surrounding scene.

FLUX SAEs (Shabalin et al., 2025) [2] examine what change a selected feature produces. D-Scope addresses which feature to select for a change specified in advance, using the requested attribute and offline visual evidence without requiring per-feature captions or target-generation activations.

We have clarified this distinction from SemanticLens and FLUX SAE steering in Section 2 of the revision as suggested.

[1] Dreyer et al. Mechanistic Understanding and Validation of Large AI Models with SemanticLens. Nature Machine Intelligence, 2025.

[2] Shabalin et al. Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning. Mechanistic Interpretability for Vision Workshop at CVPR, 2025.

W2. "Evaluation uses SigLIP2... It may hide the biases present in the evaluation. A separate evaluation using a vision-language judge is missing.""

This is a good suggestion. Following the suggestion, we conducted additional experiments using two VLM judges to assess whether the requested attribute is visible in the target region.

VLM evaluation. We selected 200 cases spanning 20 concepts before VLM scoring and compare methods across three seeds. Claude Haiku 5.5 and GPT-6 Luna each receive the source image, edited image, target mask, and query, while the retrieval method and steering strength are hidden. A score of 0 indicates an absent or incorrect target attribute, 1 indicates a partial or ambiguous attribute, and 2 indicates that the requested attribute is clearly visible on the target object within the mask. Target success is the percentage of outputs scored 2.

Strength selection. SigLIP 2 retrieves features for both methods. The VLMs evaluate outputs selected under two protocols:

  • Per-case best-of-sweep. For each retrieval method, case, and seed, we select the strength with the highest Region SigLIP Δ among ten candidates.

  • Two-fold calibration. Using the full benchmark's 100 concepts, we select one strength per retrieval method by maximizing mean Region SigLIP Δ on 50 concepts. We apply it to every case and seed from the other 50 concepts, then swap the groups. Both retrieval methods select the same strength within each test fold: ρ = 0.40 in one fold and ρ = 0.30 in the other.

The table below reports VLM success rates. Values are mean ± sample SD across three seeds.

Output judgedClaude Haiku 5.5 success (%)GPT-6 Luna success (%)
Unsteered source17.94 ± 2.3417.77 ± 1.34
Direct, per-case best-of-sweep59.56 ± 2.5565.82 ± 3.06
Contrastive, per-case best-of-sweep66.84 ± 1.7870.73 ± 2.80
Direct, two-fold calibration49.75 ± 3.6657.53 ± 2.50
Contrastive, two-fold calibration56.68 ± 1.7863.45 ± 2.54

Results show that calibration lowers absolute success rates for both methods, but both judges continue to favor contrastive retrieval. With calibrated strengths, its mean success advantage is 6.9 percentage points for Claude and 5.9 for GPT, consistent with the original target-alignment results.

We have described the VLM evaluation in Section 4.1, summarized its findings in Section 4.4, and reported the success rates in Appendix D.3 (Table 23) of the revision.


W3. "The method requires masks at inference. So, a fair comparison is with editing methods that use masks or inpainting methods."

Thank you for mentioning this. Following your suggestion, we conducted additional experiments with a masked FluxSpace adaptation [4], a baseline editing method, to compare regional target control at the same level of outside-region change.

Comparison with masked FluxSpace. We adapt FluxSpace [4] to use the same SAM 3 masks, source prompts, noise seeds, and two-step SANA-Sprint generator. D-Scope uses the mean-centered TopK dictionary. On benchmark cases, we evaluate ten strengths per method and linearly interpolate their mean strength curves to compare target gains at equal mean LPIPS-out.

The table below reports regional target gains at mean LPIPS-out = 0.15.

MethodRegion SigLIP Δ ↑Regional CLIP-L gain ↑
FluxSpace with the same SAM 3 mask0.01020.0081
D-Scope0.02010.0196

At this preservation level, D-Scope achieves 1.98× the Region SigLIP Δ and 2.42× the independent CLIP-L gain of masked FluxSpace.

"Oracle Dense" as a strong target-informed baseline. We would like to clarify that in our paper, Oracle Dense uses the same masked tokens (from SAM 3) but derives a target-informed direction from paired source and target activations. It therefore serves as a strong target-informed baseline. With contrastive retrieval and the best dictionary in each setting, D-Scope recovers 86%, 78%, and 86% of Oracle Dense's mean Region SigLIP Δ on SANA-Sprint, Nitro-1-PixArt, and the SANA teacher, respectively (Fig. 4b). This highlights the promising performance of D-Scope, i.e., being able to recover most of the target-informed baseline's regional target-alignment gain.

Masks are automatically generated. We would also like to emphasize that while D-Scope requires an object mask, it is automatically generated by SAM 3 [3] from the source image and object name, without the need to finetune the model. The mask specifies where to intervene, while D-Scope selects the SAE direction for the requested attribute change.

We have included the results and discussion in Appendix D.2 (Table 15) of the revision as suggested.

[3] Carion et al. SAM 3: Segment Anything with Concepts. ICLR, 2026.

[4] Dalva et al. FluxSpace: Disentangled Semantic Editing in Rectified Flow Models. CVPR, 2025.

W4. "...compare against prior SAE feature-selection approaches..., such as the paired-activation selection of Surkov et al. (2024)."

This is a good suggestion. Accordingly, we implemented two single-feature adaptations of the paired-activation selector of Surkov et al. [5], using the same single-feature intervention budget as D-Scope.

Paired-activation selection requires an additional generation from the full target prompt, using the same initial noise as the source. D-Scope instead selects its features from the requested attribute and offline visual evidence, without target-generation activations. We compare Top-1 Add, which selects the largest positive normalized activation change, and Top-1 Signed, which selects the largest absolute change and reverses the decoder direction when the activation decreases.

The table below reports mean ± sample SD across three seeds, with all metrics and their SDs multiplied by 10³.

Feature selectorRegion SigLIP Δ ↑Regional CLIP-L gain ↑Regional DFN gain ↑LPIPS-out ↓
Paired activations, Top-1 Add (baseline)11.84 ± 1.399.52 ± 0.4919.56 ± 1.89121.67 ± 2.87
Paired activations, Top-1 Signed (baseline)5.44 ± 1.314.88 ± 1.4810.26 ± 2.70102.86 ± 3.26
D-Scope, direct17.95 ± 0.2612.24 ± 0.6526.90 ± 1.15142.29 ± 3.80
D-Scope, contrastive19.16 ± 0.8117.76 ± 0.6529.21 ± 1.74143.44 ± 4.31

In this comparison, contrastive retrieval achieves 1.62× the Region SigLIP Δ and 1.87× the independent CLIP-L gain of Top-1 Add. Both paired selectors produce lower LPIPS-out, indicating less change outside the target region.

We have included these single-feature comparisons and the accompanying discussion in Appendix D.2 (Table 16) of the revision as suggested.

[5] Surkov et al. One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Models. NeurIPS, 2025.


W5. "The paper mentions that it needs only the text query,.... It doesn’t study the sensitivity of these reference queries to the performance."

Thank you for mentioning this. Below we first explain these reference queries' roles and report an additional ablation of masking strategies that we conducted following your suggestion.

Reference queries. Direct retrieval uses the target query alone, while contrastive retrieval additionally uses a reference query describing the object or its source appearance. For example, for the target "red dress", the benchmark provides "dress" when the source color is unspecified.

Using these reference rules, contrastive retrieval achieves higher mean Region SigLIP Δ than direct retrieval across the tested SAE families (L1, TopK, and JumpReLU) and preprocessing settings under the original best-of-sweep protocol (Table 1 of the paper).

Mask sensitivity. For both retrieval strategies, SAM 3 [3] automatically generates the intervention mask from the unsteered source image and object name, without manually drawn masks. Following your suggestion, we ran additional experiments to compare its mask with Grounded SAM 2.1 [6] and whole-image interventions. We hold the retrieved feature and strength fixed and evaluate all conditions in the table using the same source SAM 3 region. The last condition matches the total update norm to that of the SAM 3 intervention.

The table reports mean ± sample standard deviation (SD) across three seeds. Regional gains, LPIPS-out, and their SDs are multiplied by 10³.

Intervention regionRegion SigLIP Δ ↑Regional CLIP-L gain ↑LPIPS-out ↓
SAM 3 mask19.16 ± 0.8117.76 ± 0.65143.44 ± 4.31
Grounded SAM 2.1 mask18.36 ± 0.5417.38 ± 0.47146.29 ± 5.22
Whole image12.34 ± 1.5616.64 ± 0.97612.76 ± 14.39
Whole image, matched update norm11.25 ± 0.6810.75 ± 0.98283.31 ± 10.49

Results show that the two automatic segmenters produce closely overlapping masks and similar mean regional gains. Scoring on the union of their masks also gives similar regional CLIP-L gains: 16.91 for SAM 3 and 16.95 for Grounded SAM 2.1, on the same 10³ reporting scale. These results suggest that the performance is not too sensitive to the masks as long as they are sufficiently accurate. Removing the mask increases LPIPS-out to 4.27 times the SAM 3 value (third row).

We have included the masking-strategy comparison and accompanying discussion in Appendix D.2 (Table 18) of the revision as suggested.

[3] Carion et al. SAM 3: Segment Anything with Concepts. ICLR, 2026.

[6] IDEA Research. Grounded SAM 2, including SAM 2.1 support. Official implementation, 2024.

Reviewer_LkiPOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces D-Scope, a framework connecting the visual interpretation of sparse autoencoder (SAE) features to spatially localized steering in diffusion transformers. It constructs an activation-weighted SigLIP 2 centroid from each feature's highly activating image patches, retrieves a single decoder direction through direct or contrastive matching of text embeddings against the visual centroids, and injects that direction into masked residual-stream tokens. The paper studies 150 SAE configurations across two generator families and five layers, and evaluates steering on a 1,000-case benchmark spanning five attribute categories and two source-context regimes. Contrastive retrieval improves mean regional target alignment over direct retrieval under per-case best-of-sweep strength selection.

Critical strengths
  1. The paper provides a clear and inspectable connection between feature retrieval and intervention. Each retrieved centroid is linked to both supporting image patches and a specific SAE decoder direction, without requiring per-feature textual descriptions or target-generation activations for feature selection.
  2. The dictionary study goes beyond reconstruction quality by measuring utilization, visual coherence, and evidence coverage across 150 configurations.
  3. The steering benchmark separates regional target alignment from outside-region preservation and covers both under-specified and explicit-conflict prompts. The paper reports three generation seeds, provides detailed protocols, and explicitly discloses best-of-sweep selection.
Critical weaknesses
  1. The main steering evaluation uses the same representation model SigLIP 2 for feature retrieval, strength selection, and target scoring. SigLIP 2 serves all three roles, with the best output selected from ten intervention strengths for each case using the evaluation metric itself. Nevertheless, it does not establish how well a transferable strength-selection rule would perform, nor does it independently validate perceptual editing success.

  2. Oracle Prompt and Oracle Dense are useful target-informed references, but neither is a source-only alternative operating under the same information constraints as D-Scope. Although Section 2 and Appendix A discuss related visual-evidence aggregation approaches, the experiments do not isolate the individual contributions of activation weighting, patch selection, and the evidence threshold.

Questions for authors
  1. How much of the advantage of contrastive retrieval remains when the intervention strength is fixed globally or selected only on held-out concepts?
  2. How much does activation-weighted centroid construction improve performance over uniform pooling?
  3. How sensitive are the results to the patch size and the minimum evidence requirement for feature indexing?
11 comments

Thank you for your valuable comments. We appreciate your recognition of the "clear and inspectable connection between feature retrieval and intervention" in our work. Following your suggestions, we ran additional experiments using a new evaluation protocol and conducted ablations to assess the individual contributions of the method's components. Below, we address your concerns and questions point by point. We have also revised the manuscript accordingly, with all changes marked in blue.

W1. "SigLIP 2 serves all three roles... how well a transferable strength-selection rule would perform, nor does it independently validate perceptual editing success."

This is a good point. Inspired by your comment, we ran additional experiments using a new evaluation protocol. Specifically, we calibrate the steering strength on one set of concepts and one image-text encoder (i.e., CLIP-L), and then use a different image–text encoder (e.g., CLIP-H) to evaluate different methods (e.g., comparing the regional target gains of direct and contrastive retrieval) on held-out concepts. To further assess editing success, we include a separate evaluation with two VLM judges who assess whether the requested change is visible in the target region.

Experimental setting. We use the 0.6B SANA-Sprint student with two denoising steps and five layer-14 SAE dictionaries (mean-centered L1, mean-centered TopK, mean-centered JumpReLU, TopK with the None input setting, and JumpReLU with RMS normalization). In the following calibration–test comparison, SigLIP 2 retrieves features, while CLIP-L [1] selects the steering strength. DFN [2] and CLIP-H [3] measure regional target gain as the change in crop–query cosine similarity relative to the unsteered source.

Strength selection. We split the 100 target concepts into 50 calibration and 50 test concepts, with ten concepts per attribute category in each set. Each concept has ten source contexts covering both under-specified and explicit-conflict prompts. The strength-selection protocols are as follows:

  • Per-case best-of-sweep. For each test case and seed, we select the strength with the highest regional CLIP-L gain among the ten candidates.
  • Calibration and test. For each dictionary and retrieval strategy, we average the regional CLIP-L gain over the calibration cases and three seeds at each strength. We then apply the strength with the highest mean gain to every test case and seed using image-text encoders different from SigLIP 2 and CLIP-L, i.e., DFN and CLIP-H. For mean-centered TopK, both strategies select ρ = 0.30.

The first table below compares the two protocols on the same test cases for mean-centered TopK. Here, "C − D" denotes contrastive minus direct retrieval, paired by case and seed. We report the mean ± sample standard deviation (SD) across the three seeds. Regional gains, LPIPS-out, and their SDs are scaled by 10³ in the tables.

Strength selectionIndependent target scorerDirectContrastivePaired C − D
Per-case best-of-sweepDFN21.04 ± 0.8535.17 ± 1.18+14.13 ± 0.56
Calibration–testDFN12.32 ± 0.5827.23 ± 0.68+14.91 ± 0.29
Per-case best-of-sweepCLIP-H23.71 ± 0.7343.23 ± 0.96+19.52 ± 0.55
Calibration–testCLIP-H12.74 ± 0.4833.88 ± 0.70+21.14 ± 0.78

Across these cases, contrastive retrieval achieves higher mean gains for all five dictionaries under both protocols and both scorers. These results are consistent with our results in the original paper.

Fixed strength and held-out concepts under the original metric (Q1). Following your suggestion, we also ran additional experiments to compare the strategies for mean-centered TopK using Region SigLIP Δ. At fixed ρ = 0.30, we compute "C − D" (i.e., contrastive minus direct retrieval) from the mean gains of the two strategies. Best-of-sweep maximizes Region SigLIP Δ for each case and seed. Two-fold calibration maximizes mean Region SigLIP Δ on 50 concepts, applies the selected strength to the other 50, and then swaps the groups. Both strategies use the same strength within each test fold (0.30 in one fold and 0.40 in the other).

Strength selectionDirectContrastiveC − D
Fixed ρ = 0.3011.45 ± 0.3420.13 ± 0.17+8.67 ± 0.50
Per-case best-of-sweep20.88 ± 0.2229.92 ± 0.36+9.05 ± 0.23
Two-fold calibration11.30 ± 0.2719.82 ± 0.25+8.52 ± 0.48

As shown in the table, two-fold calibration reduces absolute target gains for both strategies but retains most (94%) of the contrastive advantage observed under per-case best-of-sweep (8.52 / 9.05). These results are consistent with our results in the original paper.

Independent scorers on the same edits. To evaluate the robustness of the results across different scorers, we selected 200 cases from 20 target concepts and use CLIP-L, CLIP-H, and DFN to evaluate the same edited images from these 200 cases. The table below shows results across all three generation seeds, using the two-fold SigLIP 2 calibration described above.

Target scorerMean-centered TopK, C − D
CLIP-L+5.65 ± 0.07
CLIP-H+4.10 ± 1.33
DFN+2.40 ± 0.62

For mean-centered TopK, all three scorers show higher mean regional target gains for contrastive retrieval than for direct retrieval, consistent with results in our original paper.

Editing success. We also ran additional experiments to evaluate cases using two VLM judges. Specifically, on the same 200 cases, we use two VLM judges to evaluate mean-centered TopK edits generated under the original SigLIP 2 best-of-sweep protocol and two-fold SigLIP 2 calibration. The judges were shown the source image, edited image, mask, and query, while the method and strength were hidden. A target score of 0 indicates an absent or incorrect change, 1 indicates a partial or ambiguous change, and 2 indicates a clearly visible change in the target region. Success is defined as the percentage of edits scored 2. The table below shows the results.

SigLIP 2 strength selectionJudgeDirect success (%)Contrastive success (%)
Per-case best-of-sweepClaude Haiku 5.559.56 ± 2.5566.84 ± 1.78
Per-case best-of-sweepGPT-6 Luna65.82 ± 3.0670.73 ± 2.80
Two-fold calibrationClaude Haiku 5.549.75 ± 3.6656.68 ± 1.78
Two-fold calibrationGPT-6 Luna57.53 ± 2.5063.45 ± 2.54

Under both strength-selection protocols, the two VLM judges report higher success rates for contrastive retrieval than for direct retrieval, consistent with the target-alignment results in the original paper. We have described the additional evaluation protocol in Section 4.1 and summarized the independent encoder and VLM findings in Section 4.4, including the calibration–test comparison in Table 2. The fixed-strength and two-fold calibration results, additional independent encoder comparisons, and VLM evaluation are provided in Appendix D.3 (Tables 21–23) of the revised manuscript.

[1] Radford et al. Learning Transferable Visual Models From Natural Language Supervision. ICML, 2021.

[2] Fang et al. Data Filtering Networks. ICLR, 2024.

[3] Cherti et al. Reproducible Scaling Laws for Contrastive Language-Image Learning. CVPR, 2023.

W2. "...isolate the individual contributions of activation weighting, patch selection, and the evidence threshold."

This is a good suggestion. Following your comments, we conducted additional ablations to assess the individual contributions of the method's components.

Experimental setting. We evaluate contrastive retrieval with the layer-14 mean-centered TopK dictionary on the two-step SANA-Sprint student. These experiments use the two-fold strength calibration protocol with Region SigLIP Δ described in our response to W1 above.

Activation weighting, patch selection, and patch size Here, patch size denotes the spatial dimensions, in visual-token cells, of the square evidence patch cropped around each feature's maximally activating token (4 × 4 by default). For example, to isolate the contribution of activation weighting, we implemented a uniform-pooling baseline, which uses the same activating patches and queries as D-Scope and assigns equal weight to each patch. Both methods retrieve features from the offline index without target-generation activations or per-feature text descriptions.

The following table summarizes the effects of activation weighting, patch selection, and patch size. We report Region SigLIP Δ and LPIPS-out as mean ± sample SD across three seed means, with both metrics and their SDs multiplied by 10³.

ComponentSettingRegion SigLIP Δ ↑LPIPS-out ↓
Activation weightingUniform pooling (baseline)18.32 ± 1.07146.28 ± 4.57
Activation-weighted pooling (D-Scope)20.94 ± 0.63143.53 ± 4.28
Paired difference (weighted − uniform)+2.62 ± 0.68−2.75 ± 1.52
Patch selectionRandom 64 positive patches7.82 ± 0.71131.65 ± 5.25
Top 64 by activation21.81 ± 0.65139.66 ± 7.55
Evidence patch size2 × 2 visual-token cells17.45 ± 1.09136.81 ± 6.22
4 × 4 visual-token cells (default)17.50 ± 0.50132.91 ± 5.85
8 × 8 visual-token cells19.26 ± 0.45134.36 ± 6.43

These results show that

  • On identical patches, activation weighting improves mean target gain by 14.3% and lowers LPIPS-out.
  • Similarly, selecting the highest-activation patches gives higher target gain than sampling positive patches at random, although it also increases LPIPS-out.
  • For patch size, the 2 × 2 and 4 × 4 patches yield similar target gains. Increasing the patch size from the default 4 × 4 to 8 × 8 improves target gain but also increases LPIPS-out.

These findings therefore support D-Scope's use of highly activating patches and activation weighting. At the same time, the preservation costs observed in the patch-selection and patch size comparisons are consistent with our original conclusion that improved target alignment does not always yield better outside-region preservation.

Evidence threshold. We ran additional experiments to further test whether pooling more images per feature and admitting more features to the index have different effects on steering. We vary the number of distinct images pooled per feature (J) separately from the minimum number of positive images required for indexing.

The next table below reports these comparisons on the same matched cases. Rows with the same indexing threshold measure the effect of pooling more images, while rows with the same J measure the effect of expanding eligibility. Metrics follow the reporting convention in the previous table.

Images pooled per feature (J)Minimum distinct images for indexingIndexed featuresRegion SigLIP Δ ↑LPIPS-out ↓
166418,29815.43 ± 0.70147.77 ± 3.81
161618,43215.47 ± 0.68147.65 ± 3.37
326418,29820.61 ± 0.61139.45 ± 3.52
323218,42120.61 ± 0.61139.45 ± 3.52
646418,29820.94 ± 0.63143.53 ± 4.28

These results show that with the candidate pool fixed, pooling more images per feature improves mean target alignment. However, expanding eligibility at the same J has little effect for this dictionary. At J = 32, the additional candidates never replace the top-ranked feature, which explains the identical steering scores. This pattern agrees with the original paper's finding that a larger visual index alone does not guarantee stronger steering.

The revised manuscript includes the uniform-pooling baseline and ablations of activation weighting, patch selection, patch size, and the evidence threshold in Appendix D.4 (Tables 24 and 25), with the activation-weighting result summarized in Section 4.4.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their time and constructive feedback. We appreciate the reviewers' recognition of the following strengths:

  • Our method provides a "clear and inspectable connection between feature retrieval and intervention" [LkiP], and the proposed approach is "interesting and useful" [42qi].
  • Our SAE training study is "large and systematic" [oKJ1].
  • Our experiments and ablation studies are "comprehensive" [42qi].

Regarding the raised concerns, we have conducted additional experiments and provided the following clarifications:

  • Reviewer LkiP: Added strength calibration on separate concepts and independent evaluation. Independent encoders and two VLM judges continue to favor contrastive retrieval over direct retrieval under calibrated strengths. We also added a uniform-pooling baseline and ablations of activation weighting, patch selection, patch size, and the evidence threshold.

  • Reviewer oKJ1: Clarified the contribution relative to SemanticLens and prior FLUX SAE work and added comparisons with prior methods. These include masked FluxSpace, paired-activation SAE feature selection, and Attribute Control. We also added random and mismatched feature controls, evaluated the retrieved features' supporting patches with five VLM judges, examined mask sensitivity, and expanded the qualitative comparisons.

  • Reviewer 42qi: Compared visual retrieval with description-based SAE retrieval and clarified reference-query selection for complex edits. Visual retrieval achieves higher regional target gains than both description-based baselines under direct and contrastive retrieval. We also added teacher timestep ablations and pixel-level outside-region preservation metrics, and clarified the interpretation of high reconstruction fidelity alongside low dictionary utilization.

Detailed point-by-point responses are provided below. We have also revised the manuscript accordingly, with all changes marked in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.