- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the high inference cost of DiT-based reference-conditioned generation. The authors find that the model can still produce reasonable results after some reference tokens are dropped, even without additional training. In Sparse Context, the model randomly drops reference tokens to keep 5-25% of them and is then fine-tuned with LoRA using the remaining tokens. At inference time, the model selects tokens using Canny edges for image editing and saliency maps for personalization. The results show better quality than token dropping without fine-tuning in image editing, single-reference and multi-reference generation, and video generation. The method also improves inference efficiency over the original model, especially in multi-reference settings.
Critical strengths
-
The method is easy to understand. It randomly drops some reference tokens and then fine-tunes the model with LoRA using the remaining tokens. The authors evaluate this approach on FLUX.2-Klein-4B, FLUX.2-Klein-9B, and a video generation model. It can also be combined with KV caching and token merging.
-
Training with random token dropping avoids tying the model to a particular token selection rule, allowing the selection strategy to be adapted to the task at inference time.
Critical weaknesses
-
Image editing is tested only on PIE-Bench, and single-reference personalization only on a subset of Subjects-200K. Multi-reference generation shows the largest speedups, but quantitative evaluation of identity preservation in this setting is limited.
-
The paper needs to explain more clearly how the method relates to and differs from ToPi [1] and OminiControl2 [2]. The image experiments are also limited to FLUX.2-Klein (9B and 4B).
-
The presentation and experimental details need to be clearer. For single-reference personalization, the main paper reports 20K training samples, but Appendix A.11 reports 30K. Model names are misspelled and written inconsistently, such as "FLUX.2-Klien" and "Flux-Klien2". It would also be helpful to check the GPU name "RTX-6000 Pro" in A.12 and correct any typos. KID scores are reported without explaining the metric.
Questions for authors
-
OmniContext [3] and XVerseBench [4] include multi-reference generation tasks and provide evaluation protocols. Please report results on these benchmarks and on ImgEdit [5] for image editing.
-
Please explain how the method differs from ToPi [1] and OminiControl2 [2] and what advantages it offers. If comparison results are available, please include them.
-
Please provide results of applying the proposed method to other reference-conditioned generation models, such as FLUX.2 [dev] or Qwen-Image-Edit [6].
-
A comparison with encoding reference images at a lower resolution would help show the benefit of selecting which tokens to keep.
[1] Lin, J., et al. Token Pruning for In-Context Generation in Diffusion Transformers. arXiv (2026).
[2] Tan, Z., et al. OminiControl2: Efficient Conditioning for Diffusion Transformers. CVPR Findings (2026).
[3] Wu, C., et al. OmniGen2: Towards Instruction-Aligned Multimodal Generation. CVPR (2026).
[4] Chen, B., et al. XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation. NeurIPS (2025).
[5] Ye, Y., et al. ImgEdit: A Unified Image Editing Dataset and Benchmark. NeurIPS (2025).
[6] Wu, C., et al. Qwen-Image Technical Report. arXiv (2025).
6 comments
2. Evaluation for Multi-Reference Case
To score multi-reference generation on our evaluation set, we add a per-object subject fidelity metric. Each reference object is segmented using SAM3 in both output and ground truth image and we report the CLIP and DINO similarities between the two segmented crops. We use random token selection in this case for all reference images on our trained checkpoint and we preserve identity better than the naive (no-LoRA) checkpoint at all keep fractions:
Table — Multi-reference subject fidelity (output vs. dataset GT), 200 samples.
| Method | frac | CLIP ↑ | DINO ↑ |
|---|---|---|---|
| Baseline (full tokens) | 1.00 | 0.9184 | 0.6761 |
| Naive | 0.10 | 0.8690 | 0.4655 |
| Ours | 0.10 | 0.8866 | 0.5428 |
| Naive | 0.15 | 0.8802 | 0.5171 |
| Ours | 0.15 | 0.8902 | 0.5732 |
| Naive | 0.20 | 0.8875 | 0.5446 |
| Ours | 0.20 | 0.8966 | 0.5816 |
Across every drop fraction, our method preserves per-object subject identity substantially better than naive dropping and approaches full token baseline as budget increases.
3. Comparison with ToPi[1]
We thank the reviewer for pointing to this related work. ToPi[1] is a training free, inference time method which scores the reference tokens by attention based importance and keeps an adaptive subset with the selection mask recomputed periodically during the whole denoising process. We note three points:
-
ToPi[1] is not evaluated on the Flux.2-Klein family of models that we use, and has no publicly available code. Hence a faithful reimplementation is not feasible during this discussion period.
-
Because its mask is updated periodically across timesteps, the retained token set is not fixed through the denoising trajectory, hence the method has to periodically process a full length sequence to update the mask due to which it does not yield a consistent reduction in peak memory.
-
In contrast, we assign position ids / RoPE a single time, select tokens once, then physically remove the rest of them and the kept tokens retain their original RoPE. This shortened sequence is then used for the full denoising process.
We will add [1] to our related work section, and mention this in revision.
1. Benchmarking on ImgEdit[5] and XVerseBench[4]
We thank the reviewer for this suggestion. In the following results we evaluate our trained checkpoint on ImgEdit-Bench[5] (editing) and XVerseBench [4] (multi reference generation). We skip OmniContext[3] as its official evaluation protocol requires GPT 4.1 as a judge. (API Access) Editing: ImgEdit-Bench [5] We evaluate on ImgEdit-Bench (all 737 single-turn edits spanning 9 edit types) with the same evaluation setup we used in Table1 in paper.
Table — ImgEdit-Bench (737 samples).
These results show that same observations hold on ImgEdit-Bench[5] also.
Multi-Reference Generation: XVerseBench [4]
We run all 300 XVerseBench prompts (1-3 subjects each) on Flux2-Klein9B and score them with benchmark’s official metrics:
Table — XVerseBench (300 prompts), FLUX.2-Klein-9B. For Ours the LoRA is fixed; only the inference-time selection rule differs (random vs. segmentation).
Since the selection rule at inference time is pluggable, we also sample tokens only from the segmented subject region, as XVerseBench subjects are cut-out images on white a white background. Choosing tokens in this way gives further improvement over random token selection at each keep fraction.
Furthermore, ours preserves identity and appearance better than naive dropping at every keep fraction. From visual inspection we observe that Naive scores are higher on DPG and AES because with only f% of the tokens, it barely extracts any information from the references and follows text mainly.