- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates which training images helped shape a diffusion model’s generated image. It introduces a gradient-based influence estimator TID, and a lightweight student that turns attribution into fast vector search (TIDE). The main contributions are combining diffusion-model features, online teacher supervision, and a ranking loss. Experiments report competitive attribution quality and substantial query-time speedups with cached embeddings. TIDE closely matches its teacher’s rankings with Spearman correlations around 0.86–0.89. Removing highly ranked training images and retraining also changes generated outputs, suggesting the selected examples matter.
Critical strengths
-
TIDE turns expensive attribution calculations into fast searches over cached embeddings. Its strong agreement with the teacher suggests that much of the attribution signal survives this compression, making repeated analysis more practical.
-
The authors retrain models after deleting selected training images, testing whether those selections actually affect generated outputs. Combined with comparisons against several attribution baselines, this gives concrete evidence that the method identifies consequential training data.
Critical weaknesses
-
The deletion tests miss query-specific accuracy. The authors remove the union of training images selected for multiple queries. Shuffling the query-to-selection assignments leaves that union unchanged, so the experiment cannot tell whether the method linked each generated image to the right training examples.
-
The gains over prior work aren’t convincing. Missing comparisons with Fast Data Attribution’s student and the same diffusion features without distillation make it hard to identify what TIDE adds. Misdescribing the prior method’s unlearning cost further weakens the argument for its claimed advantages.
Questions for authors
Is it reasonable to replace the squared norm of output changes with their squared sum (line 144~145)? Could you directly validate whether this approximation preserves attribution rankings on a small model?
3 comments
3. Comparison with Fast Data Attribution (W2).
Our main reason for not reproducing FDA (Wang et al., 2025) is the cost of collecting its supervision. On MS-COCO, FDA uses AbU+ (Wang et al., 2024) to rank 10,000 candidates for each of 5,000 generated queries, with 4,900 queries used for training and 100 for validation. It reports 1,250 GPU-hours for this curation, before student training. Its Stable Diffusion curation costs 15,150 GPU-hours.
To estimate the expense of an analogous unlearning-supervised pipeline on our models, we use our measured MUCS runtime and consider two different training set budgets, full training set (50k samples) and the 10k candidates. The resulting estimates are:
| Quantity | CIFAR-10 | ArtBench-10 | MS-COCO |
|---|---|---|---|
| MUCS per query, full training set | 1,374 s | 1,452 s | 3,404 s |
| MUCS per query, 10k candidates | 298 s | 323 s | 321 s |
| Supervision for 4,900 queries, full training set | 1,870 h | 1,976 h | 4,633 h |
| supervision for 4,900 queries, 10k candidates | 406 h | 440 h | 437 h |
| TIDE's complete one-off stage | 3.8 h | 4.2 h | 10.0 h |
| Estimated supervision cost (10k) / TIDE one-off cost | 107× | 105× | 44× |
The estimations show that collecting a similarly sized supervision set would exceed our complete distillation budget by a large margin. Even reducing the supervision set to 2,000 queries would leave a significant difference under the same assumptions. TIDE instead computes teacher scores online for within-batch pairs, sharing a forward–backward pass across each sample's pairwise supervision.
We use MUCS as the representative unlearning baseline because it is the strongest unlearning method in our comparisons; the original AbU performs at random level in our experiments, consistent with the MUCS paper. FDA uses the improved AbU+ teacher, but their Table 1 reports only modest gains over AbU. The same table also shows that the student generally underperforms or only matches its corresponding unlearning-based teacher. Given the substantial computational cost and these performance limitations, we do not include FDA in our empirical comparison.
4. Correction to the description of AbU (W2).
We identified and corrected a mischaracterization in Section 2.2: AbU performs unlearning for each query, rather than for each training sample. We thank the reviewer for pointing this out.
5. Validity of summed-output scalarization (Q1).
We agree that the squared sum of output changes is not generally equal to their squared norm. We therefore directly evaluate the ranking fidelity of this approximation using a controlled setting on CIFAR-10. We reuse approximately 40 subset-retrained models per seed from our CIFAR-10 benchmark and ablation experiments, across six seeds. For each subset-retrained model, we compute . We rescale all directions by a common factor to obtain a specified median relative parameter shift , then compute the Spearman correlation between exact score discrepancies and those from the linearized norm in Eq. 2 or the summed-output scalarization in Eq. 3. Averaging over 20 queries and four noise levels gives:
| Relative shift | Output linearization, Eq. 2 | Summed-output scalarization, Eq. 3 |
|---|---|---|
| 1.00 | 0.76 | |
| 1.00 | 0.76 | |
| 1.00 | 0.76 | |
| 0.97 | 0.74 |
These results show that scalarization introduces a measurable approximation error but retains substantial ranking agreement over the tested range. Separately, we evaluate the final TID scores using the same set of models. For each of 20 generated queries, we again measure the score discrepancy between the original model and each retrained model at four noise levels sampled from the Karras generation schedule (Karras et al., 2022). We compare these measurements with attribution predictions aggregated over the corresponding removed subset.
For each query and noise level, we compute Spearman correlation across the retrained models, then average these correlations over queries and seeds:
| Method | ||||
|---|---|---|---|---|
| TID | 0.54 | 0.60 | 0.65 | 0.60 |
| MUCS | 0.40 | 0.52 | 0.63 | 0.61 |
| DAS | 0.30 | 0.44 | 0.42 | 0.44 |
| CLIP | 0.20 | 0.31 | 0.39 | 0.43 |
| Random scores | 0.24 | 0.32 | 0.40 | 0.41 |
The random baseline assigns uniformly random scores to training samples and aggregates them over the same removed subsets. TID consistently exceeds most baseline methods and achieves the highest correlation at three of the four noise levels. These experiments provide empirical evidence for both the scalarization step and the fidelity of the final estimator.
References:
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Joan Serrà, Dipam Goswami, Fabio Morreale, Wei-Hsiang Liao, and Yuki Mitsufuji. Training data attribution in diffusion models via mirrored unlearning and noise-consistent skew. In Advances in Neural Information Processing Systems (NeurIPS), 2026.
Sheng-Yu Wang, Aaron Hertzmann, Alexei A. Efros, Jun-Yan Zhu, and Richard Zhang. Data attribution for text-to-image models by unlearning synthesized images. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Sheng-Yu Wang, Aaron Hertzmann, Alexei A. Efros, Richard Zhang, and Jun-Yan Zhu. Fast data attribution for text-to-image models. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
Thank you for the comments. Based on your feedback, we provide additional control-query results, a diffusion-feature baseline comparison, and an empirical scalarization analysis.
1. Query targeting in the deletion evaluation (W1).
We agree that grouped removal may obscure the query-to-selection matching. To examine whether the selected training data could affect unrelated queries, we evaluate an additional control group. Each run involves 40 query samples. The first 20 are the normal targeted queries samples that are attributed and used to determine the removal set; the remaining 20 are randomly selected and do not relate to the selections. We regenerate both groups using the same retrained model and evaluate them against the same random-removal reference. Mean counterfactual AUCs over six seeds are:
For every method and dataset, the attributed queries show substantially larger AUCs than the controls, and the AUCs of the controlled group are almost around random level (0.5). This demonstrates that the selected training data preferentially affects the queries used to construct it, rather than affecting unrelated queries, or producing degradation on the diffusion model.
We also examine how much the per-query top-2% selections overlap. Mean pairwise overlap across the 20 queries and six seeds is:
These statistics show that the methods select largely different examples for different queries. They complement the control-query experiment by establishing diversity of the selections. We also want to clarify that we follow the MUCS protocol to evaluate this to achieve a feasible cost (Serrà et al., 2026). Separately retraining for all 20 queries would multiply the retraining count by 20.
2. What distillation adds beyond diffusion features (W2).
We evaluate the requested frozen-feature baseline on CIFAR-10. It uses the same source activations as TIDE (the residual streams of all 12 DiT blocks, mean pooling, M=32) and is directly scored with cosine similarity without postprocessing. We compare its recovery of the teacher's top-k training examples with TIDE and external encoders. Mean overlap over 20 queries and six seeds is:
Diffusion features do recover more of the teacher's rankings than the external encoders like CLIP, but distillation further improves overlap. This directly establishes that the learned embedding captures substantially more of TID's attribution ranking than the underlying frozen features alone.