acceptodds
Under review as a conference paper at ICLR 2027

Training Recipes Leak Through Behavioral Fingerprints in Text-to-Image Diffusion Models

Abstract

A deployed text-to-image diffusion model exposes generated images while keeping its fine-tuning procedure hidden. Training choices such as LoRA rank and scaling, learning rate, batch size, and training steps therefore appear inaccessible to a black-box user. We show that generated images can nevertheless reveal these choices. We present the first systematic study of whether fine-tuning hyperparameters of text-to-image diffusion models can be inferred from generated images alone. Given only black-box query access, we probe a target model with a fixed set of prompts and summarize its generations into a behavioral fingerprint using semantic, perceptual, and low-level image statistics. We then train predictors on fingerprints from shadow models with known recipes to infer the target model's hidden hyperparameters. Across 297 LoRA models, we recover five of six varied hyperparameters at rates significantly above chance. LoRA rank leaves the strongest fingerprint: we recover it with 85.4% exact accuracy and 100% accuracy within one grid step. When we consider all six predictions jointly, we recover the complete recipe exactly in 9.8% of cases, compared with a 0.21% uniform-chance baseline, a 48 increase over chance. We continue to recover rank significantly above chance across changes in model architecture, base-model weights, and fine-tuning data, and we find that controlling for adapter drift does not remove the signal. We also recover rank using only 96 generated images and continue to detect its fingerprint under substantial output perturbations. In contrast, LoRA dropout exhibits weak and classifier-dependent evidence of recovery. These results show that fine-tuning leaves persistent but uneven behavioral fingerprints in generated images, allowing black-box outputs to reveal parts of an otherwise hidden training recipe.

Primary area:
generative models
Keywords:
text-to-image diffusion models, hyperparameter inference, behavioral fingerprinting, black-box attacks, LoRA, shadow models, machine learning security

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aXP1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether the fine-tuning recipe of a text-to-image diffusion model can be recovered from its generated images alone, under black-box query access. The attack builds shadow models with known recipes, summarises each model's generations into a behavioural fingerprint, trains supervised predictors on fingerprint-recipe pairs, and applies them to unseen victims.

The fingerprint is a 78-dimensional vector over seven blocks: base-model drift, output diversity, prompt-perturbation sensitivity, texture, text-image alignment, cross-prompt consistency and colour, instantiated separately with frozen DINOv2, CLIP and DeiT. The primary bank is 297 SD v1.5 LoRA models fine-tuned on an 833-image Pokemon corpus over a six-hyperparameter grid -- rank {4,16,64}, alpha {8,16,32}, learning rate, batch size, steps, dropout -- probed with 96 prompts and eight seeds. Predictors are RF, XGBoost, logistic regression and two MLPs, evaluated under three-fold, ten-repeat model-level cross-validation stratified by rank, with a fixed 300-tree RF as a selection-independent reference, three baselines, and 1,000-permutation tests with Benjamini-Hochberg correction.

Rank is recovered at 85.4% against a 34.4% stratified baseline, scaling at 72.8%, and learning rate, steps and batch at 49-56%; dropout is weak and predictor-dependent. Exact six-component joint recovery is 9.6-10.2% against a 0.21% uniform-chance baseline. Rank generalises off-grid (100% bracketing of six unseen ranks, Spearman rho about 0.81) and transfers across SD 2.1, changed base weights, a different domain and a disjoint same-domain corpus. Eight prompts yield 82.9% rank accuracy under pooled cross-validation with predictor selection. Nineteen non-adaptive output-perturbation settings only partially reduce rank recovery, and the adaptive setting is explicitly left unevaluated. Adapter-drift controls show norms alone predict rank at 99.9%, while drift-stratified and residualised behavioural features retain most of the signal.

Critical strengths
  1. The evaluation includes several statistical controls that make the claims assessable. Model-level rather than image-level cross-validation, stratified by rank, with preprocessing fit only on training folds; three distinct baselines; a fixed 300-tree RF reported alongside the best predictor so the headline does not silently depend on model selection; 1,000-permutation tests with Benjamini-Hochberg correction; and an explicit statement that the procedure does not correct for selection among five predictors, so best-of-five results are exploratory. Per-predictor q-values are given for the borderline dropout result instead of a single convenient number. The checked recovery-rate and grid calculations reconcile; the permutation p-value in Table 8 requires clarification.

  2. The characterisation of what leaks is fine-grained and honest, and the empirical investment is substantial. Leakage is reported as heterogeneous: rank strong and transferring across architecture, base weights, domain and a disjoint corpus; scaling partial; learning rate, steps and batch clearly above chance but much weaker; dropout correctly de-emphasised. Off-grid behaviour is framed as bracketing and ordering rather than exact recovery. The guidance-5.0 degradation, the non-monotonic transition between ranks 24 and 32, and the systematic confusion structure are all reported, and the discussion states plainly that the findings "establish predictive associations rather than a causal mechanism". Roughly 627 fine-tuned models is a serious amount of controlled compute.

Critical weaknesses
  1. The predictive association is supported, but the mechanism underlying rank recovery remains unresolved, limiting our understanding of generalization beyond the evaluated settings. Adapter norms alone identify rank at 99.9% -- higher than the behavioural fingerprint's 85.4% -- while the residualisation control removes only the linear association with a single scalar (85.4% to 82.7%); the paper concedes that "other relationships with adapter-update magnitude may remain". ||B|| is also not the right quantity to control for: B -> cB, A -> A/c leaves BA unchanged while changing ||B||, and the deployed update is (alpha/r)BA. The paper does define and use base-model drift features, so drift is not unmeasured; what is absent is a control that MATCHES effective-update norm or measured drift across ranks and then re-tests recovery. The feature ablation points the same way: all seven blocks independently reach 78.9-86.2%, leave-one-out never falls below 84.0%, and the 12-dimensional colour block (86.2%) edges out the full fingerprint (85.4%) with no significance test on the gap. That redundancy is at least consistent with one dominant latent factor measured seven ways. This does not make the rank signal spurious and does not undercut a predictive association on held-out models within a stated distribution; it limits the rank-specific mechanistic reading the title invites.

  2. The demonstrated claim is narrower than the framing, and the defence study is non-adaptive. Recovery is strongly heterogeneous: rank 85.4% and scaling 72.8% against 33.3% chance; learning rate 54.4%, steps 55.9% and batch 49.4% well above chance but much weaker; dropout 55.0% against a 53.8% majority baseline. Joint six-component recovery is 9.8%, so the defensible summary is around 10% exact full-recipe recovery with markedly uneven per-component performance. What is most solidly established is rank, a three-way choice on a log-spaced grid where adjacent classes differ by 4x and the endpoints by 16x, in the primary controlled protocol, with additional transfer experiments extending beyond its training domain and base model. Off-grid, the log2-rank MAEs of 0.682/0.691/0.708 give a geometric-mean multiplicative error around 1.60-1.63x -- a useful average error scale, not a prediction interval with stated coverage. Most consequentially, the perturbation defences are applied only to victim outputs while the attacker's predictors are never retrained, and Appendix E.1 says so: the reported figures are rank-recovery accuracies under a fixed attacker, not suppression rates (46.2% under x0.125 downscaling, 51.6% under strong quantisation, 54.8% under JPEG q10), and effectiveness against an adapted attacker remains untested. This limits the deployment-facing conclusions; it does not undercut the black-box leakage evidence itself.

Questions for authors

Items 1-3 would move my score; the rest are secondary.

  1. (Experiment request) Can you separate rank from update magnitude? Compensate alpha or the training budget so that rank-4, rank-16 and rank-64 adapters reach a matched effective-update norm ||(alpha/r) B A|| or matched measured base drift, then re-run behavioural rank recovery on that bank. Residualising log||B|| does not remove the confound, since B -> cB, A -> A/c leaves BA unchanged. This would test whether rank recovery persists after controlling these quantities, although it would not by itself establish a causal mechanism; I consider it the most valuable addition.

  2. (Experiment request) Please evaluate the adaptive defence setting: train shadow predictors on perturbed shadow generations matched to the victim's perturbation, then attack perturbed victims. No new diffusion models are needed. Measuring the adaptation cost would also tell deployers how much protection they actually have.

  3. (Clarification) Please discuss Asnani et al., "Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images" (arXiv:2106.07873; IEEE TPAMI 2023), which shares the paradigm -- a fingerprint from generated images, fed to a trained predictor, used to infer generative-model hyperparameters across a bank of models -- and is not cited. It targets GAN architecture and loss choices rather than diffusion fine-tuning recipes, and your claim is already scoped to text-to-image diffusion, so I am not asking you to retract it; I am asking for explicit positioning so readers can see which part of the paradigm is inherited and which is new.

  4. (Clarification) Appendix C.4 describes 1,000 permutations with an add-one correction, whose smallest attainable p-value is 1/1001 = 0.000999, yet Table 8 reports p = 0.0007. Did that table use a different test, more permutations, or an exact procedure? Raised only to get the procedure on record.

  5. (Clarification) How do results change on a finer grid -- ranks {4,8,16,32,64}, alpha over a narrower range, adjacent learning rates within 2x? Adjacent rank classes currently differ by 4x and the endpoints by 16x. Would within-grid log-space regression be a more representative headline than three-way classification? Could you also report the per-rank error distribution and error quantiles for the off-grid result, since a mean multiplicative error does not tell a reader how tight any individual prediction is?

  6. (Clarification) Section 5.9 already notes that the 8-prompt figure uses the best-performing predictor under pooled cross-validation and is not validated on independently held-out victims, which I appreciate. Could you add the matched comparison under the fixed 300-tree RF and the main protocol?

  7. (Clarification) Which fingerprint components carry the cross-setting transfer -- the encoder-agnostic pixel blocks or the base-drift features, which need the correct base checkpoint? The architecture-blind result, where DINOv2 drops from 78.3% to 60.1% while CLIP and DeiT hold at 74.6% and 71.6%, suggests the blocks behave differently. Relatedly, at guidance 5.0 the selected random forest never predicts rank 4 at all (Table 16: the rank-4 column is 0/0/0 across 675 predictions). Which feature block drives that one-directional shift?

  8. (Suggestion, optional) Releasing the fingerprint extractor and the fingerprint-label bank, plus per-model recipe and split manifests, the 96 prompts, and per-model out-of-fold predictions, would let readers check the predictors without needing model weights.

2 comments
AuthorsOpenReview ↗

Response to Reviewer aXP1 (Part 1)

We thank the reviewer for their constructive feedback. We have addressed the concerns through additional experiments and manuscript revisions, with all changes highlighted in blue. Our point-by-point responses follow.

Q1. Separating rank from effective-update magnitude.

We thank the reviewer for highlighting this confound. We agree that the norm of the LoRA B matrix is insufficient, as reciprocal rescaling of the LoRA factors leaves the effective update unchanged. We therefore measure the aggregate Frobenius norm of the effective update: ΔWℓ=αrBℓAℓ\Delta W_{\ell} = \frac{\alpha}{r} B_{\ell} A_{\ell}

\sqrt{\sum_{\ell} \lVert \Delta W_{\ell} \rVert_F^2}$$ We match rank-4/16/64 adapters from our 297-model collection within 0.15 decades of log effective-update norm, without using behavioral features or prediction outcomes. This yields **64 matched triplets (192 models)** and reduces the maximum cross-rank standardized mean difference in log norm from **0.97 to 0.11**. Using DINOv2 fingerprints, a fixed 300-tree random forest, and 3-fold × 10-repeat model-level cross-validation, rank recovery remains **85.2% (permutation p = 0.001)**, compared with 85.4% on the original collection. On the same matched cohort, update norm alone achieves **44.4%**, while update norm combined with the remaining recipe components achieves **52.9%**. Behavioral fingerprints outperform the combined control by **32.3 percentage points (95% paired bootstrap CI: 25.4–39.3)**. These findings demonstrate that measured effective-update magnitude and the remaining recipe components do not fully explain behavioral rank recovery. While matching does not establish a causal mechanism or eliminate all update-related confounds, the rank signal persists after substantially reducing the proposed confound. We have added the matching protocol, controls, and statistical results to **Section 5.8 and Appendix C.14**. **Q2. Adaptive attacker against output-perturbation defenses.** We thank the reviewer for this suggestion. We evaluate a defense-aware attacker that applies the same perturbations to its 162 shadow-model outputs, re-extracts DINOv2 fingerprints, and retrains the fixed 300-tree random forest. Evaluation uses the same 135 independently held-out victims as the non-adaptive setting. At the strongest tested setting of each defense, we obtain: | Perturbation | Non-adaptive | Adaptive | |:-------------|-------------:|---------:| | Downscaling (0.125×) | 46.2% | **79.6%** | | Quantization (6 bits removed/channel) | 51.6% | **79.6%** | | JPEG (quality 10) | 48.9% | **79.4%** | | Gaussian noise (32/255) | 75.1% | **80.7%** | The undefended baseline is **77.9%**. Across all four perturbations, adaptive recovery reaches **79.4–80.7%**, indicating that the non-adaptive reductions largely reflect train–test distribution mismatch rather than elimination of recoverable rank information. Adaptation requires no additional diffusion-model training, only reprocessing existing shadow outputs and retraining the predictor. We have added the adaptive evaluation to **Section 5.10 and Appendix E**. The tested perturbations do not reduce rank recovery below the undefended baseline against a defense-aware attacker. **Q3. Positioning relative to Asnani et al.** We thank the reviewer for highlighting this relevant work. We agree that Asnani et al. (TPAMI 2023) established the broader model-parsing paradigm, including diffusion-model experiments. We have added the citation and clarified our positioning in Related Work. Our work differs in its target and experimental setting: rather than inferring generator architecture or training loss, we investigate **fine-tuning-recipe leakage** in text-to-image diffusion models. We hold the base checkpoint, architecture, and training corpus fixed while varying six LoRA hyperparameters, and recover these properties using model-level behavioral fingerprints aggregated across controlled prompts and seeds. We additionally adapted the authors' official FEN+PN implementation to predict our six recipe components, using the same 297 models, query budget, and 3-fold × 10-repeat model-level cross-validation splits: | Hyperparameter | Ours | FEN+PN | |:---------------|-----:|-------:| | LoRA rank | **85.4%** | 79.2% | | LoRA scaling | **72.8%** | 64.2% | | Batch size | **49.4%** | 40.0% | | Training steps | **55.9%** | 51.8% | All four improvements are significant after Benjamini–Hochberg correction. Learning rate and dropout show no significant differences. This comparison evaluates complete inference pipelines rather than fingerprint representations in isolation. Our contribution is therefore not the general paradigm of model parsing, but the **systematic characterization of fine-tuning-recipe leakage in text-to-image diffusion models**, supported by a direct comparison with prior work. We have incorporated the comparison into the **Section 5.2 and Appendix C.3**.
AuthorsOpenReview ↗

Response to Reviewer aXP1 (Part 2)

Q4. Permutation p-value in Table 13.

We thank the reviewer for identifying this discrepancy. Table 13 uses a different permutation procedure from the cross-validation test: we permute the nine victims' labels against fixed predictions from the first seeded classifier fit, without retraining, using 10,000 permutations and an add-one correction. The minimum attainable Monte Carlo p-value is therefore 1/100011/10001, rather than 1/10011/1001.

The nine victims comprise three models per rank. Under permutations preserving these class counts, perfect rank recovery has an exact null probability of 3!3/9!=1/1680≈0.00063!^3/9! = 1/1680 \approx 0.0006, consistent with the reported Monte Carlo estimate (p=0.0007p=0.0007).

We have clarified the permutation procedure and evaluation protocol in Appendix C.5 and the caption of Table 13.

Q5. Off-grid rank error distribution.

We thank the reviewer for requesting a more detailed characterization of off-grid rank errors. We analyzed predictions for 48 victims at six unseen ranks (6, 8, 12, 24, 32, 48) under the original evaluation protocol.

For DINOv2, the median absolute error is 0.60 log₂-rank units (approximately 1.5× multiplicative error). 77% of predictions fall within a factor of two, with a 90th-percentile error of approximately 2.5× and a maximum observed error of 2.6×. CLIP and DeiT show similar distributions, with maximum errors below 2.8×.

The overall signed bias is small (+0.04 log₂ for DINOv2), although rank 24 tends to be overestimated and rank 32 underestimated, consistent with the non-monotonicity in Fig. 2. Since the regressor is trained on only three rank values, these results support approximate rank estimation rather than calibrated continuous prediction.

We have added the per-rank error distribution and scaling analysis to Section 5.4 and Appendix C (Table 8).

Q6. Fixed-predictor and held-out evaluation of the 8-prompt result.

We repeated the 8-prompt evaluation using a fixed 300-tree random forest, without predictor selection, under pooled model-level cross-validation (297 models) and independent held-out evaluation (162 shadows, 135 victims). Results are averaged over 20 random prompt subsets.

HyperparameterPooled CV (8 prompts)Held-out (8 prompts)Held-out (96 prompts)
LoRA rank82.6 ± 1.1%78.0 ± 2.7%77.9%
LoRA scaling65.6 ± 3.2%59.6 ± 4.5%67.9%

For comparison, the original best-of-five pooled-CV rank result was 82.9 ± 1.1%. All 20 eight-prompt subsets achieve significant held-out rank recovery (permutation p≤0.001p \leq 0.001).

Thus, low-budget rank recovery persists without predictor selection and on independently held-out victims. Scaling is more sensitive to prompt reduction, so we restrict the strongest 8-prompt claim to rank.

We have added these results to Section 5.9 and Appendix D.3.

Q7. Feature-block transfer and rank-4 collapse under CFG 5.0.

We conducted additional feature-block ablations using a fixed 300-tree random forest to examine transfer robustness and guidance-dependent rank recovery.

Feature-block transfer. Under SD v2.1 architecture transfer, removing the base-drift block improves architecture-blind rank recovery:

EncoderFull fingerprintWithout base drift
DINOv255.7%72.5%
CLIP62.0%75.9%
DeiT71.6%77.8%

Pixel-only features achieve 69.0% across all three encoders. Under data shifts, block contributions vary, with base-drift features helping in some settings but degrading recovery in others. These results indicate that transferable rank information is not confined to a single fingerprint block (see Appendix C, Table 15).

Rank-4 collapse at CFG 5.0. We compare rank-4 recall from individual fingerprint blocks:

Fingerprint blockCFG 7.5CFG 5.0
Base drift82%17%
Diversity12%0%
Cross-prompt consistency98%97%
Colour100%93%

Reduced guidance substantially weakens rank-4 recovery from some blocks, while cross-prompt consistency and color remain informative. Thus, the effect is feature-dependent rather than a uniform loss of rank information. These single-block ablations do not establish which features cause the full classifier's rank-4 collapse (see Appendix D.6, Table 26).

Q8. Release of the fingerprint extractor and evaluation artifacts.

We have prepared an anonymized reproducibility package containing the fingerprint extractor, fingerprint–label banks, recipe and split manifests, 96 probe prompts, and per-model out-of-fold predictions. The package includes scripts to reproduce the primary results without model weights or generated images. An anonymized repository link is provided in the revised manuscript's Reproducibility Statement.

Reviewer_i2FtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates whether fine-tuning hyperparameters of text-to-image diffusion models can be inferred from their generated images. The approach constructs shadow models with known training recipes, extracts behavioral fingerprints using semantic, perceptual, and low-level image statistics, and trains predictors to recover six fine-tuning settings. The main evaluation is on LoRA models, with additional experiments on unseen rank values, model and data shifts, query efficiency, and output perturbations. The strongest and most consistent finding is the recovery of LoRA rank, while other hyperparameters are recovered less reliably.

Critical strengths

The paper studies an interesting question and provides convincing evidence that fine-tuning settings can leave recoverable traces in generated images. In the main evaluation, the fixed random-forest predictor recovers LoRA rank with >80% accuracy and scaling with >70% accuracy, substantially above the corresponding chance baselines. The use of model-level splits, a fixed-predictor reference, and permutation tests strengthens the evidence for this finding within the evaluated setting.

The additional experiments provide useful insights into the extent and limitations of the leakage. Rank recovery remains above chance across changes in the base model and fine-tuning data, while the feature ablations show that simple color statistics alone achieve 86.2% rank accuracy in the main setting. These results suggest that the signal is accessible through relatively simple measurements. The paper also documents the weaker recovery and less consistent transfer of other hyperparameters, helping clarify which parts of the training recipe are most exposed.

Overall, the empirical finding in the paper is solid and interesting.

Critical weaknesses
  1. The writing and figure layout make the approach harder to follow than necessary. Figure 1 compresses the shadow-model construction, recipe grid, seven feature blocks, and prediction pipeline into a crowded overview. Many internal labels are difficult to read at the intended printed size. Tables 2 and 3 are also placed too closely together, reducing readability. The method description would benefit from a more concise explanation of the role of the feature groups before introducing the detailed notation. Simplifying and enlarging the overview figure, improving table spacing, and presenting the main design intuition more directly would make the contribution easier to assess.

  2. Closely related model-parsing methods are missing from the discussion and empirical comparisons. Neither Asnani et al. [1] nor Guo et al. [2] is cited, despite their relevance to inferring generator attributes from generated images. The authors should clarify the contribution of the proposed training-recipe inference setting and fingerprint design relative to these works. An appropriately adapted model-parsing baseline, evaluated using the same shadow-model splits and victim-query budget, would also help establish the value of the proposed approach. Comparing several standard predictors on the proposed fingerprints does not resolve this question. This omission leaves the methodological contribution less clearly established than the empirical leakage finding.

[1] Asnani, Vishal, et al. "Reverse engineering of generative models: Inferring model hyperparameters from generated images." IEEE Transactions on Pattern Analysis and Machine Intelligence 45.12 (2023): 15477-15493.

[2] Guo, Xiao, et al. "Tracing hyperparameter dependencies for model parsing via learnable graph pooling network." Advances in Neural Information Processing Systems 37 (2024): 116899-116932.

Questions for authors

How does the contribution differ from the model-parsing approaches in [1] and [2], particularly in the inference target and fingerprint design? Please clarify which aspects constitute the main advance. Can you compare your approach with a method from [1] or [2], adapted to predict the six fine-tuning settings? Please use the same training and test models and give both methods the same number of generated images. This would help clarify the benefit of the proposed fingerprint design.

[1] Asnani, Vishal, et al. "Reverse engineering of generative models: Inferring model hyperparameters from generated images." IEEE Transactions on Pattern Analysis and Machine Intelligence 45.12 (2023): 15477-15493.

[2] Guo, Xiao, et al. "Tracing hyperparameter dependencies for model parsing via learnable graph pooling network." Advances in Neural Information Processing Systems 37 (2024): 116899-116932.

3 comments
AuthorsOpenReview ↗

Response to Reviewer i2Ft (Part 1)

We thank the reviewer for their constructive feedback. We have addressed the concerns through additional experiments and manuscript revisions, with all changes highlighted in blue. Our point-by-point responses follow.

Q1: Model-Parsing Comparison

We have added Asnani et al. and Guo et al. to the related work and clarified our contribution. Unlike prior model-parsing approaches targeting generator architecture and loss functions, our work investigates fine-tuning recipe leakage in text-to-image diffusion models using behavioral fingerprints.

We adapted Asnani et al.'s official FEN+PN implementation to predict our six hyperparameters. Both approaches were evaluated on the same 297 LoRA models, 768 images per model, and identical three-fold, ten-repeat model-level cross-validation splits.

HyperparameterChanceOursBase-freeFEN+PN
LoRA rank33.3%85.4%*86.3%79.2%
LoRA scaling33.3%72.8%*74.2%64.2%
Learning rate33.3%54.4%54.3%51.0%
Batch size33.3%49.4%*49.4%40.0%
Training steps33.3%55.9%*56.4%51.8%
LoRA dropout50.0%55.0%54.2%54.1%

An asterisk indicates a significant improvement of our full method over FEN+PN after Benjamini–Hochberg correction.

Our full pipeline significantly outperforms FEN+PN on LoRA rank, scaling, batch size, and training steps. Differences in learning rate and dropout are not statistically significant. The base-free variant achieves comparable performance without requiring reference base-model generations.

These results support the effectiveness of our inference pipeline under matched evaluation conditions, although they do not isolate the contributions of the fingerprint representations alone. Details are provided in Section 5.2 and Appendix C.3 of the revised manuscript.

W1: Presentation and Readability

We thank the reviewer for these suggestions. We have revised Figure 1 to improve readability by increasing the font sizes of its internal labels. We have also rearranged Tables 2 and 3 to improve spacing and reorganized the Behavioral Fingerprint subsection to explain the intuition behind the seven feature groups before their mathematical formulation.

Thank you for your rebuttal. I do not have any other concerns and would maintain my score (weak accept).

We thank the reviewer for taking the time to evaluate our rebuttal and revised manuscript. We appreciate your constructive feedback and are glad that our additional experiments and clarifications have addressed your concerns. Your suggestions have helped strengthen the paper.

Reviewer_rqCsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates training-recipe leakage in black-box fine-tuned text-to-image diffusion models and proposes a hyperparameter inference method based on behavioral fingerprints. The method uses shadow models and generated images to construct 78-dimensional behavioral fingerprints for predicting hidden fine-tuning hyperparameters. Experiments on 297 LoRA models achieve 85.4% accuracy in LoRA rank recovery and 9.8% exact recovery of all six hyperparameters.

Critical strengths
  1. The paper explores fine-tuning hyperparameter leakage from black-box generated images, offering an interesting and novel research perspective.
  2. Experimental results demonstrate that the proposed method can effectively infer certain hidden training hyperparameters.
  3. The paper provides extensive evaluations covering cross-model generalization, feature ablation, query efficiency, and deployment-level defenses.
Critical weaknesses
  1. The method relies on strong assumptions about the attacker's prior knowledge, limiting its applicability in real-world black-box settings.
  2. Training a large number of shadow models may incur substantial computational costs.
  3. The evaluation lacks validation on more recent model architectures and only considers LoRA fine-tuning.
  4. The three encoder-based fingerprints share 40 out of 78 feature dimensions (51.3%), raising concerns about the independence of the cross-encoder evaluation.
Questions for authors
  1. As shown in Figure 2, the predicted probability of rank 64 decreases at true rank 32 across all three encoders, exhibiting non-monotonic behavior. Please explain the possible reasons for this phenomenon.
  2. How does the attack performance change as the number of shadow models decreases?
  3. Why do the 12-dimensional color features achieve higher LoRA rank recovery accuracy (86.2%) than the full 78-dimensional behavioral fingerprint (85.4%), and what additional benefits does the full fingerprint provide?
2 comments
AuthorsOpenReview ↗

Response to Reviewer rqCs (Part 1)

We thank the reviewer for their constructive feedback. We have addressed the concerns through additional experiments and manuscript revisions, with all changes highlighted in blue. Our point-by-point responses follow.

Q1. Non-monotonic rank-64 probability at true rank 32.

We thank the reviewer for highlighting this behavior. We investigated the decrease using the same random-forest classifier underlying Figure 2 across all three encoders.

The decrease is associated with differences in LoRA scaling (α\alpha) among off-grid victims. Each rank group contains eight models, but their scaling distributions differ: four rank-24 victims have α=8\alpha=8, whereas four rank-32 victims have α=32\alpha=32. Within-rank analysis shows a strong negative association between scaling and P(r=64)P(r=64) (Pearson r=−0.66/−0.65/−0.66r=-0.66/-0.65/-0.66 for DINOv2/CLIP/DeiT; permutation p<0.001p<0.001). Importantly, stratifying by α\alpha removes the rank-24/32 inversion across all three encoders.

These findings suggest that the observed non-monotonicity is associated with LoRA scaling rather than rank alone, highlighting a limitation of off-grid rank inference. As a post-hoc analysis, this does not establish causality. Details are provided in Section 5.4 and Appendix C.6 .

Q3. Color features versus the full fingerprint.

Using identical model-level splits, the 0.8-point advantage of color-only features over the full fingerprint (86.2% vs. 85.4%) is not statistically significant (95% CI: −1.2 to +2.9 percentage points; p = 0.43). Thus, color statistics alone capture a strong rank-related signal, while the additional features do not significantly improve in-distribution rank recovery.

However, the additional feature blocks provide benefits under certain distribution shifts. Under base-weight shift, color-only rank accuracy drops to 36.1% (not significantly above chance), whereas the full fingerprint achieves 50.8–61.7% across encoders. Conversely, under architecture-blind transfer, removing the base-drift block improves rank recovery from 55.7–71.6% to 72.5–77.8%, indicating that feature utility depends on the transfer setting.

These results show that the full fingerprint is not necessary for strong in-distribution rank recovery, but its additional feature blocks can improve recovery under particular distribution shifts. Details are provided in Section 5.7 and Appendices C.11–C.12.

W1. Attacker Prior Knowledge.

We conducted additional evaluations to examine the sensitivity of our attack to individual assumptions:

  • Seed control: Without seed alignment, rank recovery remains 77.5 ± 0.6%, compared with 84.9% using aligned seeds (33.3% chance). Section 5.9 and Appendix D.2.
  • Base checkpoint: Removing base-relative features achieves 72.5–77.8% rank recovery on SD 2.1 victims without requiring the victim's base checkpoint. Appendix C.12, Table 15.
  • Architecture and training data: Rank recovery reaches 75.1–77.0% under architecture transfer and 74.1–79.3% under training-data shifts. Section 5.5 and Appendix C.12.
  • Inference settings: Sampler and inference-step changes have modest effects, whereas reduced guidance substantially degrades recovery. Section 5.9 and Appendix D.6.
  • Query budget: Eight generated images achieve 78.0% rank recovery on held-out victims. Section 5.9 and Appendix D.3.

These results demonstrate that substantial rank leakage persists when several attacker assumptions are relaxed individually. We have clarified the threat model and added the seed-independent evaluation to the revised manuscript. We note that these experiments do not establish robustness when all assumptions are relaxed simultaneously.

W2/Q2. Shadow-Model Cost and Scaling.

We evaluated shadow-bank sizes of 25–162 models using 20 nested, rank-balanced subsets per size and 135 held-out victims. With only 25 shadow models, LoRA-rank recovery reaches 74.9–75.5% across encoders, compared with 78.2–78.6% using 162 shadows (33.3% chance). This retains 92.6–93.7% of the full-bank above-chance accuracy. Paired bootstrap confidence intervals for the difference include zero, although this does not establish equivalence.

Preparing 25 shadow models requires approximately 7.8 A100 GPU-hours (estimated), compared with 116 GPU-hours (measured) for our complete 297-model shadow bank. Other hyperparameters require larger banks to reach 90% of their full-bank above-chance accuracy: 75–100 shadows for scaling and learning rate, 125 for training steps, and 150–162 for batch size.

These results demonstrate that substantial LoRA-rank leakage is achievable with a relatively small, one-time offline investment, although the required shadow-model budget depends on the target hyperparameter. Details are provided in Section 5.9 and Appendix A.3.

AuthorsOpenReview ↗

Response to Reviewer rqCs (Part 2)

W3. Fine-Tuning Methods and Newer Architectures.

We thank the reviewer for highlighting this limitation. We conducted two additional evaluations beyond LoRA fine-tuning on SD 1.5/2.1.

  • Fine-tuning methods: We trained 27 full-UNet models and compared them with LoRA fine-tunes. After matching learning rate, training steps, and batch size, the fine-tuning method remains identifiable with 86.8–87.8% accuracy across encoders (50% chance). Section 5.2 and Appendix C.8.
  • Newer architectures: We evaluated 27 SDXL LoRA models. Zero-shot transfer from SD 1.5 shadows achieves only 37.0–40.7% rank accuracy (33.3% chance; not significant). In contrast, SDXL-matched shadows achieve 92.6% accuracy across all three encoders (permutation p = 0.001), including under leave-one-recipe-out evaluation. These small-scale results demonstrate rank leakage on SDXL while highlighting the limitations of zero-shot architecture transfer. Section 5.5 and Appendix C.9.

These results demonstrate that behavioral leakage extends to SDXL and that fine-tuning methods can be distinguished from generated images. Hyperparameter recovery within full-UNet fine-tuning remains untested.

W4. Independence of Cross-Encoder Evaluation.

We thank the reviewer for raising this concern. We confirm that 40 of the 78 fingerprint dimensions are shared across encoders: 20 texture and 12 color features computed directly from pixels, and 8 text-image alignment features computed using a fixed CLIP ViT-B/32 model. The remaining 38 dimensions are computed separately in each encoder's embedding space.

To assess whether shared dimensions explain cross-encoder agreement, we repeated the primary evaluation using only the 38 encoder-specific dimensions, retaining the same 297 models, model-level cross-validation splits, and random forest configuration.

LoRA-rank recovery remains 82.9%, 80.5%, and 83.8% for DINOv2, CLIP, and DeiT, respectively, compared with 85.4%, 85.0%, and 85.7% using the full fingerprint. All five previously recoverable hyperparameters remain significantly above chance across all three encoders (permutation p ≤ 0.005), while dropout remains non-significant.

These results demonstrate that shared dimensions do not solely explain cross-encoder agreement; encoder-specific representations retain substantial training-recipe information. However, because all three encoders process the same generated images, we do not claim statistically independent replications.

We have clarified the feature decomposition and added the encoder-specific ablation to the revised manuscript. Details are provided in Section 5.7 and Appendix C.11.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.