- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The search and management of existing LoRA communities mainly rely on titles, tags, descriptions, and example images, which may not accurately reflect the actual abilities learned by LoRA. To this end, the paper proposes the LoRA Scope framework, which advocates analyzing the semantic changes generated between the base model and the LoRA adaptation model, characterizing the semantic changes actually introduced by LoRA, and further learning to directly predict its semantic representation from LoRA weights. The LoRA Scope consists of three modules: LoUPE, LoUPE Score, and LoCAL, which correspond to the understanding, evaluation, and search of LoRA research.
Critical strengths
-
With the growing popularity of personalized image generation, community LoRA libraries contain a large number of adapters with different functions and quality levels. How to understand its actual ability, evaluate its reliability, and conduct efficient retrieval is a problem with clear practical value. The research problem has clear application requirements.
-
LoUPE uses LLMs to expand prompts and monitors changes in CLIP-space statistics to determine when to stop. It also uses VLM feedback to replace prompts that fail to activate the adapter effectively. Has a certain degree of innovation.
-
The author provided sufficient explanations for each module, and the paper is clear and easy to understand
-
The author conducted large-scale experiments and conducted valuable manual evaluations
Critical weaknesses
- LoUPE stops Prompt extension based on two conditions:The cumulative variance curve explained by adjacent rounds of PCA has a sufficiently small change;The cosine similarity in the direction of average difference between adjacent wheels is sufficiently high. These conditions can measure the stability of a specific statistic, but are not sufficient to ensure the convergence of a complete difference distribution.From an experimental perspective, Figure 4 mainly demonstrates that the stopping conditions designed by the author can be satisfied within a finite number of rounds, but does not fully demonstrate that these Prompts have accurately covered the actual capabilities of LoRA.
- This article lacks a comparison between LoUPE's adaptive prompt exploration and fixed prompt, random prompt extension, and LLM only fixed round extension under the same generation budget.
- The three factor product in Figure 6 did not show a higher correlation with the overall score than when Precision or Strength were used alone. Therefore, the paper has not fully demonstrated the necessity of using a multiplication combination of three indicators.
- This article mainly compares LoCAL with retrieval methods such as Stylus, Stylus+, and Stylus++, but lacks comparison with other weight retrieval baselines.
Questions for authors
see weakness
4 comments
Response to Reviewer KbyZ (2/3)
W2: Prompt-exploration baselines under the same generation budget
Weakness 2: This article lacks a comparison between LoUPE's adaptive prompt exploration and fixed prompt, random prompt extension, and LLM only fixed round extension under the same generation budget.
We compared LoUPE with fixed prompts, random prompt extension, and LLM-only extension on 100 randomly sampled adapters. With informative community prompts as starting examples, LoUPE covers more held-out usage than fixed or randomly extended prompt sets at matched prompt counts. The LLM-only comparisons further distinguish the coverage gained through prompt exploration from the activation gained through image-grounded refinement.
Coverage was evaluated on the 92 adapters with at least four distinct community prompts. We held out a subset of these prompts that was never shown to any method. Coverage is the fraction of held-out prompts whose adapter effect δ_II lies within a nearest-neighbor radius of an effect in the evaluated prompt set. Activation measures the magnitude of the adapter's effect as 1 − cos(base image, LoRA image) in CLIP space, averaged over the prompt set. LoUPE, the LLM-only variants, and random extension use the same community prompts as starting examples. The table reports the full prompt sets, with prompt counts averaged across adapters. The matched-count comparison with fixed prompts is reported below.
| Prompt set | Mean prompts per adapter | Coverage of held-out usage | Activation |
|---|---|---|---|
| Fixed prompts (PartiPrompts) | 100 | 0.47 | 0.12 |
| Random extension with PartiPrompts | 65 | 0.70 | 0.13 |
| LLM-only, initial round | 8 | 0.64 | — |
| LLM-only, half of LoUPE's rounds | 34 | 0.81 | — |
| LLM-only, same rounds as LoUPE | 65 | 0.87 | 0.21 |
| LoUPE | 65 | 0.87 | 0.22 |
-
Fixed prompts. We use the same 100 prompts, sampled at random from PartiPrompts, for every adapter. This fixed list covers 47% of held-out usage, compared with 87% for LoUPE using 65 prompts on average. After subsampling both sets to equal prompt counts for each adapter, averaging 58 prompts, coverage is 0.87 for LoUPE and 0.43 for the fixed list (difference 0.44, 95% CI [0.37, 0.50]). Mean activation is also higher for LoUPE, at 0.22 compared with 0.12 for fixed prompts.
-
Random extension. We extend LoUPE's initial round of 8 generated prompts with prompts sampled at random from the PartiPrompts list, matching the size of LoUPE's final set for each adapter. Results are averaged over 20 random draws. Random extension reaches coverage of 0.70, compared with 0.87 for LoUPE (difference 0.17, 95% CI [0.13, 0.22]), with mean activation of 0.13 compared with 0.22. Starting from the same initial coverage of 0.64, adaptive exploration therefore gains more coverage than random extension at the same prompt count.
-
LLM-only extension. We evaluate LLM-only prompt sets after the initial round, halfway through LoUPE's rounds, and at LoUPE's stopping round for each adapter. Coverage increases from 0.64 to 0.81 and 0.87, respectively. The final round adds only 1.2 percentage points. LoUPE stops between iterations 5 and 12, averaging 7.1 across adapters. These results support the stopping rule as an empirical saturation criterion that adapts the number of rounds to each adapter.
-
VLM refinement. After the same number of LLM rounds, the VLM stage increases mean activation from 0.21 to 0.22 while preserving coverage at 0.87. It replaces about 8 of the 65 prompts per adapter. The replacements have 1.7 times the activation of the prompts they replace (0.225 compared with 0.129, difference 0.096, 95% CI [0.088, 0.104]), showing that image-grounded refinement identifies prompts that elicit stronger adapter effects.
We will add these baseline comparisons and their prompt-count protocols to the revised experiments. Consistent with the evaluation scope discussed above, these results concern exploration from informative starting examples and coverage of held-out community usage.
Response to Reviewer KbyZ (3/3)
W3: Necessity of the multiplicative LoUPE Score
Weakness 3: The three factor product in Figure 6 did not show a higher correlation with the overall score than when Precision or Strength were used alone. Therefore, the paper has not fully demonstrated the necessity of using a multiplication combination of three indicators.
We use multiplication because a high LoUPE Score should require stable, precise, and sufficiently strong interventions jointly. A low value in any component suppresses the product, even when the others score highly. The correlation comparison in Fig. 6 does not by itself establish that Precision or Strength alone can replace this joint quality criterion.
Appendix C.2 (Fig. 10) illustrates their distinct failure modes. Low Stability corresponds to inconsistent effects across generations, low Precision to imprecise modifications, and low Strength to barely perceptible interventions. These examples motivate evaluating all three dimensions jointly.
W4: Other weight-based retrieval baselines
Weakness 4: This article mainly compares LoCAL with retrieval methods such as Stylus, Stylus+, and Stylus++, but lacks comparison with other weight retrieval baselines.
Our setting retrieves community LoRAs from text or image queries using candidate weights alone. Weight-learning methods such as ProbeX / Tree Experts and Learning on LoRAs (LoL) are relevant, but their published evaluations do not provide directly comparable results under this retrieval protocol. LoL's GL-equivariant encoder can be adapted to our framework with minimal changes, allowing us to compare weight encoders within the proposed setting.
We therefore conducted an encoder ablation by replacing LoCAL's W2T-based encoder backbone with LoL's GL-equivariant architecture, retaining the same SD1.5 collection, behavioral supervision, and contrastive retrieval objective. Both encoders are evaluated on the same 1,254 held-out test LoRAs, with Recall@K and MRR averaged across LoRAs.
| Weight encoder | Query modality | Candidates | R@1 (%) | R@5 (%) | R@10 (%) | MRR |
|---|---|---|---|---|---|---|
| GL-equivariant | Text | 1,254 | 1.25 | 4.42 | 7.33 | 0.0368 |
| LoCAL | Text | 1,254 | 4.07 | 11.90 | 17.49 | 0.0885 |
| GL-equivariant | Image | 1,254 | 3.05 | 9.39 | 14.11 | 0.0701 |
| LoCAL | Image | 1,254 | 6.03 | 16.21 | 22.92 | 0.1184 |
This ablation shows that encoder choice matters for retrieval and supports the effectiveness of LoCAL's representation in the studied setting.
Dear Reviewer,
The discussion period is nearing its end, and we haven’t yet seen a reply from you. We encourage you to respond to the authors’ comments, especially if you have further questions about points that may affect your assessment.
Thank you for taking the time to engage in the discussion.
Best,
Your AC
Response to Reviewer KbyZ (1/3)
We thank the reviewer for the constructive suggestions on validating prompt exploration and strengthening the evaluation. Below, we address the concerns and questions raised in the review.
W1: Stopping conditions, convergence, and coverage of actual capabilities
We tested whether LoUPE's stopping rule yields reproducible exploration and broad coverage of observed adapter behavior through two additional experiments. With LoUPE initialized from informative community prompts, independent runs show substantial overlap in adapter-effect space, and the final prompt sets cover 87% of held-out community usage.
Agreement across runs. We independently ran LoUPE twice on each of 30 adapters and compared the final prompt sets through nearest-neighbor matching in adapter-effect space (δ_II). On average, 84.5% of prompts in one run have an effect within the nearest-neighbor radius of an effect in the other run. The corresponding overlap is 89.0% between random halves of the same run and 3% between different adapters. The mean effect directions also have cosine similarity 0.925. These results support reproducibility of the explored effects across independent runs.
Coverage of held-out usage. For the 92 adapters with at least four distinct community prompts, we held out a subset that LoUPE never saw. Coverage measures the fraction of held-out prompts whose adapter effect δ_II lies within a nearest-neighbor radius of an effect in the explored set. At equal prompt counts, LoUPE covers 87% of held-out usage, compared with 43% for a fixed generic prompt list from PartiPrompts. Coverage increases from 64% after the initial round to 81% halfway through exploration and 87% at the stopping round. The final round adds only 1.2 percentage points, providing empirical evidence of diminishing coverage gains near the stopping point.
In the revision, we will describe the stopping rule as an empirical saturation criterion and report these tests as evidence of reproducibility across runs and substantial coverage of observed usage. Held-out community prompts provide a proxy for realized adapter behavior. We will make clear that these tests do not establish convergence of the full difference distribution or exhaustive coverage of all capabilities, and that the evaluation uses informative starting examples.