- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Small-Large Collaboration for training-efficient VLM personalization. The method trains a small VLM using CLIP-based clustering and meta-concept LoRA adapters. A large VLM verifies personalized concepts through test-time reflection. Experiments on Yo'LLaVA and MC-LLaVA show competitive performance with lower training costs.
Critical strengths
-
The proposed framework avoids direct fine-tuning of large VLMs and supports both open-source and closed-source models.
-
The small-large collaboration design is modular and practical. The combination of meta-concept adapters and test-time reflection provides a potentially useful approach to personalized inference.
Critical weaknesses
-
MyVLM [1] already combines external personalized concept recognition with VLM generation. RAP [2] supports new concept registration without per-concept fine-tuning. Cache-of-Thought [3] explores small-large VLM collaboration. The paper should clarify its technical novelty beyond combining existing components and provide stronger comparisons with these methods.
-
Section 3.3 uses only 10 meta-concept adapters selected through CLIP similarity. Different identities with similar appearances may select the same adapter. It is unclear how the model distinguishes such identities without explicitly preserving identity-specific reference information. This raises concerns about category-level recognition rather than genuine instance-level personalization [1,4,5].
-
Inconsistent adapter selection descriptions. Equation (1) selects one adapter using averaged embeddings from all registered concepts. However, Figure 3 suggests independent adapter selection for each concept. These strategies differ significantly in multi-concept scenarios.
-
Insufficient identity verification in reflection. Section 3.4.2 verifies broad category descriptions and spatial locations rather than unique identities. A visually similar but incorrect object may satisfy both verification questions. Incorrect spatial predictions may also cause true concepts to be removed.
-
Numerical inconsistencies. Figure 1 claims the highest average accuracy for SLC (0.933), but Table 1 gives MC-LLaVA a higher average of 0.9405. The claimed two-order training-cost reduction is only about 41 times relative to MC-LLaVA. Section 4.2.2 reports SQA accuracy as 0.900, while Table 1 reports 0.925. The GPT-4o results discussed in Section 4.2.2 also differ from the GPT-5.5 results in Table 1.
-
Section 4.3.1 claims six settings plus two baselines, but Table 2 contains six rows in total.
References:
[1] Alaluf et al. MyVLM: Personalizing VLMs for User-Specific Queries. ECCV, 2024.
[2] Hao et al. RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models. CVPR, 2025.
[3] Wu et al. Cache-of-Thought: Master-Apprentice Framework for Cost-Effective Vision Language Model Inference. arXiv:2502.20587, 2025.
[4] Nguyen et al. Yo'LLaVA: Your Personalized Language and Vision Assistant. NeurIPS, 2024.
[5] Pi et al. Personalized Visual Instruction Tuning. arXiv:2410.07113, 2024.
Questions for authors
Q1. How does the small VLM distinguish unseen identities with similar appearances and the same selected adapter? Are reference images or identity-specific embeddings used beyond adapter selection? Please provide fine-grained identity recognition and adapter collision experiments.
Q2. Is adapter selection performed per concept or per scenario? Equation (1) and Figure 3 appear inconsistent. How are multiple concepts from different categories handled during inference?
Q3. What advantages does SLC provide over retrieval-based personalization methods such as RAP? Could the authors compare SLC with direct CLIP retrieval and lightweight external concept detectors under matched settings?