- Rating
- 2
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper investigates VLM fine-tuning as a process of capability evolution rather than focusing solely on final model performance. Using four evaluation probes across multiple training checkpoints, the authors examine when different capabilities emerge, reach their peak, and how much of their performance is retained at the deployment checkpoint, which is consistently selected based on a class-name retrieval metric. The paper proposes a structured semantic routing mechanism that prevents the attribute branch from accessing class-name information, along with a stochastic dropping strategy for the class-name branch. The main finding is that class-name-free attribute recognition often peaks before the selected deployment checkpoint, and continued training may subsequently degrade this capability.
Critical strengths
-
The paper provides extensive experimental evaluations, including comprehensive ablation studies and controlled comparisons, which support its main findings.
-
The experimental setup and dataset splits are clearly documented, facilitating reproducibility.
Critical weaknesses
-
The writing and organization could be further improved to present the key contributions and findings more clearly.
-
The paper lacks comparisons with specialized fine-grained vision-language methods, such as FG-CLIP.
-
The main capability trajectories are based on only a single run per experimental configuration. The observed differences among the six backbones may reflect both model-specific characteristics and training stochasticity, making it difficult to disentangle these effects.
-
The training configurations are not standardized across backbones. For example, the warmup duration varies from 1,000 to 10,000 steps. These differences may introduce confounding factors into cross-backbone comparisons.
Questions for authors
Why are different training configurations used across backbones?