- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
LexDINO modifies DINO-style visual self-distillation by replacing the learned image-level prototype bank with fixed centroids derived from clusters of an independent caption corpus in an LLM feature space. To enable learning a structure, for which a single target prototype does not suffice, top-k routing targets supervise LexDINO. The results show gains over a DINOv3 baseline, particularly for OCR-heavy VLM tasks.
Critical strengths
The paper studies an interesting tension: DINO-style encoders are strong for dense visual tasks, whereas image-text encoders are often better VLMs and provide better text-image alignment. The proposed shared group index is a good concept and addresses the ambiguity that a fixed semantic bank would leave unresolved.
The experiments cover a wide spectrum: recognition, segmentation, depth, general VQA/reasoning, OCR/document tasks, multiple backbone sizes, and multiple downstream language models.
The authors examine various interesting elements: feature distillation versus index alignment, prototype representation, caption source.
The results are convincing.
Critical weaknesses
The paper contrasts against DINOv3, a projected SigLIP2 feature loss, and several released encoders. It does not compare against dino.txt-style DINO-language alignment, EVA alternatives, or a paired/unpaired contrastive alternative under comparable data and compute. Such comparisons would provide the argument that the shared-index formulation is beneficial rather than one effective way of injecting language.
If grouping using language is the way to go, could be inferred from non-language sources to guide the self-distillation, e.g. SigLIP2-defined grouping or another alignment without injecting LLM information.
I would be very interested in the resulting prototypes: are they interpretable, visually grounded, balanced, (non-)redundant?
Questions for authors
How much of the controlled OCR gain is attributable to the frozen SigLIP2 router rather than to the LLM-derived prototype geometry?
What happens if caption groups are formed directly in SigLIP2 text space, using the same K and cluster-size controls, instead of retaining Qwen assignments?
Can you report downstream results for a random fixed prototype bank and for the alignment term with random or permuted shared indices? This would isolate geometry, stable identities, and router supervision.
Can you diagnose the OCR gains?
Reply to Rebuttal
I am happy with the rebuttal. It isolates the contributions of the paper more precisely, by comparing to alternatives, and cross-checking multiple technical options. The gains are now smaller compared to those alternatives, but now they can be attributed to the technical novelties in the paper, and overall it yields stable improvements and is therefore more convincing. I have also read the other reviews. Altogether it made me conclude to increase my score to a strong accept.