Pocket-Dentist: Benchmarking Compact Vision-Language Models for Dental Image Understanding

Oral diseases affect approximately 3.7 billion people, yet access to specialist screening remains limited for many people. As smartphones become more widely available, compact vision-language models (VLMs) could help extend dental image assessment to settings with limited access to specialists. However, it remains unclear how well VLMs perform across dental imaging modalities and clinical tasks, or whether capable models can run within mobile hardware constraints. We introduce Pocket-Dentist, a large-scale curated dental imaging dataset, together with a comprehensive, deployment-aware benchmark for evaluating vision-language models across dental imaging modalities and clinical tasks. It includes over 50,800 images from more than 5,900 patients, spanning panoramic, periapical, and cephalometric radiographs and intraoral photographs, and covers four task types and 11 sub-tasks. We study dental-task performance across 13 representative VLMs under zero-shot and few-shot prompting, and examine the effects of fine-tuning the open-weight models. Our results show that, for zero-shot and few-shot settings, no VLM performs significantly better than others across all tasks. However, after fine-tuning, compact 4B VLMs such as Qwen3.5-4B become competitive with larger VLMs. We also observe that, even trained on large scale medical data, state-of-the-art medical VLMs provides no consistent advantage on dental understanding domain. To support practical deployment, we further proposed an on-device framework and deployed best performing VLMs. Our benchmark suggests that compact VLMs is practical for on-device use in dental image undertstanding. Our project page is available at https://anonymous.4open.science/r/Pocket-Dentist-Benchmark-Anonymous-62B3.