- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents Ophth-500K, a dataset of 467,697 image–text–audio samples curated from 29,465 publicly accessible ophthalmology videos totaling 14,769 hours. The pipeline screens video metadata, classifies sampled frames for ophthalmic relevance, segments temporally stable visual content, aligns nearby speech using Whisper timestamps, extracts ophthalmic regions using SAM3 and BiomedCLIP, masks detected identifying information, and uses Qwen3-VL-8B to convert the image and transcript into a concise clinical description.
Critical strengths
Ophth-500K is substantially larger than most public ophthalmology image–text resources and retains audio in addition to images and text. The pipeline tackles clinically relevant challenges including temporal narration offset, panel extraction, and privacy filtering.
Critical weaknesses
-
Dataset quality and clinical validity are insufficiently evaluated. The paper reports no clinician audit or measurements of alignment, caption factuality, modality classification, privacy filtering, or cross-video duplication. Retrieval on captions produced by the same pipeline mainly demonstrates in-domain learnability rather than clinical accuracy or downstream utility.
-
The release, licensing, and privacy basis is unclear. Publicly accessible videos are not necessarily redistributable. The paper does not specify applicable licenses, releasable content, platform compliance, takedown procedures, or privacy-audit results, despite labeling the dataset as public.
Questions for authors
-
Are all 467,697 Qwen-generated descriptions reviewed by humans, as the AI Use Statement may imply?
-
Do the authors have evaluation on an independently curated ophthalmology dataset or a downstream clinical task that does not use captions generated by the same pipeline?
Clinical validity and external evaluation
We thank the reviewer for the constructive feedback.
W1/Q1
The original expert verification referred to manual inspection of selected samples, not exhaustive clinical review of all 467,697 descriptions. We subsequently conducted a systematic audit of 100 training and 100 evaluation samples across seven quality criteria. Three independent medical imaging researchers assessed the samples, with disagreements adjudicated by a senior clinician. The audit includes image validity, modality, cropping, image–text alignment, invented content, clinical accuracy, and patient privacy, together with inter-rater agreement statistics. The complete results and protocol are reported in our response to Reviewer bjw1 (W1/Q1).
W2
Ophth-500K has not yet been publicly released. We plan a two-stage release strategy, subject to applicable source licenses, platform policies, institutional requirements, and relevant IRB requirements.
First, we plan to release the complete data curation pipeline, source video identifiers, temporal annotations, processing configurations, and pretrained OphthCLIP weights through GitHub and Hugging Face. Researchers can reconstruct the dataset from its original sources where access and permissions allow.
Second, we plan to progressively release curated image–text pairs following individual manual review for clinical content, privacy, and redistribution eligibility. We target approximately 50,000 samples per month, subject to the necessary reviews and approvals. Licenses and usage conditions will be documented separately for code, weights, metadata, and releasable samples.
We also plan to establish a public contact channel for content creators, rights holders, and affected individuals to request removal. Requests will be reviewed using source provenance records, and affected entries will be removed from hosted releases and reconstruction manifests where warranted. We will revise the Public designation in Table 1 and clarify the release conditions in the manuscript.
Q2
We evaluated OphthCLIP on 10 independently annotated ophthalmic classification datasets and MM-Retinal. The complete evaluation protocol, retrieval results, and caption refinement experiments are reported in our response to Reviewer bjw1 (W2/Q3).
We further separated the ten classification datasets into three OCT datasets (OCTID, OCTDL, Kermany) and seven fundus datasets (APTOS2019, IDRiD, MESSIDOR-2, PAPILA, GlaucomaFundus, Retina, JSIEC) to examine modality-specific transfer performance:
All values are balanced accuracy percentages. ZS denotes zero-shot classification; Probe denotes linear probing. † UniMed-CLIP pretraining includes some evaluated data. ‡ RET-CLIP uses a Chinese text encoder and was evaluated with English prompts.
It is worth noting that OphthCLIP was trained on only 0.47M ophthalmic image–text pairs. The reported corpus sizes are over 5.3M for UniMed, 15M for PMC-15M (BiomedCLIP), and over 24M for the BIOMEDICA archive underlying BMC-CLIP.
References
OCT datasets. OCTID, OCTDL, Kermany OCT.
Fundus datasets. APTOS2019, IDRiD, MESSIDOR-2, PAPILA, Glaucoma Fundus, Retina (Cataract Dataset), JSIEC.
Baselines. CLIP, SigLIP, PMC-CLIP, BiomedCLIP, UniMed-CLIP, BMC-CLIP (BIOMEDICA), EyeCLIP, RET-CLIP, FLAIR.