- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces ViMed-Rad, a Vietnamese-language medical imaging dataset comprising 93,781 clinical studies across X-ray, CT, MRI, and whole-body PET/CT, paired with radiology reports. Building on this dataset, the authors develop ViMed-Bench, which combines report generation and report-grounded visual question answering (VQA), and propose a unified vision-language framework with modality-specific visual encoders and a shared language model. The study also evaluates existing medical vision-language models (VLMs) in Vietnamese and investigates the effect of joint multimodal training and additional translated public data.
Critical strengths
The work addresses an important problem: the scarcity of native-language medical image-report datasets for low-resource languages, particularly for volumetric and functional-anatomical imaging. The inclusion of native clinical imaging and radiologist-authored reports across four modalities is potentially valuable. The collection of native DICOM studies paired with Vietnamese radiology reports addresses a meaningful gap in medical vision-language research. Including CT, MRI, and PET/CT alongside X-ray broadens the scope beyond conventional chest radiography datasets.
Critical weaknesses
Major concern.
-
The dataset is a central contribution of this work. However, the reproducibility statement explicitly indicates an intention to release the implementation and reproduction instructions upon publication, while the manuscript does not provide a comparably clear, verifiable commitment regarding public access to ViMed-Rad itself. Whether ViMed-Rad will be publicly released, including the native imaging data, corresponding reports, and derived VQA annotations, or whether controlled access, a data-use agreement, or an evaluation server will be provided. This distinction is particularly important because the dataset and benchmark appear to be the primary contributions. If the core dataset remains inaccessible and no practical alternative enables independent evaluation, the reproducibility and broader community impact of the work would be substantially reduced.
-
Table 1 compares datasets across modalities, formats, sample counts, text supervision, language, and data sources. However, the current presentation makes it difficult to compare the actual scale and dimensionality of the imaging data, particularly for volumetric modalities. I recommend adding the following information for each relevant dataset and modality: Number of studies and unique patients. Number of 3D volumes, where applicable. Total number of slices and average slices per volume, where applicable.
For example, a dataset containing thousands of 3D CT volumes is not directly comparable to a dataset containing a similar number of selected 2D slices. The authors should also clearly distinguish the size of the full ViMed-Rad collection from the smaller modality-specific subsets actually used in the experiments. This is particularly important for PET/CT, where the full dataset contains 6,376 scans but the experimental subset and region-level report counts differ.
The related-work discussion covers several established resources. However, it appears to omit several highly relevant works that should be discussed to better establish the novelty and positioning of ViMed-Rad as follows.
[1] Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Bench. [2] MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT [3] PETWB-REP: A Multi-Cancer Whole-Body FDG PET/CT Dataset with Corresponding Radiology Reports [4] Development of a large-scale grounded vision language dataset for chest CT analysis
-
The manuscript describes CT, MRI, and PET/CT as volumetric data and uses modality-specific encoders. However, the evaluation includes baselines with different capabilities and input representations. Some models directly encode 3D volumes, whereas others receive selected axial slices or grids of slices. The authors should report, for every evaluated model and modality: Whether the model receives a native 3D volume, individual 2D slices, a 2D slice grid, or a combination of these representations. The number of slices used, the slice-selection strategy, and the anatomical coverage. For PET/CT, whether the input contains PET, CT, fused PET/CT, or additional projections such as maximum-intensity projections.
-
The paper demonstrates that existing medical VLMs can perform substantially worse when evaluated against Vietnamese references than against English references. However, the evaluation does not appear to include the important alternative of generating an English report first and then translating it into Vietnamese using a strong translation system or LLM.
I recommend adding the following comparisons on the same test images:English report generation followed by machine translation into Vietnamese. English report generation followed by translation using a representative LLM, such as GPT. This experiment would help determine whether the observed language gap primarily reflects limitations in visual understanding, target-language generation, or the translation process. It would also clarify the practical benefit of training directly on native Vietnamese clinical reports.
- The manuscript acknowledges that the data originate from a limited number of centres and that PET/CT comes from a different source than the other modalities. This raises a potential confounding issue: observed modality-specific differences may also reflect differences in institutions, scanners, acquisition protocols, patient populations, or reporting conventions.
The authors should clarify the provenance of each modality and report whether the data splits prevent patient overlap across training, validation, and test sets. They should also provide more detailed information on acquisition protocols and, where possible, evaluate performance across distinct centres or acquisition settings. For PET/CT in particular, generalization across tracers, disease types, and acquisition protocols would be important. If external validation is not available, the manuscript should explicitly distinguish internal test performance from evidence of cross-centre generalization.
Minor Comments
Please use consistent terminology to distinguish a study, patient, volume, image, slice, and region-level report throughout the manuscript.
Please report the number of unique patients alongside the number of studies for each modality, and clearly explain the PET/CT conversion from whole-body scans to region-level report-generation and VQA examples.
Please ensure that all reported comparisons use clearly specified validation and test splits, and provide confidence intervals or statistical significance analyses where appropriate.
Please clarify which public datasets contribute to each training configuration and whether translated reports are used for training, validation, testing, or multiple purposes.
Please provide a clear data and code availability statement, including repository links, licenses, access restrictions, and the intended release timeline.
Questions for authors
I appreciate the importance of addressing the Vietnamese-language gap in medical vision-language research, and I agree that this work has the potential to make a valuable contribution by providing a multimodal dataset with native clinical reports and a benchmark spanning multiple imaging modalities, including volumetric PET/CT. However, several important questions remain regarding the accessibility and reproducibility of the dataset, its positioning relative to closely related recent resources, the fairness and comparability of the 2D/3D evaluation protocols, and the evidence supporting the benefits of the proposed unified framework. I encourage the authors to address these concerns and clarify the corresponding methodological details and experimental evidence. My current assessment is Weak Accept, reflecting the potential value of the work, but this score is tentative. I will reconsider my rating after evaluating the authors’ responses, depending on how satisfactorily these concerns are resolved.
7 comments
Response to Reviewer uiPP (Part 2)
W2. Table 1 compares datasets across modalities, formats, sample counts, text supervision, language, and data sources. However, the current presentation makes it difficult to compare the actual scale and dimensionality of the imaging data, particularly for volumetric modalities. I recommend adding the following information for each relevant dataset and modality: Number of studies and unique patients. Number of 3D volumes, where applicable. Total number of slices and average slices per volume, where applicable.
For example, a dataset containing thousands of 3D CT volumes is not directly comparable to a dataset containing a similar number of selected 2D slices. The authors should also clearly distinguish the size of the full ViMed-Rad collection from the smaller modality-specific subsets actually used in the experiments. This is particularly important for PET/CT, where the full dataset contains 6,376 scans but the experimental subset and region-level report counts differ.
Thank you for this suggestion. In the revised manuscript, Table 1 reports the number of samples, unique patients and slices of each dataset, and Table 3 gives, for each ViMed-Rad modality of the full cohort, the studies, unique patients, images or volumes per study, total slices and slices per volume. Figure 4 and Table A1 separate the full collection from the subset used in the experiments: they trace each modality from the full cohort to the benchmark splits and report unique patients, studies and report-generation targets per split. For PET/CT, they distinguish the 6,376 scans of the full cohort from the 2,757 scans of the benchmark subset, which carry 8,271 region-level reports. For the Reviewer's convenience, the table below consolidates these numbers and adds the number of volumes and slices of the benchmark subset.
Dataset scale (consolidated from Tables 1, 3 and A1 of the revised manuscript). Comparison of medical imaging-text datasets. 2D/3D: planar/volumetric imaging; Studies: imaging examinations (scans for CT-RATE and ViMed-PET, cases for RadGenome-Brain MRI, captioned cases for M3D-Cap); Patients: unique patients; Images / vol.: 2D images, or 3D volumes for 3D modalities (DICOM series for ViMed-Rad MRI); Slices: total 2D slices over all volumes, with the mean per volume in parentheses. n.r.: not reported by the original publication (PMC-OA: 1.65M subfigures cropped from 381K figures of PubMed Central articles; OmniMedVQA: 118K images pooled from 73 public datasets; MedMD: 2D and 3D scans pooled from web and public sources). For ViMed-Rad, the full cohort and the benchmark subset used in all experiments are listed separately; patients are counted within each modality.
| Dataset | X-ray | CT | MRI | PET/CT | Format | Studies | Patients | Images / vol. | Slices | Text Type | Lang. | Source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MIMIC-CXR | 2D | – | – | – | DICOM | 227,835 | 65,379 | 377,110 | – | Report | EN | PhysioNet |
| CT-RATE | – | 3D | – | – | NIfTI | 25,692 | 21,304 | 50,188 | >14.3M | Report | EN | CT-RATE |
| RadGenome-Brain MRI | – | – | 3D | – | NIfTI | 1,007 | n.r. | 3,408 | n.r. | Report | EN | Public MRI |
| PMC-OA | 2D | 2D | 2D | 2D | JPEG | n.r. | n.r. | 1.65M | – | Caption | EN | PMC OA |
| OmniMedVQA | 2D | 2D | 2D | – | PNG | n.r. | n.r. | 118K | – | VQA | EN | Websites |
| M3D-Data | – | 3D | – | – | Numpy | 42,496 | n.r. | 120K | n.r. | Text & VQA | EN | Websites |
| MedMD | 2D | 3D | 3D | – | Numpy | n.r. | n.r. | 16M | n.r. | Text & VQA | EN | Websites |
| ViX-Ray | 2D | – | – | – | PNG | n.r. | n.r. | 5,400 | – | Report | VI | Hospital |
| ViMed-PET | – | – | – | 3D | DICOM | 2,757 | 2,182 | 5,514 | 1.57M (284) | Report & VQA | VI | Hospital |
| ViMed-Rad (Ours), full cohort | ||||||||||||
| X-ray | 2D | – | – | – | DICOM | 62,148 | 54,077 | 89,514 | – | Report | VI | Hospital |
| CT | – | 3D | – | – | DICOM | 13,247 | 11,830 | 68,581 | 19.59M (286) | Report | VI | Hospital |
| MRI | – | – | 3D | – | DICOM | 12,010 | 11,410 | 155,567 | 5.59M (36) | Report | VI | Hospital |
| PET/CT | – | – | – | 3D | DICOM | 6,376 | 6,309 | 12,752 | 3.58M (280) | Report | VI | Hospital |
| Total | 93,781 | |||||||||||
| ViMed-Rad (Ours), experimental subset | ||||||||||||
| X-ray | 2D | – | – | – | DICOM | 37,369 | 34,570 | 40,242 | – | Report & VQA | VI | Hospital |
| CT‡ | – | 3D | – | – | DICOM | 5,428 | 5,023 | 26,267 | 8.02M (305) | Report & VQA | VI | Hospital |
| MRI | – | – | 3D | – | DICOM | 3,649 | 3,648 | 62,407 | 2.68M (43) | Report & VQA | VI | Hospital |
| PET/CT | – | – | – | 3D | DICOM | 2,757 | 2,182 | 5,514 | 1.57M (284) | Report & VQA | VI | Hospital |
Response to Reviewer uiPP (Part 3)
W3. The related-work discussion covers several established resources. However, it appears to omit several highly relevant works that should be discussed to better establish the novelty and positioning of ViMed-Rad as follows.
[1] Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Bench. [2] MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT [3] PETWB-REP: A Multi-Cancer Whole-Body FDG PET/CT Dataset with Corresponding Radiology Reports [4] Development of a large-scale grounded vision language dataset for chest CT analysis
Thank you for this valuable suggestion. We have revised the Related Work section to include and discuss all four references: PET-Bench, MetaStructAtlas, PETWB-REP and RadGenome-Chest CT. These resources add spatial or metabolic supervision, but each covers a single modality in English: RadGenome-Chest CT grounds chest CT reports in segmented regions, MetaStructAtlas grounds whole-body PET/CT findings in organs, PET-Bench probes PET perception through hierarchical VQA, and PETWB-REP pairs 490 whole-body PET/CT scans with reports. In contrast, ViMed-Rad pairs native Vietnamese reports across four modalities with report-grounded VQA tasks of increasing difficulty. This discussion is now available in the Related Work section (Section 2) of the revised manuscript.
W4. The manuscript describes CT, MRI, and PET/CT as volumetric data and uses modality-specific encoders. However, the evaluation includes baselines with different capabilities and input representations. Some models directly encode 3D volumes, whereas others receive selected axial slices or grids of slices. The authors should report, for every evaluated model and modality: Whether the model receives a native 3D volume, individual 2D slices, a 2D slice grid, or a combination of these representations. The number of slices used, the slice-selection strategy, and the anatomical coverage. For PET/CT, whether the input contains PET, CT, fused PET/CT, or additional projections such as maximum-intensity projections.
We thank the Reviewer for this suggestion. We have added Table A4 (Appendix D) to the revised manuscript, which summarizes the native image input of each zero-shot baseline as specified in its original publication:
Table A4. Native image input of the zero-shot baselines, as specified in their original publications.
| Model | Native input | X-ray | CT | MRI |
|---|---|---|---|---|
| RadFM | 2D and 3D; 3D ViT | 512×512, depth 4 | 256×256×D, D ≤ 64 | 256×256×D, D ≤ 64 |
| LLaVA-Med | Single 2D image; CLIP ViT | Single image | – | – |
| MedVersa | 2D (Swin-B) and 3D (3D U-Net); 224×224 | Multi-view, 224×224 | Seg. only† | Seg. only† |
| M3D | 3D CT; 3D ViT | – | 32×256×256 | – |
| MedGemma 1.5 | 2D, 896×896; volumes as 2D slices | Frontal (+ prior) | ≤ 85 axial slices | ≤ 85 axial slices |
–: not supported. †3D encoder trained for segmentation only; no CT/MRI report generation.
Building on this table, Appendix D details the input actually given to each model. RadFM and M3D encode the CT and MRI volume directly. Models without native volumetric support receive evenly spaced axial slices: LLaVA-Med as a 4×4 grid of 16 slices, MedVersa as 16 separate slices, and GPT-4o and DeepSeek-v4.1 as 4×4 grids, one per available MRI sequence. For the few-shot baselines, CT and MRI (FLAIR) are tiled into one 5×5 grid of 25 slices. None of the baselines supports PET/CT natively; they receive PET slices as eight 5×5 grids (200 slices per region) in the few-shot setting, and 4×4 grids plus a coronal maximum-intensity projection in the zero-shot setting (GPT-4o and DeepSeek-v4.1). X-ray images are provided as acquired.
Response to Reviewer uiPP (Part 1)
We thank the Reviewer for the careful and constructive review, and for recognizing the value of native Vietnamese image–report data across four modalities, including volumetric PET/CT. We address each point below; the corresponding changes are included in the revised manuscript.
We thank the Reviewer for raising this important point. The complete dataset will be released upon acceptance, including all de-identified native DICOM studies, the radiologist-authored Vietnamese reports, the VQA annotations, the data splits, and the preprocessing, training and evaluation code. It will be hosted on Hugging Face as a gated repository under the CC BY-NC-SA 4.0 licence with a data use agreement: users submit a short access form and agree to non-commercial research use, no re-identification and no redistribution. During the discussion period, a de-identified sample is already available at https://anonymous.4open.science/r/anonymous_project-42BD/README.md: all 37,369 chest X-ray studies of the benchmark subset, and 140 studies each of CT, MRI and PET/CT (100 training, 20 validation, 20 test), with their reports, VQA items and splits.
We have revised the Reproducibility Statement accordingly: it now states the content of the release, the licence, the access procedure, the timeline and the link to the sample.