acceptodds
Under review as a conference paper at ICLR 2027

CariesBench: A Hierarchical Multi-view Benchmark for Dental Caries Screening and Diagnostic Reasoning from 3D Intraoral Scans

Abstract

Dental caries assessment is a structured clinical process requiring lesion localization, identification of the affected tooth and surface, and severity grading rather than simple image-level recognition. 3D intraoral scans (IOS) preserve detailed dental geometry and surface morphology and support standardized rendering from complementary viewpoints, enabling fine-grained visual assessment while introducing a distinct need for anatomically grounded multi-view reasoning. Despite recent advances in multimodal large language models (MLLMs) and dental vision-language benchmarks, existing evaluations do not jointly assess hierarchical arch-to-tooth-to-surface reasoning, cross-view evidence integration, and surface-level caries assessment. To address this gap, we introduce CariesBench, a hierarchical benchmark constructed from 5,932 3D IOS, comprising 85,455 standardized multi-view renderings and 133,994 clinically grounded visual question answering instances with expert quality control. CariesBench comprises three complementary task families: Arch-level Clinical Screening (ACS), Tooth-level Diagnostic Reasoning (TDR), and Structured Clinical Decision-Making (SCD). Using the CariesBench training set, we further fine-tune Qwen3-VL-8B-Instruct to obtain IntraoralGPT. Extensive evaluation of 24 representative MLLMs reveals a pronounced global-to-local reasoning gap, with substantially stronger performance on arch-level screening than on fine-grained tooth- and surface-level reasoning. CariesBench-specific supervision substantially improves the Qwen3-VL-8B-Instruct baseline, increasing the 19-task macro-average clinical accuracy from 52.7% to 69.5% and TDR Overall from 34.4% to 44.7%. Despite these gains, fine-grained TDR remains challenging, highlighting persistent difficulty in anatomical localization and cross-view evidence integration. CariesBench thus provides a clinically grounded testbed for evaluating hierarchical multi-view reasoning in digital dentistry.

Primary area:
datasets and benchmarks
Keywords:
Multimodal Large Language Models, Dental Benchmark, 3D Intraoral Scans, Dental Caries, Multi-view Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_bTTFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CariesBench, a benchmark for evaluating dental caries assessment from standardized multi-view renderings of 3D intraoral scans. It contains 5,932 scans from 2,966 patients, 85,455 rendered images, and 133,994 VQA instances across 19 question types covering arch-level clinical screening, tooth-level diagnostic reasoning, and structured clinical decisions. The construction pipeline separates structured clinical annotations from LLM-based language realization and includes automated auditing and expert validation. Evaluation uses patient-disjoint training, validation, and test partitions. The authors benchmark 24 general-purpose and medical/dental MLLMs and report substantially lower performance on tooth- and surface-level tasks than on arch-level screening. They also fine-tune Qwen3-VL-8B-Instruct with LoRA to obtain IntraoralGPT, reporting an increase in the 19-task macro-average accuracy from 52.7% to 69.5%. Fine-grained surface assessment remains challenging, with the resulting TDR Overall score reaching 44.7%.

Critical strengths

S1: The benchmark connects arch-, tooth-, and surface-level assessment through views rendered from the same underlying anatomy. Its combination of surface-specific clinical states, complementary observations, and multiple question formats provides a useful resource for studying dental MLLMs beyond image-level recognition.

S2: The patient split is applied before downstream sampling and training-set rebalancing, and the construction explicitly separates clinical labels from LLM-generated prose. The reported 20% expert validation, double assessment of 800 instances, and evaluation of 24 baseline models strengthen the empirical foundation. The same-backbone fine-tuning comparison also provides evidence that the benchmark supervision is useful within the reported setting.

Critical weaknesses

W1: The definition and distribution of Sound Tooth Verification (SV) are not sufficiently consistent to establish the validity of this subtask. Appendix F.2 (p. 18) states that non-sound teeth are used for SA/FS, whereas sound teeth are used for SV. Taken literally for the test set, this makes the SV target constant: always predicting “sound” would achieve 100% under the binary metric defined in Appendix K.1. However, Fig. 2(e) distributes SV instances across L0–L7, with only 24.0% assigned to L0. It is unclear whether the construction description is incomplete or the figure uses a different label definition. This matters because SV contributes one-third of TDR Overall and supports the comparison with SA/FS in Finding 1. The actual test-target counts, their relation to the plotted labels, and a constant/majority-answer baseline are needed to resolve this ambiguity.

W2: The experiments do not sufficiently distinguish the observed task difficulty gap from the specific explanation in terms of anatomical localization and cross-view integration. Finding 2 reports a 29.1-point difference between mean ACS and TDR scores. As the authors acknowledge, these groups differ in task format as well as anatomical granularity; Table 8 and Appendix K also show different prediction targets and scoring units. This supports a descriptive performance gap on the constructed tasks, but does not isolate its source. The paper does not report a comparison that holds the clinical target and answer format fixed while varying the available views, or an input ablation establishing dependence on cross-view evidence. The five-step response protocol and final-label accuracy do not independently establish such dependence. A limited paired analysis on existing test cases, or a more restricted interpretation of the findings, would materially strengthen the benchmark's central scientific claim.

Questions for authors

Q1: Does the SV test set contain both sound and non-sound teeth? Please reconcile Appendix F.2 with Fig. 2(e), specifying what the L0–L7 labels in that plot represent and reporting the actual binary target counts for SV in each split. What does an always-sound or majority-answer predictor achieve, and would correcting the description or construction change the reported TDR and aggregate results?

Q2: What evidence separates a limitation in cross-view integration from differences in task definition and answer difficulty? On a limited subset of existing held-out targets, can the authors compare a relevant single view with all views while holding the clinical target, label space, and answer format fixed? Alternatively, please clarify which conclusions are intended only as descriptive comparisons across the current tasks. Such evidence or clarification would affect my assessment of the claimed reasoning insight.

7 comments
AuthorsOpenReview ↗

Response to Reviewer bTTF

We thank you for the careful assessment of CariesBench. We particularly appreciate your recognition of the benchmark’s hierarchical design, patient-level partitioning, clinical annotations, and expert quality control. We address two concerns regarding the finalized SV task construction and the interpretation of the cross-task performance disparity. We clarify the finalized SV construction and label mapping, report the full-benchmark L0--L7 distribution and the binary class distributions before and after training-set rebalancing, provide an always-sound baseline, and present controlled within-task multi-view ablations. We also distinguish the descriptive performance gap across task families from the evidence provided by controlled multi-view comparisons.


W1: Ambiguity in Sound Tooth Verification (SV) Construction and Class Distribution. The definition and distribution of Sound Tooth Verification (SV) are not sufficiently consistent to establish the validity of this subtask. Appendix F.2 states that non-sound teeth are used for SA/FS, whereas sound teeth are used for SV. Taken literally for the test set, this makes the SV target constant: always predicting "sound" would achieve 100% under the binary metric defined in Appendix K.1. However, Figure 2(e) distributes SV instances across L0--L7, with only 24.0% assigned to L0. It is unclear whether the construction description is incomplete or the figure uses a different label definition. This matters because SV contributes one-third of TDR Overall and supports the comparison with SA/FS in Finding 1. The actual test-target counts, their relation to the plotted labels, and a constant/majority-answer baseline are needed to resolve this ambiguity.

Q1: SV Task Definition, Clinical-status Distribution, and Constant-answer Baseline. Does the SV test set contain both sound and non-sound teeth? Please reconcile Appendix F.2 with Figure 2(e), specifying what the L0--L7 labels in that plot represent and reporting the actual binary target counts for SV in each split. What does an always-sound or majority-answer predictor achieve, and would correcting the description or construction change the reported TDR and aggregate results?

A1: We thank you for identifying this inconsistency. An intermediate version of our SV construction script contained a sound-only eligibility condition. This condition was removed before submission, and the finalized SV construction allows both sound and non-sound teeth to be included. However, Appendix F.2 inadvertently retained an outdated description from the intermediate construction protocol, stating that non-sound teeth were used for SA/FS whereas only sound teeth were used for SV. This statement did not describe the finalized benchmark. All reported experiments were conducted using the finalized benchmark containing both sound and non-sound SV instances. Thus, the inconsistency was confined to the outdated appendix description and did not affect the evaluation data or reported results. We apologize for this oversight and have removed the superseded SV-specific eligibility statement from Appendix F.2, while retaining the shared clinical annotation framework across SA, FS, and SV.

In the finalized benchmark, SA, FS, and SV are based on the same L0--L7 clinical-status taxonomy, derived from structured surface-level annotations. Their differences lie in anatomical scope and prediction format, rather than in the underlying clinical annotation system. For SV, the tooth-level binary target is derived from the relevant surface annotations: a tooth is classified as Sound (L0) if all its relevant annotated surfaces are L0, and as Non-sound (L1--L7) if at least one relevant annotated surface has a clinical status from L1--L7. Thus, L0--L7 represent the underlying clinical-status categories, whereas the final SV prediction space is binary (Sound/Non-sound). The L0--L7 distribution reported in Figure 2(e) therefore describes the underlying clinical-status composition of the full TDR-SV benchmark set.

The full TDR-SV Benchmark set contains 10,116 instances, including 2,425 L0 and 7,691 L1--L7 instances. Figure 2(e) shows the L0--L7 clinical-status distribution of the full TDR-SV Benchmark set. Before training-set rebalancing, the full benchmark is partitioned into a training pool of 7,969 instances (953 Sound and 7,016 Non-sound), together with 1,072 validation instances (735 Sound and 337 Non-sound) and 1,075 test instances (737 Sound and 338 Non-sound). After task-specific training-set rebalancing, the training split contains 6,041 instances (1,550 Sound and 4,491 Non-sound), while validation and test remain unchanged. The post-rebalancing task-specific splits therefore contain 8,188 instances, comprising 3,022 Sound and 5,166 Non-sound instances.

AuthorsOpenReview ↗

Response to Reviewer bTTF

Table R1. L0--L7 Clinical-status Distribution in the Full TDR-SV Benchmark Set

Clinical statusNumber of instancesPercentage (%)
L02,42523.97
L11,44414.27
L26776.69
L31,85718.36
L41,11110.98
L57777.68
L61,09010.78
L77357.27
Total10,116100.00

For the task-specific SV splits, the actual binary target counts are reported in Table R2.

Table R2. SV Binary Class Distribution Before and After Training-set Rebalancing

Data partition / stageSound (L0)Non-sound (L1--L7)Total
Pre-rebalancing training pool953 (11.96%)7,016 (88.04%)7,969
Training (after rebalancing)1,550 (25.66%)4,491 (74.34%)6,041
Validation735 (68.56%)337 (31.44%)1,072
Test737 (68.56%)338 (31.44%)1,075
Post-rebalancing task-specific splits3,022 (36.91%)5,166 (63.09%)8,188

We agree that overall accuracy alone is insufficient for evaluating SV under the observed test-set class distribution. We therefore report an always-sound baseline using Accuracy, Macro-F1, Balanced Accuracy, and non-sound recall, as shown in Table R3. The test split contains 737 Sound and 338 Non-sound instances among 1,075 samples. Since sound teeth form the majority class, the always-sound predictor corresponds to the majority-class baseline. It achieves 68.56% Accuracy, 40.67% Macro-F1, 50.00% Balanced Accuracy, and 0.00% non-sound recall. This baseline highlights why accuracy should be interpreted alongside class-sensitive metrics.

Table R3. Always-Sound Baseline on the SV Test Set

MetricAlways-sound baseline
Sound instances737
Non-sound instances338
Total test instances1,075
Accuracy (%)68.56
Macro-F1 (%)40.67
Balanced Accuracy (%)50.00
Non-sound Recall (%)0.00

The clarification documents the finalized SV construction used in all reported experiments and does not introduce any post-submission changes to the benchmark or evaluation protocol. The evaluation targets and prediction records remain unchanged. Consequently, the reported TDR Overall and 19-task macro-average results are unaffected. The additional class-distribution statistics and always-sound baseline provide further context for interpreting the existing evaluation results.

AuthorsOpenReview ↗

Response to Reviewer bTTF

W2: Insufficient Evidence Separating Task Difficulty from Anatomical Localization and Cross-view Integration. The experiments do not sufficiently distinguish the observed task difficulty gap from the specific explanation in terms of anatomical localization and cross-view integration. Finding 2 reports a 29.1-point difference between mean ACS and TDR scores. As the authors acknowledge, these groups differ in task format as well as anatomical granularity; Table 8 and Appendix K also show different prediction targets and scoring units. This supports a descriptive performance gap on the constructed tasks, but does not isolate its source. The paper does not report a comparison that holds the clinical target and answer format fixed while varying the available views, or an input ablation establishing dependence on cross-view evidence. The five-step response protocol and final-label accuracy do not independently establish such dependence. A limited paired analysis on existing test cases, or a more restricted interpretation of the findings, would materially strengthen the benchmark's central scientific claim.

Q2: Controlled Single-view versus Multi-view Comparison. What evidence separates a limitation in cross-view integration from differences in task definition and answer difficulty? On a limited subset of existing held-out targets, can the authors compare a relevant single-view with all views while holding the clinical target, label space, and answer format fixed? Alternatively, please clarify which conclusions are intended only as descriptive comparisons across the current tasks. Such evidence or clarification would affect my assessment of the claimed reasoning insight.

A2: We thank you for highlighting this important issue. We agree that the ACS--TDR performance gap is a descriptive benchmark-level disparity and cannot be attributed solely to anatomical granularity, cross-view integration, or any other individual factor. Across the 24 baseline MLLMs, mean ACS Overall accuracy was 67.3%, compared with 38.2% for TDR Overall, corresponding to a descriptive benchmark-level performance difference of 29.1 percentage points. Because ACS and TDR differ in anatomical scope, prediction targets, diagnostic complexity, answer formats, and scoring units, this difference cannot identify the specific source of the performance gap. We have therefore revised Finding 2 to describe this gap as a benchmark-level performance disparity, without attributing it to any single underlying factor or presenting it as a causal effect of anatomical granularity.

To address your request for a controlled single-view versus multi-view comparison, we conducted additional within-task ablations on TDR-FS and ACS-SC using their respective existing patient-disjoint held-out test sets. Within each task, all input conditions were evaluated using the same clinical targets, label spaces, answer formats, prompting protocols, image-resolution settings, model configurations, and deterministic decoding settings. Only the visual input condition was varied. For the repeated-view control, Panel 1 was repeated for TDR-FS and View 5 for ACS-SC, with the number of visual input slots matched to that of the full multi-view condition. These comparisons allow us to examine whether the observed performance differences are associated with access to distinct visual observations under a matched input budget, rather than simply an increased number of visual input slots.

The controlled comparisons are summarized in Table R4 and R5.

Table R4. Comprehensive multi-view and view-wise ablation on TDR-FS

Input conditionAccuracy (%)Macro-F1 (%)Balanced Acc. (%)Δ\Delta (pp)95% CI (pp)
Full multi-view29.116.728.1Ref.--
Panel 123.511.620.85.6[3.9, 7.7]
Panel 223.010.018.66.1[4.4, 8.2]
Panel 323.29.918.75.9[4.2, 7.8]
Panel 422.38.217.36.8[5.0, 8.8]
Panel 522.18.417.47.0[5.3, 9.0]
Repeated Panel 1 (equal budget)23.210.619.45.9[4.1, 7.8]
Mean single-view22.89.618.66.3[4.7, 8.0]

On TDR-FS, full multi-view input achieved 29.1% Accuracy, 16.7% Macro-F1, and 28.1% Balanced Accuracy. Its Accuracy exceeded the highest-accuracy fixed single-view condition (Panel 1, 23.5%) by 5.6 percentage points, the mean single-view baseline (22.8%) by 6.3 percentage points, and the equal-budget repeated-Panel-1 control (23.2%) by 5.9 percentage points. The paired 95% CIs excluded zero for all five individual panels, the repeated-view control, and the mean single-view baseline.

Reviewer_x3yZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CariesBench, a benchmark that renders 3D intraoral scans into multiple views and evaluates caries-related visual question answering at the dental-arch, tooth, and surface levels. It contains 5,932 scans and 133,994 VQA instances, uses patient-level splits, and evaluates 24 multimodal models. The authors also fine-tune Qwen3-VL-8B-Instruct to obtain IntraoralGPT. The main contribution is the dataset and evaluation framework; the model itself offers limited methodological novelty. The results suggest that fine-grained tooth and surface assessment remains challenging.

Critical strengths
  1. The dataset is relatively large, and its arch-, tooth-, and surface-level tasks reflect the progression of clinical assessment. This provides a useful setting for evaluating dental visual understanding beyond image-level recognition.
  2. Patient-level data splits reduce the risk of leakage, and the evaluation of 24 models provides a useful overview of current performance and remaining difficulties.
Critical weaknesses

1.The experiments lack a direct single-view versus multi-view comparison on the same cases and questions. Without this control, it is difficult to determine whether performance depends on combining evidence across views. Evaluation under different acquisition or rendering conditions would also help establish generalization. 2. The description does not clearly establish whether the SV task includes both sound and non-sound teeth. If it contains only sound teeth, always predicting “sound” could achieve a high score without using the images. Class counts and a constant-answer baseline are needed to assess this concern.

Questions for authors
  1. On the same test cases and questions, how does performance change when a model receives a single view rather than the full multi-view input? Please keep the prompting and relevant image-resolution settings comparable so that the difference can be interpreted.
  2. How many sound and non-sound teeth are included in the SV test set? What accuracy would be obtained by always answering “sound”? Please clarify the task construction and how it supports evaluation of visual verification.
  3. Have results from a different institution, scanner, or rendering configuration? If not, please state the resulting limitation on claims of generalization. External evaluation would be useful additional evidence.
4 comments
AuthorsOpenReview ↗

Response to Reviewer x3yZ

We sincerely thank you for the thoughtful and constructive evaluation of CariesBench. We appreciate the recognition of its hierarchical benchmark design, patient-level data splitting, and comprehensive evaluation of multimodal large language models. We have carefully considered the concerns regarding controlled multi-view evaluation, the construction and class distribution of Sound Tooth Verification (SV), and external generalization. In the current revision, we have added controlled multi-view comparisons, clarified the finalized SV task definition and its clinical class distribution, and reported a constant-answer baseline. The external-generalization evaluation is being prepared separately and is not included among the completed experiments reported here.


W1(a): Insufficient Controlled Multi-view Evaluation. The experiments lack a direct single-view versus multi-view comparison on the same cases and questions. Without this control, it is difficult to determine whether performance depends on combining evidence across views.

Q1: Same-case Single-view versus Multi-view Comparison. On the same test cases and questions, how does performance change when a model receives a single-view rather than the full multi-view input? Please keep the prompting and relevant image-resolution settings comparable so that the difference can be interpreted.

A1: We thank you for emphasizing the importance of controlled comparisons. We agree that evaluating different visual input conditions on identical test instances is necessary to characterize the contribution of complementary anatomical observations. To address this concern, we conducted additional controlled multi-view ablation experiments on two representative tasks, TDR-FS and ACS-SC, as reported in Appendix M.

For each task, we compare full multi-view input with each fixed single-view condition, the mean single-view baseline, and an equal-budget repeated-view control. In the repeated-view condition, the same anatomical view is repeated across the visual input slots to match the input budget of the full multi-view condition. All conditions within each task use identical evaluation instances, clinical targets, label spaces, answer formats, prompting protocols, image-resolution settings, model configurations, and deterministic decoding. Only the available visual observations vary across input conditions, allowing us to examine the contribution of complementary anatomical information under fixed evaluation protocols.

We evaluated all individual fixed views (Panels 1--5 for TDR-FS and Views 1--6 for ACS-SC) and identified the highest-accuracy fixed single-view condition within each task. The quantitative results are summarized in Tables R1 and R2.

Table R1. Comprehensive multi-view and view-wise ablation on TDR-FS

Input conditionAccuracy (%)Macro-F1 (%)Balanced Acc. (%)Δ\Delta (pp)95% CI (pp)
Full multi-view29.116.728.1Ref.--
Panel 123.511.620.85.6[3.9, 7.7]
Panel 223.010.018.66.1[4.4, 8.2]
Panel 323.29.918.75.9[4.2, 7.8]
Panel 422.38.217.36.8[5.0, 8.8]
Panel 522.18.417.47.0[5.3, 9.0]
Repeated Panel 1 (equal budget)23.210.619.45.9[4.1, 7.8]
Mean single-view22.89.618.66.3[4.7, 8.0]

On TDR-FS, full multi-view input achieves 29.1% accuracy, outperforming the best fixed single-view condition by 5.6 percentage points, mean single-view performance by 6.3 percentage points, and equal-budget repeated-view input by 5.9 percentage points. The paired 95% CIs excluded zero for all five individual panels, the repeated-view control, and the mean single-view baseline, indicating statistically supported improvements over these comparators.

AuthorsOpenReview ↗

Response to Reviewer x3yZ

Table R2. Comprehensive multi-view and view-wise ablation on ACS-SC

Input conditionAccuracy (%)Macro-F1 (%)Balanced Acc. (%)Δ\Delta (pp)95% CI (pp)
Full multi-view53.216.818.5Ref.--
View 149.212.914.54.0[0.8, 7.4]
View 249.513.315.13.7[0.7, 6.8]
View 350.314.716.42.9[-0.3, 6.0]
View 452.414.616.30.8[-1.5, 3.0]
View 552.614.115.90.6[-2.3, 3.4]
View 651.113.615.42.1[0.1, 4.4]
Repeated View 5 (equal budget)51.114.516.12.1[1.0, 6.1]
Mean single-view50.913.915.62.3[0.1, 4.5]

On ACS-SC, full multi-view input achieves 53.2% Accuracy, 16.8% Macro-F1, and 18.5% Balanced Accuracy. Its Accuracy exceeds mean single-view performance by 2.3 percentage points. The corresponding paired 95% CI [0.1, 4.5] excludes zero. However, the CIs for Views 3, 4, and 5 include zero, whereas those for Views 1, 2, and 6 exclude zero. The repeated-View-5 control also has a CI that excludes zero ([1.0, 6.1]). We therefore do not claim statistically supported Accuracy improvements over Views 3, 4, or 5.

Taken together, these results provide task-dependent evidence regarding the contribution of complementary anatomical observations. On TDR-FS, full multi-view input shows positive Accuracy differences with paired 95% CIs excluding zero against all five fixed views, the repeated-view control, and the mean single-view baseline. On ACS-SC, the corresponding CIs exclude zero for Views 1, 2, and 6, the repeated-view control, and the mean single-view baseline, but include zero for Views 3, 4, and 5. These findings suggest that the contribution of multi-view input depends on the diagnostic requirements and the complementary anatomical information available across views, rather than simply on increasing the number of visual input slots. We have incorporated the experimental protocol, statistical analysis, and detailed view-wise results into Appendix M.

AuthorsOpenReview ↗

Response to Reviewer x3yZ

W2: Ambiguity in SV Task Definition and Class Distribution. The description does not clearly establish whether the SV task includes both sound and non-sound teeth. If it contains only sound teeth, always predicting "sound" could achieve a high score without using the images. Class counts and a constant-answer baseline are needed to assess this concern.

Q2: SV Class Distribution and Constant-answer Baseline. How many sound and non-sound teeth are included in the SV test set? What accuracy would be obtained by always answering "sound"? Please clarify the task construction and how it supports evaluation of visual verification.

A2: We thank you for identifying the inconsistency in the original description of SV. An intermediate version of our SV construction script contained a sound-only eligibility condition. We removed this condition before submission, and the finalized SV construction allows both sound and non-sound teeth to be included. However, Appendix F.2 inadvertently retained an outdated description from the intermediate construction protocol, stating that non-sound teeth were used for SA/FS whereas only sound teeth were used for SV. This statement did not describe the finalized benchmark. All reported experiments were conducted using the finalized benchmark containing both sound and non-sound SV instances. Therefore, the inconsistency was confined to the outdated manuscript description and did not affect the evaluation data, held-out test labels, or reported results. We apologize for this oversight and have removed the superseded SV-specific eligibility statement from Appendix F.2.

In the finalized benchmark, SA, FS, and SV are based on the same L0--L7 clinical-status taxonomy, derived from structured surface-level annotations. Their differences lie in anatomical scope and prediction format, rather than in the underlying clinical annotation system. For SV, the tooth-level binary target is derived from the relevant surface annotations: a tooth is classified as Sound (L0) if all its relevant annotated surfaces are L0, and as Non-sound (L1--L7) if at least one relevant annotated surface has a clinical status from L1--L7. Thus, L0--L7 represent the underlying clinical-status categories, whereas the final SV prediction space is binary (Sound/Non-sound).

The complete L0--L7 distribution in the full TDR-SV Benchmark set is reported in Table R3.

Table R3. L0--L7 Clinical-status Distribution in the Full TDR-SV Benchmark Set

Clinical statusNumber of instancesPercentage (%)
L02,42523.97
L11,44414.27
L26776.69
L31,85718.36
L41,11110.98
L57777.68
L61,09010.78
L77357.27
Total10,116100.00

For the held-out SV test set, 737 teeth are classified as Sound and 338 as Non-sound, giving 1,075 test instances. An always-sound predictor therefore achieves 68.56% accuracy. This majority-class baseline provides a reference for assessing whether a model can identify non-sound conditions rather than simply predicting the majority class. The corresponding baseline metrics are summarized in Table R4.

Table R4. SV Test-set Class Distribution and Always-Sound Baseline

MetricAlways-sound baseline
Sound instances737
Non-sound instances338
Total test instances1,075
Accuracy (%)68.56
Macro-F1 (%)40.67
Balanced Accuracy (%)50.00
Non-sound Recall (%)0.00

The always-sound baseline demonstrates why overall accuracy alone is insufficient for evaluating SV. Although this predictor achieves 68.56% accuracy due to the majority class, it completely fails to identify non-sound teeth, resulting in zero non-sound recall and a Balanced Accuracy of 50.00%. Reporting this baseline alongside the class distribution provides a transparent reference for interpreting SV performance and assessing whether predictions capture both clinical conditions.

The finalized SV construction, the distinction between the full pre-sampling L0--L7 distribution and the binary test-set distribution, and the always-sound baseline are clarified in this response. These revisions clarify the task construction established before submission rather than introducing a new task or modifying the held-out test labels.


We sincerely thank you again for these constructive comments. We believe that the additional controlled multi-view comparisons, clarification of the SV construction and class distribution, and the always-sound baseline strengthen the methodological clarity and empirical transparency of CariesBench. We hope that these revisions address your concerns and provide a clearer account of the benchmark's empirical findings.

Reviewer_x3yZOpenReview ↗

Comment

Thank you for the detailed response and the additional experiments. The authors have addressed my main concerns, particularly by providing controlled single-view versus multi-view comparisons and clarifying the construction and class distribution of the SV task.

The new ablation results provide useful evidence for the contribution of multi-view information, although the improvements are relatively modest for ACS-SC. The clarification of the SV labels and the always-sound baseline also resolves my concern about the task definition and helps interpret the reported accuracy. I appreciate the authors' acknowledgment that external generalization remains to be evaluated.

Overall, I am satisfied with the rebuttal. I believe the additional analyses and clarifications strengthen the paper, and I will maintain my weak-accept recommendation.

Reviewer_ZtC7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents CariesBench, a hierarchical multi-view benchmark for dental caries screening and diagnostic reasoning from 3D intraoral scans. The benchmark introduces three complementary task categories spanning arch-level screening, tooth-level diagnostic reasoning, and structured clinical decision-making, aiming to evaluate anatomical understanding and cross-view evidence integration in MLLMs. The authors construct clinically grounded VQA instances using standardized multi-view renderings and structured clinical annotations, supported by expert quality control. Experiments across 24 MLLMs reveal a substantial global-to-local performance gap, with models struggling particularly on fine-grained tooth- and surface-level tasks. The paper further introduces IntraoralGPT through benchmark-specific fine-tuning, demonstrating improved diagnostic accuracy while highlighting the persistent challenges of fine-grained anatomical reasoning and multi-view evidence integration.

Critical strengths
  1. CariesBench introduces a hierarchical evaluation framework spanning arch-level screening, tooth-level diagnostic reasoning, and structured clinical decision-making. This design enables a more comprehensive assessment of fine-grained dental understanding beyond conventional image-level recognition.
  2. CariesBench provides a large-scale dataset with structured clinical annotations and expert quality control, enhancing its reliability and value for future research.
Critical weaknesses
  1. It is unclear whether the current experiments sufficiently support the paper's claim of evaluating cross-view reasoning capabilities. Additional experiments, such as single-view versus multi-view comparisons and view ablation studies, could help validate whether models effectively integrate complementary information across different views.
  2. Although the authors acknowledge that ACS and TDR differ in task format and anatomical granularity, the current evidence does not sufficiently disentangle the effects of anatomical granularity from other factors, such as task difficulty and evaluation criteria. A more detailed comparative analysis or additional controlled experiments would help better support the claimed global-to-local reasoning gap.
Questions for authors
  1. Can the authors provide further evidence, such as single-view comparisons or view ablations, to validate the benchmark's ability to assess cross-view reasoning?
  2. How can the authors better distinguish the effects of anatomical granularity from task difficulty and evaluation criteria when analyzing the global-to-local reasoning gap?
  3. Could the authors report class-balanced metrics, such as Macro-F1, for applicable classification tasks to better account for the imbalanced clinical severity distribution?
  4. Some labels and numerical annotations in Figure 2 are relatively small, making the statistical distributions difficult to read. Improving the figure's readability would enhance the overall presentation.
4 comments
AuthorsOpenReview ↗

Response to Reviewer ZtC7

We sincerely thank you for the positive assessment of CariesBench and for the constructive comments. We particularly appreciate your recognition of the hierarchical benchmark design and the clinical quality of the annotations. Following the suggestions, we have conducted additional controlled multi-view experiments, incorporated class-balanced evaluation metrics, clarified the interpretation of the global-to-local performance gap, and improved the readability of Figure 2. Our point-by-point responses are provided below.


W1: Insufficient Evidence for Cross-view Reasoning. It is unclear whether the current experiments sufficiently support the paper's claim of evaluating cross-view reasoning capabilities. Additional experiments, such as single-view versus multi-view comparisons and view ablation studies, could help validate whether models effectively integrate complementary information across different views.

Q1: Additional Experimental Evidence for Cross-view Evaluation. Can the authors provide further evidence, such as single-view comparisons or view ablations, to validate the benchmark's ability to assess cross-view reasoning?

A1: We sincerely thank you for this important suggestion. We agree that controlled comparisons between single-view and multi-view inputs are necessary to better characterize the contribution of complementary visual observations.

To address this concern, we have added controlled multi-view ablation experiments on two representative tasks, TDR-FS and ACS-SC, as detailed in Appendix M. For each task, we compare full multi-view input with each fixed single-view condition, the mean single-view baseline, and an equal-budget repeated-view control, in which the same anatomical view is repeated across the visual input slots to match the input budget of the full multi-view condition. All comparisons use identical evaluation instances, clinical targets, label spaces, answer formats, prompting protocols, image-resolution settings, model configurations, and deterministic decoding within each task.

We evaluated all fixed single-view conditions (Panels 1--5 for TDR-FS and Views 1--6 for ACS-SC) and identified the highest-Accuracy fixed single-view based on their evaluation results.

The controlled comparisons are summarized in Table R1 and R2.

Table R1. Comprehensive multi-view and view-wise ablation on TDR-FS

Input conditionAccuracy (%)Macro-F1 (%)Balanced Acc. (%)Δ\Delta (pp)95% CI (pp)
Full multi-view29.116.728.1Ref.--
Panel 123.511.620.85.6[3.9, 7.7]
Panel 223.010.018.66.1[4.4, 8.2]
Panel 323.29.918.75.9[4.2, 7.8]
Panel 422.38.217.36.8[5.0, 8.8]
Panel 522.18.417.47.0[5.3, 9.0]
Repeated Panel 1 (equal budget)23.210.619.45.9[4.1, 7.8]
Mean single-view22.89.618.66.3[4.7, 8.0]

On TDR-FS, full multi-view input achieves 29.1% accuracy, outperforming the highest-scoring single-view condition by 5.6 percentage points, mean single-view performance by 6.3 percentage points, and equal-budget repeated-view input by 5.9 percentage points. The paired 95% CIs excluded zero for all five individual panels, the repeated-view control, and the mean single-view baseline, supporting the benefit of complementary visual observations on this task.

Table R2. Comprehensive multi-view and view-wise ablation on ACS-SC

Input conditionAccuracy (%)Macro-F1 (%)Balanced Acc. (%)Δ\Delta (pp)95% CI (pp)
Full multi-view53.216.818.5Ref.--
View 149.212.914.54.0[0.8, 7.4]
View 249.513.315.13.7[0.7, 6.8]
View 350.314.716.42.9[-0.3, 6.0]
View 452.414.616.30.8[-1.5, 3.0]
View 552.614.115.90.6[-2.3, 3.4]
View 651.113.615.42.1[0.1, 4.4]
Repeated View 5 (equal budget)51.114.516.12.1[1.0, 6.1]
Mean single-view50.913.915.62.3[0.1, 4.5]

On ACS-SC, full multi-view input achieves 53.2% Accuracy, 16.8% Macro-F1, and 18.5% Balanced Accuracy. Its Accuracy exceeds mean single-view performance by 2.3 percentage points. The corresponding paired 95% CI [0.1, 4.5] excludes zero. However, the CIs for Views 3, 4, and 5 include zero, whereas those for Views 1, 2, and 6 exclude zero. The repeated-View-5 control also has a CI that excludes zero ([1.0, 6.1]). We therefore do not claim statistically supported Accuracy improvements over Views 3, 4, or 5.

AuthorsOpenReview ↗

Response to Reviewer ZtC7

For statistical reliability, we compute 95% confidence intervals using 10,000 paired patient-level bootstrap replicates. All eligible predictions belonging to each sampled patient are retained together, and identical patient-level bootstrap samples are used across input conditions to preserve pairing.

Taken together, these results provide task-dependent evidence for the contribution of complementary anatomical observations. On TDR-FS, full multi-view input shows positive Accuracy differences with paired 95% CIs excluding zero against all five fixed views, the repeated-view control, and the mean single-view baseline. On ACS-SC, the corresponding CIs exclude zero for Views 1, 2, and 6, the repeated-view control, and the mean single-view baseline, but include zero for Views 3, 4, and 5. We therefore avoid claiming a uniform multi-view advantage across all individual views in both tasks. These results suggest that the contribution of multi-view input depends on the diagnostic requirements and the complementary anatomical information available across views, rather than simply on increasing the number of visual input slots. We have incorporated the experimental protocol, statistical analysis, and detailed view-wise results into Appendix M.


W2: Confounding Factors in the Global-to-Local Performance Gap. Although the authors acknowledge that ACS and TDR differ in task format and anatomical granularity, the current evidence does not sufficiently disentangle the effects of anatomical granularity from other factors, such as task difficulty and evaluation criteria. A more detailed comparative analysis or additional controlled experiments would help better support the claimed global-to-local reasoning gap.

Q2: Distinguishing Anatomical Granularity from Task Difficulty. How can the authors better distinguish the effects of anatomical granularity from task difficulty and evaluation criteria when analyzing the global-to-local reasoning gap?

A2: We thank you for highlighting this important issue. We agree that the performance disparity between ACS and TDR cannot be attributed solely to anatomical granularity, since these task families also differ in task formulation, diagnostic complexity, prediction targets, answer formats, and evaluation criteria.

Across the 24 baseline MLLMs, the mean ACS Overall accuracy is 67.3%, compared with 38.2% for TDR Overall, yielding a descriptive benchmark-level performance difference of 29.1 percentage points.

We have revised Finding 2 to present the ACS--TDR performance difference as a descriptive benchmark-level disparity, without attributing it to any single underlying factor or interpreting it as a causal effect of anatomical granularity. The controlled within-task multi-view ablations provide separate empirical evidence regarding the contribution of complementary visual observations, but do not independently explain the cross-task performance gap.


Q3: Class-balanced Evaluation Metrics. Could the authors report class-balanced metrics, such as Macro-F1, for applicable classification tasks to better account for the imbalanced clinical severity distribution?

A3: Thank you for this valuable suggestion. We agree that overall accuracy alone may not adequately characterize model performance under imbalanced clinical class distributions. Macro-F1 and Balanced Accuracy complement Clinical Accuracy by providing class-balanced assessments of predictive performance.

Following your suggestion, we have added Macro-F1 and Balanced Accuracy to the view-wise evaluation of TDR-FS and ACS-SC, with all fixed-view and controlled conditions reported together in Table R1 and R2.

As shown in Table R1 and R2, full multi-view input outperforms mean single-view input on all three metrics in both tasks. Macro-F1 increases from 9.6% to 16.7% on TDR-FS and from 13.9% to 16.8% on ACS-SC; Balanced Accuracy also increases from 18.6% to 28.1% and from 15.6% to 18.5%, respectively. These class-balanced metrics complement Clinical Accuracy by providing a comprehensive assessment of predictive performance across the predefined clinical classes.

We have also added the mathematical definitions of Macro-F1 and Balanced Accuracy in Appendix K.3, including their class-wise averaging procedures and corresponding evaluation conventions. The complete view-wise results are provided in Appendix M.

AuthorsOpenReview ↗

Response to Reviewer ZtC7

Q4: Figure Readability. Some labels and numerical annotations in Figure 2 are relatively small, making the statistical distributions difficult to read. Improving the figure's readability would enhance the overall presentation.

A4: We thank you for this helpful suggestion. We have revised Figure 2 to improve its overall readability. Specifically, we enlarged the category labels, legends, numerical annotations, percentages, and total counts across the figure. We also enlarged panels (b), (c), and (e), staggered the dentist-score annotations in panel (c) to prevent overlap, and increased the vertical spacing between the two rows of panels. These revisions improve the readability of the statistical distributions and numerical values without altering the underlying data.


We sincerely thank you again for the constructive feedback. We believe that the additional controlled multi-view experiments, comprehensive class-balanced evaluation metrics, more carefully qualified interpretation of the global-to-local performance disparity, and improved figure presentation have strengthened the empirical evidence and clarity of CariesBench.

I want to maintain my current score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.