acceptodds
Withdrawn from ICLR 2027

SpineDiag: A Multimodal Spine Scoliosis Benchmark for Radiation-Free Diagnosis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Adolescent idiopathic scoliosis (AIS) is a common spinal deformity. However, large-scale diagnosis and repeated follow-up are limited by the radiation exposure of full-spine X-ray. To this end, we build SpineDiag, a paired multimodal clinical AIS dataset with 155 subjects and radiation-free modalities (visual observations, depth maps, and point clouds, annotated keypoints, and physician-verified diagnosis descriptions). To build a unified spine representation, we first design the 2D spinal keypoint protocol from medical expertise, then lift this 2D anatomical representation with depth to reconstruct a 3D kinematic spine model in simulation. We further propose a benchmark suite with four tasks: (1) visual spine perception, (2) visual diagnosis with video only, (3) visual diagnosis with video and 2D keypoints, and (4) visual diagnosis with videos and depth map. The results are assessed with keypoint and geometric metrics, a clinically Visual Question Answering (VQA) score, and reported LLM verification diagnosis score. We train and evaluate pose estimators and vision-language models (VLMs) on these tasks. The experimental results expose open problems of current models in radiation-free diagnosis, which needs further investigation and more robust models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Medical Image, Human Pose

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_U92YOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

SpineDiag introduces a multimodal scoliosis dataset of 155 subjects, pairing body-surface images, depth, anatomical keypoints, and clinical descriptions. It benchmarks pose estimation and VLM-based assessment for radiation-free spine evaluation.

Critical strengths

The dataset addresses a clinically meaningful problem and provides paired modalities rarely available in existing spine benchmarks. Surgeon-reviewed annotations and subject-level splits are valuable design choices. The experiments also highlight that good keypoint accuracy does not necessarily imply accurate curvature estimation.

Critical weaknesses
  1. Unvalidated reference consistency. Cobb references mix radiographic measurements with estimates derived from visual annotations (Section 2.2). Without validating their agreement and reporting results separately, the clinical meaning of pooled Cobb errors remains unclear.
  2. Confounded modality comparisons. Table 6 evaluates different input settings on different image subsets. Although acknowledged, this prevents attributing performance differences to structured keypoints or depth, weakening a central empirical conclusion.
  3. Insufficient validation of diagnosis scores. The clinical justification for the VQA weighting and the agreement between LLM-based scoring and independent clinician assessments are not established (Sections 3.2 and 4.3). Surgeon verification of reference descriptions does not validate the scoring of model outputs.
Questions for authors
  1. What proportion of Cobb references comes from each source, and do the main findings hold when evaluated exclusively against radiographic measurements?
  2. Do the conclusions about structured keypoints and depth remain valid when all input settings use identical training and test cases?
  3. How well do the weighted VQA and LLM-verification scores agree with independent clinicians’ assessments of diagnostic correctness?
Reviewer_2aAROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SpineDiag introduces a multimodal dataset of 155 subjects, pairing back images and depth data with anatomical keypoints and physician-verified diagnostic descriptions. It benchmarks pose estimators and vision-language models on keypoint prediction and diagnosis using subject-level splits. The reported results show improved VQA performance after fine-tuning, higher scores in keypoint-based settings than in point-cloud settings, and a gap between keypoint localization accuracy and Cobb-angle accuracy.

Critical strengths
  1. Useful paired annotations. Combining images, depth, landmarks, and diagnostic attributes could help researchers study how perception errors affect downstream predictions. The annotations were also reviewed by specialists.
  2. Sensible evaluation choices. The paper uses subject-level splits and evaluates intermediate keypoints separately from final answers. These choices are useful for avoiding frame-level leakage and examining where models fail.
Critical weaknesses
  1. Small dataset and uncertain rankings. The dataset contains 155 subjects, leaving roughly 31 test subjects per split. The paper provides insufficient evidence that the small performance differences in Table 6 support reliable model rankings.
  2. Uncontrolled comparison across inputs. Different input settings use different image subsets. The claim that structured keypoints are more effective than point clouds lacks a controlled comparison on the same cases. The reported differences could reflect differences in case difficulty.
Questions for authors
  1. Do keypoints still outperform point clouds on the same cases? Please compare both settings on a common subset with identical training, validation, and test cases, using the same model and training budget. This would directly test whether the reported advantage persists after controlling for case composition.
  2. How stable are the model rankings across the three subject-level splits? Can you provide confidence intervals for the performance differences in Table 6?
Reviewer_FsEgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

SpineDiag is a clinical dataset and benchmark for radiation-free scoliosis assessment. It contains 155 subjects, 171 visits, and 961 annotated frames, with RGB images, depth data for a subset, point clouds, anatomical keypoints, and clinician-reviewed diagnosis descriptions. The authors evaluate pose estimators and vision-language models on keypoint detection and diagnosis tasks using subject-level splits.

Critical strengths
  1. Collecting paired clinical images, depth information, and anatomical annotations is valuable. The annotation and clinician-review process makes this a useful resource for studying scoliosis beyond standard human pose datasets.

  2. Evaluating both keypoint detection and diagnosis helps reveal whether better visual predictions translate into better assessment. Keeping each subject within one data split is also a sound choice.

Critical weaknesses
  1. The dataset is small, and its added value over recent resources is not sufficiently established. Scoliosis1K, Scoliosis1K-Pose, and SpineBench are missing from the discussion. These works address different settings, but comparing their coverage would help explain what SpineDiag uniquely enables.

  2. The clinical scope of the diagnosis benchmark is limited. Its nine questions focus on curve presence, location, direction, and visible asymmetry, without directly testing severity grading or progression. The separate geometric Cobb evaluation partly addresses this gap, but its references mix X-ray measurements with estimates from visual annotations. This makes the clinical meaning of the reported performance unclear.

Questions for authors
  1. What does SpineDiag enable that Scoliosis1K, Scoliosis1K-Pose, and SpineBench do not? Please clarify the differences in annotations and supported tasks.

  2. How many Cobb references come from X-rays and how many from visual annotations? Can the existing angle results be reported separately for these sources?

  3. Since every visit has a Cobb reference, why is severity grading excluded from the diagnosis questions? Is this a task-design choice or a limitation of label reliability?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.