- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces 3D-VL-Fly, a 3D vision-language benchmark for post-disaster damage assessment. Built from aerial reconstructions following Hurricane Ian, it contains 64 scenes and 308 buildings with instance masks, attributes, and language annotations. The benchmark supports VQA, language-guided segmentation, attribute recognition, and instance segmentation, with evaluations of pretrained models and a fine-tuning experiment.
Critical strengths
- Extending 3D vision-language evaluation to real disaster scenes provides a useful resource for studying building damage and outdoor spatial understanding.
- Majority baselines and separate damage-attribute results help distinguish general building recognition from damage-assessment capability.
Critical weaknesses
- Annotators access high-resolution photographs that point-cloud models do not receive. The paper acknowledges that some details may be less observable, but does not assess which labels can be reliably inferred from point clouds alone. Low scores may therefore reflect missing input evidence as well as limited model capability.
- The paper introduces both referring and reasoning segmentation, but the task description mainly lists directional and attribute-based references, and results are split only by direction. Please clarify how reasoning queries are distinguished from referring expressions and report their performance separately.
Questions for authors
- The dataset contains 64 scenes, but experiments use only 49. Why were the remaining 15 not included, and do they differ from the evaluated subset in damage severity or reconstruction quality?
- How are building-to-building and building-to-road distances defined: between centers, nearest boundaries, or otherwise? Is reconstruction scale error considered when generating distance-based answers?