- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Med3DNavi, a training-free framework that enables VLMs to actively explore 3D CT volumes. The framework develops a programmable viewer and a Structured Volumetric Reading Protocol (SVRP). It introduces audit and autonomous exploration routes to guide evidence collection and reasoning. Experiments on DeepTumorVQA and 3D-RAD demonstrate improvements over existing VLMs and agentic approaches.
Critical strengths
-
The paper addresses an important limitation of existing agentic frameworks such as 3DMedAgent, where inaccurate statistics derived from upstream visual tools may affect subsequent reasoning. The proposed audit mechanism enables dynamic visual inspection and potential correction of these measurements.
-
The programmable viewer integrates multi-planar viewing, HU windowing, ROI zooming, and mask overlays, providing flexible visual access to 3D CT volumes without requiring additional model training.
-
SVRP organizes visual exploration into orientation, investigation, and adjudication, with dedicated audit and autonomous routes. The independent verification paths offer a reasonable approach to collecting and cross-checking visual evidence.
-
The paper evaluates two benchmarks and multiple VLM backbones, with additional analyses of routing behavior, viewer capabilities, and verification paths. The results provide encouraging evidence for the proposed framework.
Critical weaknesses
-
Although the audit route is a valuable contribution, the experiments mainly demonstrate improvements in final task accuracy. A more direct evaluation of error detection, measurement correction, and the risk of incorrectly modifying accurate tool outputs would better establish the reliability of this mechanism.
-
The multi-path reasoning design is interesting, but the experiments do not clearly distinguish the benefits of context isolation from those of additional visual observations and reasoning steps. A comparison with shared-context multi-path verification would help clarify its contribution.
-
The framework requires multiple VLM and viewer interactions, but its computational overhead is not fully characterized.
Questions for authors
See weakness.
2 comments
Thanks authors for their effort in rebuttal. Most of my concerns have been addressed and I would keep my score as 3.
Reply to Reviewer Zw9J
We sincerely thank the Reviewer for the positive assessment, high confidence in our work, and insightful suggestions. We are delighted that you found our audit mechanism, programmable viewer, and SVRP protocol well-motivated and effective. Below, we address each of your three concerns with direct empirical evaluations.
1. Direct Evaluation of Audit Reliability (W1)
We report the audit transition matrix:
On the Measurement subtasks, visual auditing reduces the Mean Absolute Percentage Error (MAPE) from 7.77% to 3.08%. We acknowledge that VLM-based visual auditing provides directional and heuristic calibration rather than exact voxel-level re-measurement. Consequently, residual errors or false overrides typically occur when candidate options are closely spaced and the agent slightly over-estimates the magnitude of a minor boundary discrepancy, shifting the adjusted estimate past the true option toward an adjacent distractor.
Nevertheless, this behavior mirrors real-world radiological reading: a radiologist inspecting a 3D CT scan cannot mentally integrate exact voxel counts to tell whether a kidney is 178.3 or 188.3 cm3 by naked eye alone, and likewise relies on segmentation tools for precise quantification. However, when an automated segmentation mask is imperfect, the clinician excels at identifying whether the mask suffers from under-segmentation or boundary leakage, and determining the direction and rough magnitude of the bias to avoid trusting a flawed report blindly (such as the 158.0 cm3 distractor in Option C).
Med3DNavi inherits this exact division of labor. Furthermore, this insight points to an exciting future extension: closing the loop not only at the visual inspection level, but also at the tool execution level—allowing the VLM to issue spatial prompts (e.g., positive clicks on unsegmented voids or negative clicks on leaked boundaries) to interactive 3D segmentation models to trigger mask refinement and re-computation.
2. Isolating Context Isolation from Observation/Reasoning Budget in Multi-Path Verification (W2)
Following your exact suggestion, we conducted a controlled ablation comparing Isolated-Context Multi-Path against Shared-Context Multi-Path under the exact same planner strategies:
(†
# Cumul. Imagescounts cumulative image inputs across multi-turn VLM invocations;*denotes parallel execution of the K isolated verification paths.)Enforcing context isolation provides three advantages:
3. Computational Overhead Characterization (W3)
As noted in our Limitations (Appendix D), multi-stage agentic frameworks inevitably incur higher token consumption and latency than single-pass 2D VLM inference. However, when compared against peer agentic methods, Med3DNavi achieves the best accuracy under a comparable computational budget.
(†
# Cumul. Imagescounts cumulative image inputs across multi-turn VLM invocations)We hope these additional experiments fully address your concerns.