- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SurgRoute, a selective multi-agent verification framework for surgical video understanding. The framework consists of three components: Joint Probe Gate (JPG), which learns when additional verification is beneficial using procedural priors and joint interaction-phase evidence; Multi-Agent Review and Repair (MAR), which aggregates feedback from multiple LLMs through constrained correction rules and early stopping; and Tracker-Assisted Correction (TC), which incorporates instrument tracking, label constraints, and temporal phase aggregation. Experiments on SurAgent-Bench using six VLM backbones demonstrate improvements in surgical recognition and report generation, alongside reduced token consumption compared with exhaustive multi-agent review.
Critical strengths
-
The paper addresses a practically relevant problem in surgical video understanding, where prediction errors and inconsistencies across related tasks can affect downstream report generation.
-
The framework demonstrates consistent improvements across six VLM backbones and includes diagnostic experiments examining review allocation, correction rules, and computational efficiency.
Critical weaknesses
-
The methodological novelty is relatively limited. The proposed framework primarily combines established techniques, including gradient-boosted routing, multi-LLM rating aggregation, rule-based correction, and temporal voting. The paper does not sufficiently demonstrate a substantial methodological contribution beyond integrating these existing components.
-
The experimental evidence does not convincingly support the central motivation for selective multi-agent verification. Always-review achieves only a marginal improvement over skip-review in triplet F1 (46.43 vs. 46.21), despite nearly doubling token consumption. Selective review provides an even smaller improvement over skip-review (46.40 vs. 46.21). Moreover, the substantial contribution of Tracker-Assisted Correction and the absence of a review-free baseline retaining the other correction mechanisms make it difficult to establish the necessity and independent effectiveness of multi-agent verification.
Questions for authors
-
Could the authors provide an ablation of JPG + TC without MAR to isolate the contribution of multi-agent verification? Given the marginal improvement of always-review over skip-review, what evidence establishes the necessity of additional LLM-based review?
-
Could the authors independently evaluate the contributions of procedural priors, instrument tracking, label correction, and temporal phase aggregation? The current component ablations combine multiple mechanisms, making it difficult to attribute performance improvements to individual components.
-
Could the authors provide sensitivity analyses for the procedural prior thresholds (0.01 and 0.70), tracker confidence threshold (0.35), and temporal voting window (60 seconds)?
-
Could the authors report end-to-end inference latency for skip-review, always-review, and selective-review settings to complement the token consumption analysis?
-
How well does the framework generalize across different surgical datasets and procedures, particularly given its dependence on procedure-specific priors, manually defined correction rules, and tracker performance?