acceptodds
Withdrawn from ICLR 2027

SurgCouple: Coupling Global Context with Local Cues via Chain of Spatio-temporal Evidence for Surgical Video Understanding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Similar surgical phases and instrument configurations can involve distinct instrument–tissue interactions, so global context alone cannot discriminate local events. Existing multimodal LLMs, however, lack a mechanism for selectively attending to sparse, spatial-temporal cues. We introduce **SurgCouple**, the first surgical video reasoning framework to **couple global context with hypothesis-guided spatial-temporal evidence discovery**. An Evidence Chain progressively discovers temporal, spatial, and fine-grained evidence, with intermediate reasoning states providing hypothesis-guided feedback to the latter two stages to focus on the informative regions. A Reasoning Chain maintains an evolving interpretation of the surgical event, refining the hypothesis space as evidence accumulates. A Consistency Verifier ensures that local judgments remain grounded in the evidence actually selected and coherent across stages. With a frozen Qwen3-VL-8B, SurgCouple achieves 89.31% final-answer accuracy and a 58.92% valid chain rate, surpassing 81.94% and 34.01% for Qwen3-VL-8B, and 88.92% and 52.63% for GPT-5.5.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Surgical video reasoning, Evidence-grounded reasoning, Spatiotemporal evidence selection, Multimodal large language models.

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_BoauOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

SurgCouple addresses surgical video question answering. It builds an Evidence Chain and a Reasoning Chain on a frozen Qwen3-VL-8B backbone. The model selects relevant clips, then uses earlier reasoning states to guide spatial and fine-grained evidence selection. A consistency verifier provides additional supervision during training. On SurgCoT, the authors report 89.31% final-answer accuracy and 58.92% joint accuracy across the three stages. The paper also includes component ablations and transfer results on two external datasets.

Critical strengths

1.The paper addresses a meaningful problem. Important changes in surgical videos are often brief and localized. Using global context to guide progressively finer evidence selection is a sensible approach. 2.Comparisons using the same backbone show substantial gains, especially in third-stage accuracy and joint accuracy across all three stages. These results support the value of hierarchical evidence selection for this task.

Critical weaknesses

1.The contribution of the central mechanism is not clearly isolated. Table 2 includes a coupling ablation, but the paper does not specify which connections or losses are removed. This makes it hard to tell whether the gains come from reasoning feedback or changes in evidence supervision and training. The evidence-matching rules are also underspecified. These gaps make the core results difficult to interpret and reproduce. 2.The claims about consistency verification are stronger than the evidence supports. The verifier distinguishes original chains from artificially corrupted ones. This does not establish that it can assess whether the evidence supports a conclusion. Joint accuracy only measures whether all three answers are correct. It does not directly measure reasoning consistency. In Figure 3, all three methods reach the correct final answer. The example shows better intermediate answers and localization, but does not establish that the final decision depends on the selected local evidence.

Questions for authors

1.Which modules, connections, and losses are retained or removed in the coupling ablation? Are evidence supervision and training settings kept the same? 2.How are temporal, spatial, and fine-grained predictions matched to the annotated evidence? Please provide the thresholds and matching rules. 3.Have you evaluated text-only inputs, mismatched videos, or inputs with key evidence masked out? These checks would help establish whether the model actually uses local visual information.

Reviewer_aokrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SurgCouple keeps Qwen3-VL-8B frozen and adds a coarse-to-fine evidence scorer, a three-stage reasoning cascade, and a Consistency Verifier that is used only during training, evaluated on SurgCoT. The authors describe it as the first surgical video reasoning framework to couple global context with hypothesis-guided spatial-temporal evidence discovery. The coupling is implemented as a gated cross-attention block (Eq. 3). The stage questions and the knowledge input K come from SurgCoT, and SurgCoT's evidence clues supervise evidence selection during training but are not given at inference.

Critical strengths

This manuscript identifies a real gap: the same surgical phase can involve different local instrument–tissue events, so global context alone cannot resolve them. SurgCoT's stage questions and evidence clues turn this gap into a concrete supervised prediction problem.

Critical weaknesses
  1. The "coupling" described in the method is a sigmoid gate applied to multi-head cross-attention (Eq. 3), together with concatenating the reasoning states into the scoring inputs. In my view, such standard gating and conditioning should not be characterized as a novel reasoning principle.
  2. Temporal clip selection uses only the question and K; Section 3.2 excludes R1 by design so that the temporal scope does not merely confirm the hypothesis. Because spatial and fine-grained selection are then confined to the selected clips, this question-only step bounds all later evidence, and a wrong clip choice cannot be corrected by the reasoning state. No ablation lets R1 condition temporal selection, so the stated benefit of excluding it is not shown either.
  3. The abstract says the Consistency Verifier "ensures that local judgments remain grounded in the evidence actually selected," and Section 4.2 credits it with enforcing cross-step coherence. However, the verifier is used only during training (Section 3.4), so nothing checks a chain at inference. Table 2 shows that removing it costs 11.29 VCR, which supports it as a training regularizer, but not as a verification step of the deployed model.
  4. The comparison does not separate coupling from an extra source of supervision. Section 4.2 states that the Qwen3-VL-8B variants share "the backbone, architecture, and supervision," but only SurgCouple is trained with SurgCoT's evidence-clue annotations: Fixed-Aggregation Fine-Tuning replaces learned selection with fixed aggregation, and CoT Fine-Tuning is trained on stage-wise and final answers only (Appendix C). The "w/o Learned Evidence Selection" row in Table 2 also removes the selector and its supervision together, and the Limitations attribute the gains to these evidence clues. This matters because the final-answer margin over GPT-5.5 is only 0.39 points, and the large gains are on the stage-level metrics, where the extra supervision is most likely to help.
  5. Section 2 states that prior grounding methods "retrieve evidence from a fixed query in a single pass," and the reasoning-dependent selection in Eqs. 1–3 is the stated difference. However, Section 2 does not discuss multi-step or iterative evidence-selection methods that also condition later selection on intermediate states, and the only staged surgical work it covers is the SurgCoT benchmark itself. Without that comparison, the claim that this is the first framework of its kind is unsubstantiated.
Questions for authors
  1. Please name the operator in Equations 1–3 that iterative or multi-step evidence-selection methods cannot express, and explain why such methods are not discussed in Section 2.
  2. Could the authors either give a fine-tuned baseline the same evidence-clue supervision, or train SurgCouple's learned selector without it, so that the effect of coupling is separated from the effect of the extra labels?
  3. Is there a variant in which R1 also conditions temporal selection? It would show whether excluding it helps, as Section 3.2 argues.
Reviewer_ysTVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SurgCouple for surgical video understanding using a frozen Qwen3-VL-8B backbone. An Evidence Chain progressively selects temporal clips, spatial regions, and fine visual cues, while intermediate latent reasoning states guide subsequent selection. A Consistency Verifier supplies additional training supervision. On SurgCoT, the method reports 89.31% final accuracy and 58.92% joint correctness across three stage answers. Additional experiments report transfer to Surgical-VQA and Video-MME without parameter updates.

Critical strengths

The architecture provides a concrete mechanism for connecting global interpretation with localized visual information. The paper includes supervised comparisons using the same backbone rather than relying exclusively on comparisons with unadapted models. SurgCouple reports 89.31% final accuracy and 58.92% VCR, compared with 84.03% and 38.68% for CoT Fine-Tuning.

Critical weaknesses
  1. VCR measures whether three stage answers are correct, not whether their reasoning is valid or grounded in the selected evidence. It excludes both evidence matching and final-answer correctness.

  2. Algorithm 2 labels every uncorrupted model-generated chain as positive for the verifier, without specifying whether incorrect answers or unsupported evidence are filtered out. Distinguishing these chains from shuffled or scope-violating negatives may teach corruption detection rather than semantic validity. The verifier’s ability to identify naturally occurring reasoning errors is not established.

3.The evidence-selection ablation simultaneously removes scoring heads, evidence supervision, and evidence teacher forcing. It therefore does not isolate learned selection from additional supervision. Table 2 includes a coupling ablation, but its implementation is insufficiently specified to determine whether supervision, capacity, and evidence budgets remain matched.

  1. Equation (6) leaves the distance function and projection-head constraints unspecified. If both projections can learn constant outputs, they can minimize the alignment objective without preserving meaningful evidence–reasoning correspondence. Detaching the input evidence features does not itself prevent this solution. The implementation and any diagnostics against projection collapse should be clarified.
Questions for authors
  1. From existing predictions, what fraction of examples has all three stage answers, the final answer, and all three evidence matches correct? Are any evidence removal or replacement diagnostics available?

  2. Are uncorrupted chains with incorrect answers or mismatched evidence still positive verifier examples? What evidence shows that the verifier recognizes natural reasoning errors rather than synthetic corruption patterns?

  3. What exactly is removed in the “w/o Bidirectional Coupling” variant? Are evidence supervision, teacher forcing, parameter capacity, and selection budgets preserved?

  4. What temporal threshold, spatial mapping, and fine-grained matching rules produce Evidence Score? How are candidate regions and their semantic categories obtained without annotated clues at inference?

  5. What distance function and projection architectures are used in Equation (6)? Are the evidence-side projection heads frozen or learned, and what prevents trivial alignment?

  6. Please specify the reasoning and decoder architectures, aggregation operator, teacher-forcing schedule, and gradient handling for TopK selection.

Reviewer_zWQUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SurgCouple, a framework for surgical video understanding that couples an evidence chain with a hierarchical reasoning chain to ground reasoning in visual evidence. Built on a frozen Qwen3-VL-8B backbone, the evidence chain progressively selects temporal, spatial, and fine-grained visual cues, while intermediate reasoning states guide subsequent evidence selection through gated cross-attention.

Critical strengths
  1. The paper identifies an important limitation: relying solely on global surgical context may overlook the local evidence needed for accurate decision-making. Coupling an evidence chain with a reasoning chain offers a structured alternative to reasoning over uniformly aggregated visual features.
  2. The ablation results support the contributions of the proposed components. Removing learnable evidence selection reduces the evidence score from 74.62% to 51.73%, while removing the consistency verifier lowers VCR from 58.92% to 47.63%.
Critical weaknesses
  1. The current evaluation does not sufficiently substantiate the paper’s central claim of evidence-grounded reasoning. Direct validation of whether the selected evidence is correct and supports the corresponding answers is lacking.
  2. The original reasoning chains may themselves contain errors or unsupported reasoning. The experiments do not establish whether the consistency verifier can detect naturally occurring reasoning errors or cases where the evidence does not support the conclusions.
  3. The methodological contribution and the source of the performance gains require further clarification. The three-stage reasoning questions, knowledge, and evidence cues are inherited from SurgCoT, making it difficult to isolate the gains attributable to the proposed bidirectional coupling mechanism.
Questions for authors
  1. Could the authors report a joint metric that requires both correct stage-wise answers and accurate localization of the evidence supporting those answers? Could the consistency verifier also be evaluated on naturally occurring erroneous or unsupported reasoning chains, rather than only artificially corrupted ones? These evaluations would help clarify how reasoning and evidence each contribute to the results.
  2. Given that SurgCoT already provides hierarchical reasoning stages and spatiotemporal evidence cues, what is SurgCouple’s key methodological innovation beyond this framework? Could the authors further isolate and evaluate the respective contributions of bidirectional coupling and consistency verification?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.