acceptodds
Under review as a conference paper at ICLR 2027

COCAgent: Chain-of-Causality Multi-Agent Reasoning for Evidence-Based Medical Decision Making

Abstract

Strabismus subtype diagnosis requires integrating ocular alignment across gaze positions, deviation patterns, and clinical criteria, yet existing deep learning and vision-language models learn direct mappings from images to predictions, with reasoning that remains implicit and explanations weakly grounded in clinical evidence. We propose COCAgent (Chain-Of-Causality Agent), an multi-agent framework for evidence-based strabismus subtype diagnosis. COCAgent links multimodal ophthalmic evidence through a Chain of Causality: it constructs subtype-specific causal chains, verifies candidate diagnoses against structured clinical criteria, and generates the final diagnostic report and subtype prediction, while a verification mechanism filters out unsupported or inconsistent reasoning. Such causality-grounded diagnosis requires data that expose causal structure, which no existing strabismus dataset provides. We therefore further construct StrabBench, a clinical benchmark comprising the photograph of the nine cardinal positions of gaze, subtype annotations, doctor-in-the-loop diagnostic reports, and task-specific causal graphs with node-level analysis. On StrabBench, COCAgent achieves 93.3% weighted recall, 96.1% BERTScore, and 83.4% CosSim, improving both subtype prediction and report quality over existing approaches.

Primary area:
interpretability and explainable AI
Keywords:
Chain of Causality, Multi-Agent Reasoning, Decision Making, Evidence-Based

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_MYfgOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents COCAgent for diagnosing strabismus subtypes and generating reports from nine-gaze photographs. It uses ocular measurements to build a graph, selects evidence paths with SHAP scores, and checks candidate diagnoses against clinical rules. The authors also introduce StrabBench, a dataset of 1,065 patients with subtype labels, clinician-reviewed reports, and evidence annotations. On a 104-patient test set, the system reports 92.0% weighted F1 and 93.3% accuracy.

Critical strengths
  1. The clinical problem is worth studying. Showing the gaze-position findings behind a prediction could help clinicians check the result.
  2. StrabBench provides reports and position-level annotations, not just subtype labels. These could help assess whether a model picks up the right findings, even when its final diagnosis is correct.
Critical weaknesses

1. Several inconsistencies in the evaluation and annotations make the results hard to trust.

Supplementary Sec. 2.3.1 states that a wrong diagnosis gets zero credit at all downstream levels. Averaged over the same cases, these scores therefore cannot exceed diagnosis accuracy. Yet in Supplementary Table 2, InternVL3.5-8B has 6.7% accuracy but 28.5% Position Alignment and 38.4% Treatment BERTScore. Qwen3.5-4B also has 17.3% accuracy but 20.4% Position Alignment. These numbers do not fit the stated protocol.

Main-paper Table 4 has another counting issue. With one chain per case over 104 cases, mean chain lengths of 1.31, 1.85, and 1.19 imply 136, 192, and 124 total edges. None can give the reported Cov@10 values of 92.5%, 80.8%, and 94.1% under the supplement’s pooled-edge formula, even after rounding. E.g., for 124 edges, the neighboring possible values are 116/124 = 93.5% and 117/124 = 94.4%. Position Alignment also changes definition: the main paper requires exact Position/Position Group matches, while the supplement expands groups into individual positions before computing coverage.

There are clear mismatches in the benchmark examples too. In Supplementary Fig. 3, Example 4 is labeled Exotropia and described as Non-A/V, but its graph root says A-pattern Exotropia. Example 6 is labeled and described as Vertical, yet its graph root says V-pattern Esotropia. The same report calls Position 2 “downgaze,” contradicting both the fixed mapping and its own later text. Some reference reports also infer numerical upgaze–downgaze differences and clinical thresholds from photographs, although the supplied rules explicitly say uncalibrated images cannot establish these thresholds.

2. The inference procedure and ablations do not line up, and the causal role of the chain remains unproven.

Sec. 4.3 says the final label is the candidate that passes verification. COC is used afterward to generate the report. Why, then, does adding COC in Table 3(c)→(d) change accuracy from 90.4% to 89.4%? The supplementary verification prompt also asks for a New_Label, but its role in the main pipeline is unclear. Without knowing how the final label is actually chosen, it is hard to interpret these ablations.

The gains do not clearly come from the chain either. Rules and verification alone reach 90.4% accuracy. Adding COC lowers accuracy and drops AccAng from 98.1% to 93.3%. The full system reaches 93.3% accuracy, only three more correct cases overall than the no-chain version, with Evidence Matching also added. A comparison against non-causal evidence selection or memory under the same setup is missing, so the benefit of causal structure is not isolated.

There is also a mismatch in what counts as a chain. Eq. 6 defines one continuous path to one target, but Supplementary Fig. 1 calls a path to Esotropia plus a separate edge to V-pattern a six-link chain. How the two three-class DAGs are combined into six-class candidates and a joint explanation is not explained clearly. Evidence Matching is described as choosing between two chains, yet the results say it “removes exactly those links.” Where does this edge removal happen? Also, an edge outside the ten most frequent CMU edges may still occur in the population statistics; that alone does not make it clinically unsupported.

The more basic issue is whether these relationships are causal at all. The paper interprets discovered feature dependencies as direct causes without adequately stating the assumptions or independently checking the directions. SHAP explains contributions to a classifier’s output, not clinical causation. Checking paths against frequencies produced by the same pipeline does not establish causality either. The COC prompt simply tells the model that each variable causes the next and asks it to explain the links. That may yield a plausible account, but it does not test whether those links are true.

3.The labels are described as being assigned from nine-gaze photographs, while the supplied rules state that these images cannot establish the quantitative thresholds for A/V patterns. Were the subtype labels confirmed using independent clinical measurements, or do they represent image-based judgments of a compatible trend? This distinction matters for what the classification results actually establish. The paper also gives little quantitative evidence of annotation reliability beyond clinician review, which is particularly concerning given the conflicting examples. Finally, Gemini drafted the reference reports and is also the backbone of COCAgent, but there is no independent clinical assessment to separate factual report quality from similarity to the reference style.

4.Qwen3.5-9B and Hulu-Med-7B both get 2.9% accuracy, while InternVL3.5-8B gets 6.7%. The larger models also do much worse than their smaller counterparts. This is possible, but the paper gives little explanation. Were the image inputs and model-specific formats checked? How were label variants, refusals, and unparseable answers scored? Without these checks or examples of failed outputs, it is hard to tell whether the low scores reflect diagnostic failures or problems with the evaluation setup.

Questions for authors

Pls see weaknesses above.

Reviewer_LB85OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript presents COCAgent, an evidence-driven agent framework for strabismus subtype diagnosis, and introduces StrabBench to evaluate structured diagnostic evidence. COCAgent first extracts structured ocular features and builds a causal graph. It then organizes diagnostic evidence through SHAP-based path selection, integrates CMU and clinical rule validation, and generates the final diagnosis and report. Experiments show improvements over the evaluated baselines in both subtype classification and report generation.

Critical strengths
  1. Given current VLM limitations in fine-grained visual analysis and evidence-based reasoning, combining structured ocular features with clinical validation is a reasonable way to improve VLM-based diagnostic reliability.
  2. StrabBench adds diagnostic reports and evidence-level annotations to conventional classification datasets. This supports research on evidence localization and diagnostic reasoning, rather than evaluating final predictions alone.
  3. COCAgent integrates structured feature extraction, evidence organization, clinical validation, and report generation into a coherent workflow. It shows clear improvements in both subtype classification and report generation. Ablation experiments further confirm the effectiveness of the clinical validation mechanism.
Critical weaknesses
  1. Prior work such as CI-GNN has already introduced causal feature analysis and SHAP-based explanations for strabismus classification. COCAgent demonstrates the engineering effectiveness of structured evidence selection and clinical rule validation. However, the independent contribution of COC relative to existing feature explanation and simpler evidence organization methods remains unclear. Table 3 shows that the performance gain mainly comes from “verification with knowledge-guided rules,” while adding COC actually leads to a slight drop in W-R.
  2. Table 1 does not directly compare Gemini-3-Flash-Preview, the backbone used by COCAgent. This makes it difficult to determine whether the gains come from workflow design or from the backbone itself. In addition, because this model also contributed to the reference-report generation process, it may introduce bias into text similarity metrics.
  3. In Supplementary Table 2, some baselines score higher on “downstream task score” than on “diagnostic accuracy,” yet the paper explicitly states that incorrect diagnoses should receive zero in downstream tasks. This conflicts with the definition of Hierarchical report-tree evaluation. In addition, in some benchmark examples, the labels do not match the evidence-graph titles.
Questions for authors
  1. Table 3 shows the additional benefit of combining COC with evidence matching. Can the gains from the “causal graph structure” and from “SHAP-based evidence selection” be separated further?
  2. When the Verifier revises the initial diagnosis, how does the framework ensure that the selected COC and the generated report remain consistent with the final subtype? An example of a “revised diagnosis” and its corresponding evidence chain would be helpful.
  3. Gemini-3-Flash-Preview should be included in the evaluation. This direct comparison is currently missing, so the attribution of COCAgent’s gains is not clean enough.
  4. Tables 1 and Supplementary Table 2 show that Qwen3.5, InternVL3.5, and Hulu-Med degrade markedly as model size increases. Could this phenomenon stem from genuine diagnostic errors, prediction collapse, or issues with output formatting and label parsing? Per-class predictions and invalid-output rates would help; otherwise these anomalous results are hard to interpret.
Reviewer_zh7bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents COCAgent, a multi-agent framework for strabismus subtype diagnosis that combines landmark-based feature extraction, causal graph discovery, Shapley-weighted Chain-of-Causality construction, Causal Memory Units, clinical-rule verification, and VLM-based report generation. The paper also introduces StrabBench, a single-center benchmark of 1,065 patients with nine-gaze-position photographs, six subtype labels, ophthalmologist-reviewed reports, and sample-level causal graphs containing abnormal gaze-position nodes and textual analyses.

Critical strengths
  1. The paper targets a clinically meaningful problem, because strabismus subtype assessment requires comparing ocular alignment across multiple gaze positions rather than merely recognizing a single image pattern.

  2. The proposed benchmark goes beyond image-level labels by providing physician-reviewed reports and gaze-position-level evidence annotations, which could support more fine-grained study of grounding and evidence-based diagnostic generation.

  3. The framework attempts to decompose the diagnostic pipeline into interpretable stages, including feature selection, evidence-chain selection, clinical verification, and report generation, and the paper includes component-level ablations.

Critical weaknesses
  1. Could the authors clarify why the graph learned from observational landmark features and diagnostic labels should be interpreted as a causal graph? In G=(V,E)G=(V,E), the target YY is a disease subtype label rather than an intervention variable, and edges Xi→YX_i\to Y estimated from a retrospective single-center dataset may reflect correlation, annotation conventions, or confounding rather than causality. Please specify the causal assumptions, including causal sufficiency, faithfulness, acyclicity, absence of latent confounders, and why they are plausible in this setting.

  2. How is causal discovery performed when the feature vector X∈R9×N\mathbf{X}\in\mathbb{R}^{9\times N} contains highly correlated measurements derived from the same composite photograph and potentially from the same eye landmarks? Please provide the exact HCM algorithm, conditional-independence tests, treatment of continuous versus discrete variables, significance thresholds, bootstrap stability analysis, and the learned graph structure for each task.

  3. Why does the proposed “causal” feature filtering retain every feature XiX_i for which a directed path Xi⇝YX_i\leadsto Y exists in GG? This criterion does not distinguish direct causes, mediators, colliders, descendants, or potentially spurious paths, and its validity depends completely on the correctness of the estimated edge directions. Please provide an analysis showing whether the selected features are stable across resampling and clinically consistent with independent expert knowledge.

  4. Could the authors explain the conceptual consistency between causal discovery and Shapley attribution? The Shapley value φi\varphi_i produced by Fcls(x;θcls)\mathcal{F}_{cls}(\mathbf{x};\theta_{cls}) measures model-dependent predictive contribution, not causal effect. How does assigning φi\varphi_i as a node weight in Gshapley⁡G_{\operatorname{shapley}} establish a causal chain rather than a predictive-feature importance path?

Questions for authors
  1. Why are report-generation baselines evaluated zero-shot while COCAgent relies on a supervised classifier, a causal graph learned from the training set, class-specific CMUs accumulated from training chains, and hand-provided subtype-specific rules? The comparison in Table 1 mixes fundamentally different supervision levels. Please include stronger baselines that use the same training data and landmark features, such as a supervised classifier plus Gemini report generator, a rule-based classifier, a graph-free feature classifier with verification, and a non-causal retrieval or prototype-memory baseline.

2.Could the authors rule out report-style leakage caused by using Gemini-3-Flash-Preview both to draft the benchmark reports during StrabBench construction and as the LVLM backbone of COCAgent? Although ophthalmologists revise the drafts, the reference reports may retain Gemini-specific phrasing, structure, and vocabulary, which could inflate BLEU-1, ROUGE-1, BERTScore, and CosSim for the same model family. Please report evaluation against independently written clinician reports or use blinded physician scoring of factual correctness and clinical usefulness.

3.Could the authors provide inter-rater agreement and annotation provenance for subtype labels, abnormal gaze positions, node-level analyses, causal graph edges, and final reports? The paper states that senior ophthalmologists reviewed annotations, but does not report the number of annotators, independent annotation versus consensus procedures, disagreement rates, Cohen’s κ\kappa, or whether annotators had access to clinical information beyond the photographs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.