- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Summary
This paper studies the role of evidential authority in clinical multimodal learning. Based on their theoretical insights, the authors propose ACF, which combines authority specification, protected graph fusion, and constrained projection to control how different modalities influence the final decision. The paper provides theoretical analysis and evaluates the method on controlled tasks and an MRI-to-semantic retrieval benchmark.
Critical strengths
Strengths
- The paper discusses the distinction between predictive utility, reliability, and authority, which is a relevant question for clinical multimodal learning.
- The authors provide several controlled experiments to examine the behavior of different components under predefined authority constraints.
Critical weaknesses
Weaknesses
- But now multi-modal learing in medical domain often consider modalities over two, so a main concern is that - does this proposed framework generalize to high modality learning scenarios? It will be better to see more limitation analysis or reltaed work discussing such efforts in the community [1,2].
- My main concern is that the core contribution is not sufficiently validated on real-world data. The authority mechanism is mainly evaluated on synthetic tasks, while the UTSW experiments focus on representation geometry and graph communication. I wonder why the proposed authority mechanism is not directly evaluated on a real clinical prediction task?
- I also question whether the proposed protected graph is necessary. Can you elaborate more on the motivation of this design?
- Another concern is the assumption of predefined authority rules. I think determining which modality should have authority is itself a difficult problem in real clinical applications. It may vary across task, for example, for cariovascular diseases detection, ECG might be useful. And for pleural effusion, Chest Xray might be more userful. I wonder whether the proposed framework is useful without reliable authority specifications.
[1] Liang P P, Lyu Y, Fan X, et al. High-modality multimodal transformer: Quantifying modality & interaction heterogeneity for high-modality representation learning[J]. arXiv preprint arXiv:2203.01311, 2022.
[2] Huang Y, Du C, Xue Z, et al. What makes multi-modal learning better than single (provably)[J]. Advances in Neural Information Processing Systems, 2021, 34: 10944-10956.
[3] Wu C, Shuai Z, Shen L. Rethinking Medical High-Modality Learning Under Missingness--A Long-Tailed Distribution Perspective[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 5641-5651.
Questions for authors
See weakness
4 comments
Response to Reviewer 14nY- Part 2
3. Motivation for the protected graph
APGM provides a stable reference state for auditing declared source dependencies while the final prediction uses auxiliary context. In an implementation audit, one can hold the authorized evidence, parameters, and compiled specification fixed, vary the auxiliary inputs, and inspect the designated state before readout. A change in that state reveals an auxiliary dependency across the declared boundary, even when the final prediction satisfies every output constraint. APGM enforces this dependency requirement; ACP enforces the output relations. Tasks requiring output compliance alone can use projected joint fusion.
Appendix Table B.8 provides a concrete check: a deliberately inserted auxiliary bypass produces mean state shifts of 0.062627 on S1 and 0.056688 on S2 while output AVR remains 0. Thus, an output-compliance check can pass while the designated state's source dependencies have changed.
The saved S1 reliability interventions also show retained auxiliary predictive utility: ACF residual Brier decreases from 0.281343 to 0.076133 as auxiliary reliability increases from 0.60 to 0.95, at fixed and , while its designated-state shift remains 0. In the strongest tested S1 conflict, mean residual Brier is 0.308853 for ACF and 0.288601 for learned-joint, with zero AVR for both. The evidence supports the audit property together with useful auxiliary prediction in the reliability settings; predictive performance depends on the intervention.
The revised Overview states the task requirement (pp. 4–5, lines 214–219), and the Illustrative example shows how auxiliary information reaches the final prediction while the designated state remains protected (p. 5, lines 221–234). Table 3 explicitly labels the deliberate-bypass audit (p. 8, lines 388–389). The reliability and conflict comparisons above are presented in this response.
4. Dependence on reliable authority specifications
With usable inputs and no active authority relation, ACF returns its unconstrained prediction; its authority-specific guarantees concern the relations supplied by the task protocol. The reviewer's examples highlight why the target and decision relation matter. The protocol determines which evidence statement supports a particular exclusion or probability order; validity and availability determine whether that relation is activated.
An incorrect policy that remains active has a different effect from an inactive relation. Table 1 measures its predictive cost even when the model satisfies the supplied constraints.
Table 1: Predictive sensitivity to authority-policy misspecification (Appendix Table B.5). Entries are mean Brier scores over five data seeds, each averaging three model initializations. Both policies yield zero AVR with respect to their supplied constraints.
| Task | Correct policy: Brier | Incorrect policy: Brier | Brier increase | AVR under either policy |
|---|---|---|---|---|
| S1 | 0.082380 | 0.271208 | 0.188828 | 0 |
| S2 | 0.196675 | 0.280580 | 0.083905 | 0 |
Constraint adherence and predictive correctness are assessed separately. The conditional risk guarantee assumes that the true conditional mean belongs to the feasible set. For evidence-state changes, missing or invalid supporting evidence withdraws its relation; available evidence with unknown validity can remain auxiliary. Contradictory active constraints or absence of usable inputs lead to abstention (Appendix Table B.6).
The revised example distinguishes the protocol-supplied mapping from activation conditions (p. 5, lines 221–225) and describes withdrawal and fallback behavior (p. 5, lines 231–234). The existing risk statement gives the feasible-set assumption (p. 6, lines 318–320), and the conclusion summarizes the observed cost of policy misspecification (p. 8, lines 418–419).
Addendum: Response to Weakness #4
Following our earlier response, we added a retrospective downstream experiment on UTSW glioma grading to give a concrete example of how a task protocol can specify an authority relation.
Rule specification. WHO CNS5 assigns grades 2 and 3 to IDH-mutant, 1p/19q-codeleted oligodendroglioma. For this experiment, we set when a curated UTSW record contains both molecular findings, a documented IDH testing method (IHC or NGS), and a prespecified oligodendroglioma eligibility flag derived from the historical tumor type. The clinical classification specifies which grades the molecular evidence can constrain within this scope. This retrospective task conditions on existing molecular reports and historical diagnostic information. Historical tumor types within the prespecified scope, including grade-specific subtypes, map to a single eligibility flag shared by both models. This flag both identifies rule applicability and enters the molecular-report features of both models; the original diagnostic text is excluded from model inputs, and the target grade is withheld from rule compilation. The rule activates for 58 of 618 labeled patients. Five additional molecular candidates stay in the cohort with this rule inactive because their historical types fall outside the prespecified scope. This is a specification for this grading task. Applying the framework to another target would require an evidence-to-constraint mapping for that target. As explained in our earlier response, with usable inputs and no active authority relation, ACF returns its unconstrained prediction. [1, 2]
A further example of a guideline-defined output relation is the assignment of CNS WHO grade 4 to IDH-mutant astrocytoma with documented homozygous deletion of CDKN2A/B, as recommended by [3].
Both fusion models receive the same evidence, eligibility flag, and output projection. ACF additionally protects the active report state from MRI updates, while MRI remains available through auxiliary states and the grading head.
Table 1: Five-fold out-of-fold UTSW grading results (618 patients). Brier is dimension-normalized; AVR is computed over the 58 activated relations. Lower Brier and AVR and higher accuracy are better.
| Method | Brier | Accuracy | AVR |
|---|---|---|---|
| Label prior | 0.1623 | 68.0% | 1 |
| Label prior + projection | 0.1406 | 74.0% | 0 |
| Molecular + projection | 0.0944 | 79.6% | 0 |
| Learned-joint + projection | 0.0892 | 80.6% | 0 |
| ACF | 0.0928 | 79.3% | 0 |
The clinical specification supplies a concrete class exclusion for this task, and both fusion models satisfy all 58 active constraints. Learned-joint has higher observed accuracy and lower Brier than ACF; the paired 95% interval for the Brier difference (ACF minus learned-joint) is . Appendix C reports the activation protocol and full results (PDF p. 32, lines 1682–1692 and 1713–1719).
The revised manuscript presents the downstream experiment in Section 4.4 (PDF p. 8, lines 416–431) and the paired case in Figure 4 (p. 9, caption lines 469–473).
References
[1] Louis et al. (2021). The 2021 WHO Classification of Tumors of the Central Nervous System: a summary. Neuro-Oncology, 23(8), 1231–1251.
[2] Reddy et al. (2026). The University of Texas Southwestern Glioma Dataset — MRI, Molecular Markers and Segmentations. Scientific Data, 13, 934.
[3] Brat et al. (2020). cIMPACT-NOW update 5: recommended grading criteria and terminologies for IDH-mutant astrocytomas. Acta Neuropathologica.
Thank you for the rebuttal, I will maintain my positive scores.
Response to Reviewer 14nY- Part 1
Thank you for recognizing the distinction between evidential authority, predictive utility, and reliability, and for raising questions about modality scale and clinical applicability. We clarify the supported scope, the role of protected computation, and the conditions for using authority specifications below.
All page and line references below refer to the revised manuscript PDF.
1. Extension to more modalities
Both AIS tasks use four sources, and the formulation admits M sources with relation-specific authorized sets. S1 specifies one relation per record, and S2 specifies two, with designated sources varying across records. Each relation activates when its supporting evidence is valid and available. The non-interference argument follows the permitted computational dependencies for each relation at fixed parameters, compilation, and authorized inputs.
The empirical scope is four sources and up to two simultaneous relations. At larger source counts, dense attention increases the pairwise computation per relation; predictive performance at that scale would require further evaluation.
The suggested works address complementary aspects of this extension: HighMMT studies modality and interaction heterogeneity [1], Huang et al. analyze conditions for predictive benefit from additional modalities [2], and REMIND examines long-tailed modality combinations under medical missingness [3]. ACF addresses the supplied task specification governing computational dependencies and output relations. The revised Section 3.1 states the evaluated scope (p. 5, lines 235–237), and Section 3.2 gives the general M-source formulation (p. 5, lines 241–255).
2. Evidence from real clinical data
The evaluation separates controlled tests of authority from real-data tests of the fusion backbone. AIS provides known decision relations and controlled changes to source assignments, validity, reliability, and disagreement. This permits testing whether the mechanism follows a supplied relation while measuring the predictive consequences of that relation. The reason for using AIS is this experimental control over authority and its alternatives.
UTSW evaluates MRI-to-semantic retrieval and attributes differences to representation and communication choices. Its matched comparisons yield 6.485 mAP points for the evaluated SPD-versus-Euclidean configuration and 2.482 points for uniform-versus-identity communication (Appendix Table B.12). A direct clinical authority evaluation would additionally require a validated mapping from evidence statements to decision constraints and testing under real-source disagreement. The present authority-specific conclusions concern the controlled AIS setting.
The revised Section 3.1 distinguishes these settings (p. 5, lines 235–237), the abstract assigns the AIS and UTSW results accordingly (p. 1, lines 024–029), and the conclusion states the clinical validation requirement (p. 8, lines 421–423).
References
[1] Liang et al. *High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learning.
[2] Huang et al. What Makes Multi-Modal Learning Better than Single (Provably). NeurIPS 2021.
[3] Wu et al. *Rethinking Medical High-Modality Learning Under Missingness—A Long-Tailed Distribution Perspective.