- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes RiVaT-Fuse for image–tabular prediction with sample-dependent, direction-dependent modality trust. The fused representation minimizes a strongly convex quadratic energy, combining agreement with modality embeddings through positive-definite diagonal-plus-low-rank trust matrices and additive, multiplicative, and relational interaction evidence. A differentiable linear solve computes the representation. Training additionally incorporates a state-binned entropic robust-loss surrogate, consistency regularization, quadratic prediction heads, and a task-head coupling penalty. The paper establishes uniqueness and local sensitivity identities for the latent solution and evaluates the framework on patient-disjoint mBRSET partitions for DR grading, DME detection, and derived referable-status prediction. RiVaT-Fuse achieves the highest DR point estimates among three external-method adaptations under a shared 512-pixel supervised protocol: accuracy 0.8693, QWK 0.8472, and macro-F1 0.5737.
Critical strengths
-
A coherent and inspectable fusion formulation. Equations 2–6 define an explicit estimation objective with direction-dependent trust. Positive definiteness guarantees a unique solution, and numerical residual and derivative checks connect the analysis to the implemented solve. Retrained diagonal/scalar variants and frozen matrix interventions provide complementary evidence that directional structure affects DR predictions. This makes the proposed inductive bias concrete and testable.
-
Promising DR performance with appropriate treatment of patient dependence. RiVaT-Fuse improves DR point estimates over all three matched external adaptations. Against MADDi-R50, the accuracy improvement is 0.0270, with a paired patient-bootstrap interval of [0.0040, 0.0532] (Table 8). Patient-disjoint partitions, missing-label masks, and patient-cluster resampling are appropriate for repeated retinal photographs. These results support the practical promise of the complete configuration.
Critical weaknesses
-
The evaluation does not sufficiently isolate a reliable generalization benefit of the proposed fusion rule. Section 4.1 acknowledges that recipe development followed inspection of earlier test results. Validation-only checkpoint selection does not by itself resolve potential selection bias from that earlier development; the relationship between those evaluations and the final test patients needs clarification. Principal comparisons also use one training seed, while patient-bootstrap intervals exclude training and model-selection variation. Furthermore, image-only and conventional fusion controls use 224-pixel inputs without augmentation, whereas RiVaT-Fuse uses 512-pixel inputs and a different recipe (Tables 1–2). The matched external adaptations are valuable, but matched simple controls are needed to determine whether the additional fusion machinery provides a meaningful advantage.
-
The benefit specifically attributable to modality-uncertainty adaptation remains insufficiently demonstrated. Corruption experiments primarily compare adaptive states against privileged per-photo clean-state fixing, rather than external methods or retrained scalar/global-state/no-robust-loss variants under identical corruptions. Effects are mixed: at noise SD 0.10, adaptation improves DR QWK from 0.3341 to 0.3626 but reduces DME AUROC from 0.7044 to 0.6558. The global-state variant also nearly matches DR macro-F1 (0.5734 versus 0.5737), while achieving higher QWK and auxiliary AUROCs (Table 7). I appreciate the authors’ explicit limitations on calibration and robustness claims. Nevertheless, the predictive value of sample-dependent trust under uncertain inputs remains insufficiently separated from generic additional capacity and regularization.
Questions for authors
-
The paper states that recipe development and later mechanism analyses followed inspection of earlier test results. Which test results were inspected, and did they influence the architecture, hyperparameters, loss weights, training procedure, or threshold choices? Was the final reported test split completely untouched by all model-development decisions? If not, do the authors have results from a strictly held-out evaluation using a frozen recipe?
-
RiVaT-Fuse uses 512-pixel inputs, augmentation, and a distinct training recipe, whereas the simple image and fusion baselines use 224-pixel inputs without augmentation. Do the authors have image-only, concatenation, gated-fusion, or attention baselines trained under the same 512-pixel supervised protocol? Were these comparisons repeated over multiple training seeds? If not, how can the reported gains be separated from improvements due to image resolution and training-protocol differences rather than the proposed matrix-valued trust and variational solve?
-
Under the same corruption suite, did you directly compare the full RiVaT-Fuse model with the global-state, scalar-trust, no-robust-loss, and external-method variants? After accounting for differences in clean-input performance, do the sample-dependent states consistently improve performance under corruption for each task? Since the current results show gains for DR but possible degradation for DME, are these differences supported by uncertainty intervals across training seeds or patient-level resampling?
-
Please identify the non-substitutable contribution of RiVaT-Fuse relative to existing uncertainty-aware dynamic fusion, differentiable optimization layers, and low-rank tensor fusion. Is the main novelty the matrix-valued trust geometry, the quadratic latent solve, the combination of three interaction branches, the state-conditioned robust objective, or their integration? Could the authors specify component by component which parts are genuinely new and which are combinations of established modules?
Response on evaluation isolation, uncertainty adaptation, and non-substitutable contribution
Thank you for the detailed review and for highlighting the coherent, inspectable formulation and the appropriate handling of patient dependence.
Development timeline and the “test-set influence” remark.
Any earlier wording suggesting that test results influenced development was a polishing error. One author inadvertently confused this project with an entirely unrelated project strongly supported by MIT Medical School and affiliated hospitals; that project uses a different dataset whose test set is completely opaque. The two projects are wholly unrelated. The correct timeline (now explicit in the revised Appendix B.1) is that the protocol was frozen before the reported retraining. The final test partition (194 patients, zero overlap) was never used for architecture, hyper-parameter, loss-weight or threshold decisions. Checkpoint selection used only the validation composite score S (Eq. 10). All reported numbers were produced under this frozen recipe on the held-out patient-disjoint test set.
On 512-pixel simple controls.
The 224-pixel table (unimodal vs. simple early fusion) serves only to show that even elementary early fusion already outperforms unimodal models and uses the official ResNet-50 recommended resolution. We did not rerun these weak controls at 512 pixels because their performance was already too poor to be meaningful. The relevant comparisons are those in Table 1: HEALNet-R50, MADDi-R50 and TIP-R50 trained under the identical 512-pixel supervised protocol. Against MADDi-R50 the DR accuracy gain is +0.0270 with paired patient-cluster bootstrap interval [0.0040,0.0532] (Table 8). Gains therefore compare systems that differ in the fusion rule, not in resolution or training recipe.
Uncertainty adaptation under corruption.
The 34-condition suite is diagnostic: adaptive states versus privileged clean-state fixing on identical corrupted inputs. At noise SD 0.10 the adaptive model improves DR QWK from 0.3341 to 0.3626 while DME AUROC moves from 0.7044 to 0.6558; we report this mixed outcome and state that clean-state fixing is not a deployment baseline. Global-state and scalar-trust variants were fully retrained (Table 3 / Table 7); the adaptive construction improves primary DR metrics relative to both. We claim only measurable, task-dependent effects of sample-conditioned directional trust.
Non-substitutable contribution.
The organizing principle is the sample-wise strongly convex quadratic estimation objective (Eq. 4) whose unique minimizer is obtained by a differentiable linear solve (Eq. 5). Inside it we integrate positive-definite low-rank-plus-diagonal trust matrices (Eq. 2) that let reliability vary across latent directions, plus additive, multiplicative and relational interaction vectors that shift the estimation target, and a state-binned entropic surrogate used only at training. Proposition 1, residual checks ≤2×10−7, the whitened-operator identity ≤9.55×10−15, and finite-difference agreement 3.23×10−8 make the inductive bias concrete and falsifiable. This is distinct from generic OptNet layers, scalar uncertainty weighting, or low-rank tensor fusion that constructs the representation directly.
On the overall rigor.
The paper supplies a uniquely solvable sample-wise estimation objective, numerical verification that the implementation matches the analytic object to high precision, patient-disjoint evaluation that eliminates cross-patient leakage, dual experimental protocols, and systematic ablations plus mechanism diagnostics. Gains against strong external baselines are supported by patient-level intervals. What is uncommon is that the fusion rule itself is an explicit, strongly convex estimation problem whose solution can be verified both analytically (Proposition 1) and numerically (residual ≤2×10−7, whitened-operator identity ≤9.55×10−15, finite-difference agreement 3.23×10−8). Combined with the patient-disjoint gains over matched flagship methods in Table 1 and Table 8, this yields an inductive bias that is both theoretically well-posed and empirically inspectable, an unusually high standard of evidence for multimodal fusion under modality uncertainty. The result is a framework whose core claims can be checked rather than merely asserted.
Regarding citations, we have removed the small number of individual references that could be viewed as weakly related. All clarifications above appear in the revised PDF.
The core claims rest on the well-posed latent estimation problem, the verified geometry, and the patient-disjoint gains against matched external methods.