- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ViSTA augments a frozen vision-language model with numerical information from irregular clinical time series. An event encoder pools measurements, timestamps, observation gaps, and variable identities, and a cross-attention bridge adds zero-initialized, RMS-scaled residuals at the original chart-token positions. The pretrained decoder and language output head remain unchanged. Evaluation uses MIMIC-IV for creatinine-defined AKI during ICU hours (24,48], in-hospital mortality, and eleven categories of automatically generated temporal multiple-choice QA, across Qwen3.5-2B/4B/9B. Outcome prediction and supervised QA use separately trained adapters. ViSTA-Joint achieves the highest reported mean AUROC/AP among the compared Qwen adaptations at every scale. Supervised QA improves substantially over unadapted models but remains below LoRA, several specialized clinical predictors also outperform ViSTA on prediction.
Critical strengths
-
A favorable trainable-parameter tradeoff for outcome prediction. At 4B, ViSTA uses 0.582M trainable parameters, versus 6.193M for Chart LoRA and 7.032M for Text LoRA. Its AKI AP is 0.4650 versus the strongest competing adaptation's 0.4508, and mortality AP is 0.2651 versus 0.2464. ViSTA-Joint leads the reported prediction means across all three backbone sizes. This consistency makes the compact interface promising, although trainable parameter count alone does not establish lower runtime or memory use.
-
QA gains span several operations, with promising transfer after outcome training. At 4B, ViSTA-Separate achieves 69.27% accuracy, compared with 39.40% for unadapted text and 59.18% for the strongest of ChatTS, MLLM4TS, ITFormer, and OpenTSLM. Removing numerical summaries still leaves 66.45% versus 59.25% for the category-wise best of those four baselines. This supports usefulness beyond one especially favorable category. Chart/Text LoRA remain stronger at 72.09%/74.15%, making this a promising parameter-accuracy tradeoff. At 9B, outcome-only training yields 51.00% QA accuracy for ViSTA-Joint, versus 47.00% for unadapted text and 43.67% for outcome-trained Text LoRA. This suggests useful cross-task behavior, although its robustness requires repeated-seed validation.
Critical weaknesses
-
The distinctive benefit of the visual residual interface is not isolated. Prior MLLM4TS already combines numerical and plot features, while ITFormer connects temporal encoders to frozen language models. I therefore view ViSTA's incremental contribution as correction at native visual positions while preserving the pretrained output interface. Table 8 removes visual tokens or numerical residuals only at inference. These interventions change the input distribution, and the residual-only branch still depends on visual features through its queries and RMS scaling. They do not establish an advantage over a comparably trained numerical interface. The initialization/scaling ablations are useful, and Appendix F correctly acknowledges the limitation, but matched retraining controls are central to judging the proposed mechanism and its novelty.
-
The comparative advantage needs stronger robustness and implementation evidence. The 4B AP gains over the strongest adaptations are 0.0142 for AKI and 0.0187 for mortality. The reported bootstrap SDs quantify patient-sampling uncertainty for one trained seed, training variability and paired uncertainty for these outcome differences remain unreported. Overlapping marginal SDs alone establish neither significance nor its absence. Table 8 also reports the zero-initialization and RMS-scaling training ablations without uncertainty estimates, leaving the robustness of these design-specific gains unclear. Moreover, Section 4 and Appendix D omit baseline search ranges and selected configurations, including LoRA ranks/target modules and details of the modified interfaces and output heads. This is particularly consequential for understanding how an outcome-trained, task-head-based baseline such as MLLM4TS is evaluated on QA. These gaps make the claimed performance and parameter comparisons difficult to independently assess.
Questions for authors
-
Controlled experiment: With the same numerical encoder, backbone, and comparable trainable-parameter/token budgets, how does ViSTA compare with a numerical-only interface and a model using fixed, patient-independent chart tokens, each trained under its own input condition? These controls would determine whether the proposed visual pathway contributes beyond numerical adaptation.
-
Clarification and robustness: Can the authors provide the selected baseline architectures, output-head handling for outcome-to-QA evaluation, LoRA settings, and hyperparameter search ranges? Please report repeated-seed results and paired patient-bootstrap intervals for the principal 4B prediction comparisons and for the zero-initialization and RMS-scaling training ablations. This would materially affect my confidence in the reported ranking and design-specific gains.
4 comments
Dear Reviewer ki91, a brief follow-up: the discussion period closes this Wednesday, November 18 (AoE). We would be grateful for any feedback on our response and are happy to clarify anything before then. Thank you for your time.
Dear Reviewer ki91. We understand this is a busy period and appreciate your time. As the author–reviewer discussion closes tomorrow, November 18 (AoE), we would be grateful to know whether our response addresses your concerns. We can still clarify anything before the deadline and we look forward to discussing. Thank you again for your review.
Dear Reviewer, today is the last day of the author–reviewer discussion (November 18, AoE). Since November 9, we have provided all the analyses you requested. These are matched controls trained under their own input conditions, including fixed, patient-independent chart tokens; three-seed results with paired bootstrap intervals for the principal comparisons and the design ablations; and full baseline implementation details (Appendices L–N). We would be grateful if you could take them into account in your final assessment, and we remain available for any questions until the deadline.
Dear Reviewer, thank you again for the detailed review. As the discussion period ends on November 18, we wanted to check whether our response addresses your questions. Following your suggestions, we added matched controls trained under their own input conditions (numerical-only interfaces and fixed, patient-independent chart tokens), three-seed results with paired bootstrap intervals for the principal 4B comparisons and the design ablations, and complete baseline implementation details (Appendices L–N). We would be glad to clarify anything further before the deadline.