- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces ExpertRoute, a visual-token routing method for chest X-ray report generation. It trains a lightweight gaze scorer on radiologists’ eye-tracking data from REFLACX, then combines the predicted expert-priority regions with an independent semantic-coverage selector. The method retains exactly 192 of the original 1,369 visual tokens before LLM decoding, while keeping the vision encoder, projector, and report-generation backbone frozen. The experiments use MIMIC Validation, a locked MIMIC held-out cohort, and external OpenI data. At the 192-token setting, the method improves report-level RadGraph and GREEN scores over full-context inference across all three cohorts, with a reported 16.2% reduction in generation latency. However, finding-level CheXbert performance is not uniformly improved, and the approach does not reduce computation in the upstream vision encoder.
Critical strengths
-
Combines radiologist-derived visual priorities with semantic coverage, addressing the limitations of relying on either expert attention or model-derived selection alone.
-
Evaluates on a locked patient-level MIMIC cohort and an external OpenI dataset, with matched prompts and a frozen backbone. Includes recent token-reduction methods and controlled variants that help isolate the contribution of the proposed composition strategy. Reports paired patient-cluster bootstrap confidence intervals for key held-out comparisons. The RadGraph and GREEN gains over full-context inference have intervals excluding zero. Measures actual latency rather than relying only on token counts.
Critical weaknesses
-
Evaluation relies primarily on automated metrics. Better RadGraph or GREEN scores do not establish that generated reports are clinically safer or more diagnostically accurate. Prospective radiologist assessment is absent.
-
Full-context inference performs better on finding-level CheXbert F1 in the locked cohort. ExpertRoute therefore does not establish that compact routing preserves all clinically relevant evidence.
-
ExpertRoute is not consistently the best compact-context method across metrics and datasets. For example, VisionZip has higher RadGraph and GREEN point estimates on MIMIC Validation, while VisPruner has stronger finding-level F1 on the locked cohort. Reducing LLM input tokens by 86% yields a 16.2% latency reduction. This is understandable because the vision encoder and projector still process all 1,369 patches, but the practical speedup is less substantial than the token reduction might suggest.
-
The gaze scorer is trained on REFLACX, whose supervision is primarily based on frontal radiographs. The experiments do not establish transfer across different backbones or a broad range of clinical settings.
-
The 192-token operating point needs careful interpretation. It is the best observed budget for the proposed policy among the tested values, not a universal optimum. The best routing strategy changes with the budget, and the paper does not establish how robust this operating point is to other datasets or generators.
Questions for authors
- Can the authors provide radiologist evaluation of report correctness, omissions, and potentially harmful errors to establish whether the automated-metric improvements translate into clinical benefit?
- Why does ExpertRoute underperform full-context inference on finding-level F1 in the locked cohort, and which findings are most affected?
- How much of the improvement comes from gaze supervision itself versus the independent semantic selector? Are the gains consistent across multiple random seeds and alternative gaze-scoring architectures?
- What are the separate runtime and memory costs of the vision encoder, projector, semantic selector, and LLM? How would earlier-stage routing or approximate semantic selection affect end-to-end performance?
- Does the method transfer to other vision-language backbones, hospitals, and radiographic views without retraining the gaze scorer?
- How sensitive are the results to the choice of 192 tokens, and can the operating point be selected using validation data without relying on the final evaluation cohorts?