- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper combines a Bayesian LoRA predictor with external belief updates, information-gain-based test selection, and value-of-information stopping. Experiments on four binary diagnosis datasets report improved discrimination and calibration over several LLM baselines.
Critical strengths
1.Separating prediction from explicit belief tracking makes the diagnostic process easier to inspect.
2.The MLE and MC-Dropout comparisons share the training data and external controller with BAMD, providing useful evidence about the contribution of Bayesian parameter estimation.
Critical weaknesses
1.The experiments do not establish the central claim that architecture, rather than model scale, limits sequential diagnosis. The evaluation covers four small tabular binary tasks, and the small-versus-large model comparison changes both training and inference procedures. A conventional probabilistic predictor using the same external controller is also missing. The controlled MLE comparison supports a benefit from Bayesian estimation, but not the broader architectural conclusion.
2.The controller does not consistently optimize the utility used to justify continued testing. Equation 13 checks whether any remaining test warrants continuation, whereas Equation 9 selects by information gain per cost. These criteria need not select the same test, and the selected test may have negative net VOI. The paper does not establish conditions that prevent this mismatch.
Questions for authors
1.Can the authors provide a conventional probabilistic predictor using the same training data, observed features, and external controller? This would help distinguish the contribution of the LLM from that of the controller.
2.Whenever Equation 13 permits continuation, is the test selected by Equation 9 guaranteed to have nonnegative net VOI? If not, how often does this mismatch occur in the reported experiments?