- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a controlled benchmark of LLM probabilistic acquisition by generating synthetic Bayesian networks with known ground-truth distributions, and then verbalizing them into text for continuous pretraining. Through training over this corpus that faithfully represent these specified joint distributions, the authors show that while the LLM models the individual conditional probability closely, but the difference between different outcomes tend to be underestimated on the log-odds scale (which the authors referred to as "relation compression". Further experiment against different documentation rules and conditional updates reveal that the model tend to systematically under-express the update magnitude for strong evidences, while still undesirably being sensitive to irrelevant information.
Critical strengths
- The paper works with a controlled setting where exact probabilistic inferences are possible, and present this in a more LLM compatible experiment design than many previous works that also probe probabilistic inference capabilities.
- The evaluation goes beyond probability prediction accuracy, and the relational compression seems interesting.
- documentation experiments look new and thoughtfully designed with consequential results
Critical weaknesses
- The effect of additional independent data is unclear. It's unclear that whether additional independent examples can help beyond just more training epochs. Some sample-efficiency analysis can be helpful.
- The experiment design and probability readout centers around one probability readout, of which the broad application can be less convincing. In particular, the controlled setting usefully isolates probabilistic learning from reporting bias, but its implications for real-world probabilistic knowledge remain unclear. Other probabilistic modeling setting might better distinguish fidelity to a training corpus from recovery of world probabilities under selective reporting.
- The comparison experiments still do not isolate what makes probability recovery difficult. For example, it is hard to interpret the 5-variable results vs 3-variable results in Table 1. It seems that 5-variable should be the more challenging, but under different experiment settings the result does not direct compare and 5-variable ones are sometimes better.
Questions for authors
- Will more training examples reduce the compression / large error under rare events?
- Is there a principled explanation of the compression behavior for various attenuation / compression effect?
Engaging with the authors
Dear reviewer,
the authors have posted a reply to your review. Please have a look and interact with the authors as needed.
Thank you, Your AC