- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper provides a broad empirical comparison of fine-tuned and retrieval-based LM world models, showing that fine-tuning is usually stronger while RAG is more sample-efficient and highly environment-dependent. It proposes hierarchical retrieval combined with fine-tuning, which improves downstream utility across evaluated settings.
Critical strengths
- Direct comparisons between fine-tuning and retrieval over self-collected experience are useful for LM-based world modeling, and the experiments cover several types of text environments rather than relying on a single benchmark.
- This is appropriate because a more textually faithful prediction does not necessarily lead to better action selection. The reported results also usefully show that neither FT nor RAG is uniformly preferable across environments.
Critical weaknesses
- The Hybrid gain is not attributed. Hybrid adds retrieval at inference and fine-tunes on retrieval-augmented inputs, and the paper does not specify which retriever it uses; no ablation isolates these factors. The 17/20 result therefore does not show that learned parametric/non-parametric fusion is responsible. This matters because Hybrid is the paper’s main proposed system.
- The oracle-normalized aggregate is fragile. The oracle only provides true next observations for five policy-proposed actions, and an imperfect LM value estimate still selects the action, so it is not a strict upper bound; Chameleon’s -4.50 normalized utilities also suggest a small denominator. This matters because the 77% vs. 60% headline depends on this normalization.
Questions for authors
- Report results for hierarchical retrieval without fine-tuning and for an FT model given the same retrieved transitions at inference time, using the same backbone, buffer, and evaluation budget. These comparisons are necessary to identify the source of the Hybrid improvement.
- Provide the raw utility, ground-truth-WM utility, number of evaluation episodes, and number of independent runs for each setting. Also report whether the 77% versus 60% result remains under aggregation based on raw rewards or after excluding settings with near-zero oracle utility.
- Clarify the implemented Chameleon reward. Appendix A states that the evaluated player receives when accused, but also states that this outcome cannot occur because the player is never assigned the Chameleon role. An incorrect accusation of that player appears possible.
- Typo. ("Three players is in any case" should be "are")