- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes ReplayBind, a training framework for language world models intended to strengthen the association between actions and their resulting textual observations. Given two observed action–outcome branches sharing the same visible state, ReplayBind compares the likelihood of the recorded action–outcome assignment against a crossed assignment in which the outcomes are exchanged. It combines forward outcome prediction and inverse action reconstruction, followed by observation restoration and balanced replay to retain full-observation generation ability.
The method is evaluated on ALFWorld, SciWorld, TextWorld, and WebShop using action–outcome assignment margins, action reconstruction, generated branch separation, observation Token F1, and ten-step recursive prediction. The paper reports improvements over matched controls and several external language/world-model baselines.
Critical strengths
-
The matched ablations are reasonably careful.
-
The paper provides several diagnostics beyond overall observation overlap. Branch margin, both-correct rate, distinctive-content F1, inverse action reconstruction, and recursive prediction attempt to examine action-dependent differences rather than relying solely on Token F1.
Critical weaknesses
-
The shared-state assumption is not adequately verified.
ReplayBind relies on comparing alternative actions from the same state, but only ALFWorld provides replay-verified branches. Other environments use identical visible observations, which do not guarantee identical underlying states. This undermines the interpretation of the learned differences as action-induced effects. -
The methodological novelty is limited.
The core contribution is essentially a crossed pairwise likelihood objective with forward and inverse supervision, followed by conventional observation restoration and replay. Its distinction from existing contrastive or ranking-based training methods is insufficient to establish a substantial methodological advance. -
The experiments do not demonstrate actual planning benefits.
The evaluation focuses on observation reconstruction, likelihood margins, and textual branch separation. Even recursive prediction follows fixed logged actions rather than model-guided decisions. Consequently, improved prediction fidelity is not shown to translate into better planning or closed-loop agent performance. -
The headline performance gains are confounded by extensive prior training.
ReplayBind inherits substantial domain-specific adaptation and counterfactual supervision that external baselines do not receive. Although matched ablations isolate the crossed-loss contribution, their improvements are considerably smaller than the headline gains, making the actual contribution of ReplayBind difficult to assess. -
The branch evaluation provides limited evidence of generalization.
The primary TextWorld evaluation contains only 15 branch pairs from two conversations. The additional holdout expands coverage but does not verify identical underlying states across branches. This limits the evidence for reliable action-conditioned prediction across diverse environments.
Questions for authors
-
For SciWorld, TextWorld, and WebShop, how can the authors establish that two branches with the same visible text actually correspond to the same underlying environment state rather than different hidden histories or latent states?
-
What capability does the crossed-assignment objective provide beyond standard pairwise ranking or contrastive supervision over positive and negative action–outcome pairs?
-
Can the authors provide evidence that ReplayBind improves closed-loop planning or task success when its generated futures are actually used by an agent to choose actions, rather than only reconstructing fixed logged trajectories?
-
How should the large gains over external models in Table 1 be interpreted when ReplayBind inherits extensive domain-specific adaptation and counterfactual supervision that those external models do not receive?
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the replay-verified state comparisons, additional objective-level baselines, and closed-loop planning evaluation.
However, several fundamental concerns remain. First, while the crossed-assignment objective demonstrates some empirical advantages, its methodological distinction from existing contrastive and assignment-based learning objectives remains limited, particularly given the relatively modest improvements over matched alternatives. Second, although the new closed-loop experiments provide useful evidence, the observed task-success improvements are small and their robustness remains unclear. Third, the shared-state assumption is verified primarily in ALFWorld, leaving the interpretation of action-induced differences in the other environments insufficiently established.
Overall, I acknowledge the improvements introduced during rebuttal, but I remain unconvinced that the current evidence sufficiently establishes the methodological novelty and practical significance of the proposed framework. I therefore maintain my original rating of 1 (Clear Rejection).