- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how historical interaction trajectories can inform the selection of current candidate actions in language agents. It introduces the Trajectory-Context Action Credit interface and a training-free implementation, TrajCredit. The method retrieves trajectory slices relevant to the current task, observation, and interaction history, then combines local action similarity, procedural-stage consistency, and subsequent trajectory progress to score candidate actions. These credit scores are subsequently fused with those of a base selector. The authors evaluate TrajCredit on ScienceWorld and ALFWorld through controlled action-ranking experiments, retrieval-confusing candidates, component ablations, short-horizon value alignment, and bounded environment interventions. The reported results show improvements in candidate-action ranking across the evaluated configurations and gains in short-horizon environment progress on ScienceWorld.
Critical strengths
- The paper addresses a practically relevant problem with a clear motivation.
- The proposed implementation is lightweight and modular.
- The experiments provide encouraging evidence of utility.
Critical weaknesses
- Converting past experience into action-specific scores and using those scores to influence action selection is already central to related research. For example, Retrospex [1] combines action values learned from historical experience with language-model action likelihoods, while Neural Episodic Control [2] uses experience memory to support action-value estimation. The paper acknowledges these connections but primarily distinguishes its contribution through an explicit candidate-level interface and a training-free implementation. The definition of in Equation (4) is sufficiently general to encompass existing experience-based action-scoring approaches. It does not establish distinctive properties, constraints, or analytical results that would make the interface a substantial conceptual contribution. As currently presented, the formalization mainly provides an organizing perspective on an existing problem.
- Combining Equations (14)–(17) gives Thus, TrajCredit can be understood as a neighborhood-based action-support score whose weights are modulated by stage and outcome information. Its components consist of text retrieval, string similarity, a scalar stage kernel, outcome weighting, and standardized linear fusion. Although this construction is reasonable, the paper does not sufficiently establish what new decision-making mechanism or generalizable scientific insight emerges from this particular combination. This concern is especially apparent on ALFWorld, where . The three-term formulation therefore reduces to a weighted combination of two distinct signals. Results on this environment cannot directly establish independent contributions from all three evidence views.
- The main comparisons evaluate whether adding the complete TrajCredit module improves a base selector. However, two particularly informative controls are missing: Base + local support under the same retrieval and tuning protocol, and a TrajCredit-only selector. In Table 2, outperforming the base selector under retrieval-confusing candidates does not, by itself, rule out the possibility that a simpler retrieval-score fusion would achieve comparable gains. Furthermore, Table 8 shows continued improvement as the fusion coefficient increases up to 16. Without a credit-only comparison, it remains unclear whether the gains reflect complementary information from the base selector and trajectory credit, or predominantly the influence of the memory-based score. Consequently, the current evidence does not sufficiently distinguish the benefit of the proposed multi-view construction from the benefit of introducing a strong memory-based scoring signal. This distinction is central to assessing the paper’s methodological contribution.
- Equations (17)–(18) specify that are nonnegative and sum to one. Section 5.1 further states that all coefficients are selected on an independent development split and frozen before testing. However, the paper does not report the actual component weights, their search space, the specific selection criterion, or whether they are shared across environments, backbones, and base selectors. The sensitivity analysis in Table 8 concerns the external fusion coefficient , which controls the combination of the base selector and TrajCredit. It does not assess sensitivity to the three internal component weights. Since this weighted combination constitutes the core of the method, these omissions limit reproducibility and make it difficult to determine whether the reported gains depend on a particular weight configuration.
[1] Xiang Y, Shen Y, Zhang Y, et al. Retrospex: Language agent meets offline reinforcement learning critic. EMNLP2024.
[2] Pritzel A, Uria B, Srinivasan S, et al. Neural episodic control. ICML2017.
Questions for authors
- Beyond a training-free implementation and an explicit interface, what substantive change does the method introduce in its mechanism or conclusions? Please clarify its relationship to experience-derived action values and candidate-action rescoring.
- $\textbf{Can the authors provide Base + and TrajCredit-only comparisons within the existing evaluation settings?}$ Please use the same development-set tuning principles and report paired differences against the full method. These comparisons would help isolate the contribution of the multi-view construction.
- Please report the final values of , their search space and selection criterion, whether they are selected jointly with , and whether they are shared across experimental settings. How much improvement does the selected configuration provide over equal weighting, and do the main conclusions remain stable under reasonable changes to these weights?