- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Core problem: Generating a skill retrospectively from a trajectory and then using that skill to increase the likelihood of actions in the same trajectory may mistake the reconstruction of past behavior for evidence of transferable learning. EvoCred investigates when such hindsight should influence credit assignment, on what grounds, and to what extent. Method: We first derive the standard GRPO advantage from within-group task rewards. For each target trajectory, we use a skill generated from another trajectory for the same task, and compare the likelihoods with and without the skill while holding the original actions, history, and prefix fixed. The resulting change in recursive support is filtered through gates for raw strength and directional compatibility, then standardized, passed through a tanh transformation, and centered using absolute base-credit weighting to produce a positive, bounded advantage multiplier. Skills may only redistribute credit within each trajectory’s existing budget; they cannot alter the sign of any nonzero advantage, create support where the base advantage is zero, or change the total absolute credit. All skill-related operations are used only during training. Novelty: The main contributions lie in the source of the skills, the admission criteria governing their influence, and exact per-trajectory budget conservation. The insight that the environment determines the update direction while the teacher modulates token-level magnitude has precedent in RLSD. Recursive support, sign alignment, and positive bounded multipliers are also consistent with AgentOPSD, while the reuse of trajectories within the same group has precedent in SDPO. The paper should therefore be positioned as a principled combination of existing components with novel constraints, rather than claiming every foundational component as original. For related-work positioning, the authors could discuss Ning Yang et al., Token-Importance Guided Direct Preference Optimization (ICLR 2026; https://openreview.net/forum?id=cMEnMVvMw9) when distinguishing token-importance weighting from provenance-aware credit redistribution. A broader conceptual comparison with Ning Yang et al., Proactive Constrained Policy Optimization with Preemptive Penalty (AAAI 2026; https://doi.org/10.1609/aaai.v40i32.39978) could clarify the difference between penalty-based constraint handling and the exact sign and credit-budget invariants used by EvoCred; PCPO does not address hindsight provenance and is not a directly interchangeable experimental baseline. Evaluation: The experiments cover three Qwen backbones and three task suites—ALFWorld, Search-QA, and WebShop—and include two environment-specific component ablations, skill diagnostics, and interventions using erroneous skills.
Critical strengths
SEED, Skill-SD, OPID, and AgentOPSD all treat an increase in the likelihood of a trajectory’s own actions under a trajectory-derived privileged teacher as a valid learning signal. Yet a skill is itself retrospectively inferred from the success label of that same trajectory. Its ability to increase the likelihood of the source trajectory’s actions may therefore amount to little more than reproducing source-specific observations and actions, rather than providing genuine evidence that those actions deserve credit.The paper’s value lies in showing that this problem cannot be solved through temporal aggregation or bounded reweighting. It must be addressed at the level of evidence provenance. The proposed framework is also conceptually clean: it separates the authority that determines credit from the mechanism that distributes credit, and elevates provenance to an explicit constraint. Specifically, the teacher trajectory must differ from the target trajectory, and the pairing must be fixed before any conditional scores are observed. This makes the framework broadly transferable.
Critical weaknesses
Classical debiasing through cross-fitting or sample splitting relies on samples being independent and identically distributed across folds. By contrast, the k sibling trajectories are correlated samples generated by the same policy on the same task. As policy entropy decreases, sibling trajectories are increasingly likely to exhibit similar behavioral patterns and failure modes. A skill derived from trajectory A is therefore not independent of trajectory B. The paper directly treats this insufficiently characterized correlated process as a falsification mechanism, yet provides no analysis of how sibling similarity affects debiasing effectiveness and includes no experiments varying the number of sibling trajectories.The empirical evidence likewise does not support a clean causal attribution. The primary evidence for the proposed mechanism is the ALFWorld self-source ablation, where performance increases from 84.3 percent to 89.6 percent. However, this comparison changes several factors simultaneously. Under the self-source condition, the target receives a restatement of its own experience. Under cross-source, outcome-aware pairing with reusable donors, it instead receives a skill derived from another trajectory. This changes the skill’s textual content, quality, and alignment with the target’s specific actions. The observed gap therefore cannot be attributed solely to the removal of resubstitution bias; it may simply result from replacing the original skill with one that contains a different amount or type of information.More concerningly, outcome-aware pairing selects the donor distribution according to trajectory outcomes before any scoring takes place. This may introduce a new form of selection bias, but the paper provides no analysis of this possibility.
Questions for authors
1.In the actual implementation underlying Table 2, does the “without centering” variant use q divided by two or u divided by two? Please provide the configuration or implementation corresponding to this row and reconcile the descriptions in Section 3.5 and Appendix C.1. If the implementation uses q divided by two, could the authors add a comparison that removes only the final weighted-centering step while retaining all other transformations? This distinction directly determines whether the independent empirical contribution of credit conservation is supported. 2.What is the complete outcome-aware donor-selection and fallback procedure? How is the corruption in Figure 4 generated, what proportion of skills are corrupted, and which skills receive the corruption? Which specific operators are disabled in the “No Hard Constraints” condition? Please report the task, backbone, number of repetitions, and meaning of the shaded regions, and explain the setup of the 240-step experiment. These details are necessary to determine whether the diagnostic and intervention results support the paper’s proposed mechanism. 3.Does the advantage of source exclusion and adaptive admission persist when donor success status is matched as closely as possible, or when the gate is fixed to produce an average influence magnitude similar to that of the full method? If additional training is required, a limited multi-seed evaluation on one representative task against the strongest neighboring baseline would be sufficient.
2 comments
Please follow up before the discussion period ends
Dear reviewer,
We are less than a day away from the end of the discussion period. Please read the authors' response and any revisions, and post a brief follow-up indicating which concerns remain and whether your assessment has changed. Please raise any outstanding questions promptly so the authors have time to respond.
Given the limited time, please focus on clarifications and existing evidence. You may identify gaps in the experimental support, but please do not expect the authors to conduct new experiments before the discussion period closes.
Thank you for your contribution!
Best,
Your AC
Thank you again for your careful review and helpful suggestions. As we are now past the halfway point of the rebuttal period, we would appreciate hearing whether our responses and additional experimental results have fully addressed your concerns. If so, we would be grateful if you would consider raising your score in light of the new evidence. If any questions remain, please let us know so we can address them during the discussion period.
We will incorporate the new results and the clarifications from this discussion into the revised manuscript.