acceptodds
Under review as a conference paper at ICLR 2027

RELACE: RETROSPECTIVE LIKELIHOOD-BASED ACTION CREDIT ESTIMATION FOR LONG-HORIZON LANGUAGE AGENTS

52 views

Abstract

Group Relative Policy Optimization (GRPO) avoids a separate critic by estimating advantages from rollout groups. For multi-turn agents, however, trajectory-level supervision provides coarse, noisy credit: terminal rewards do not locate errors and can penalize useful actions alongside mistakes. Group-in-Group Policy Optimization (GiGPO) and subsequent methods refine supervision through state-conditioned comparisons, but their credit estimates remain sensitive to downstream decisions and outcomes. We introduce \RELACE (Retrospective Likelihood-based Action Credit Estimation), a critic-free framework that integrates retrospective action assessment with state-conditioned advantage estimation. RELACE evaluates executed actions through teacher-forced likelihood scoring under both their original contexts and outcome-augmented contexts. Comparing these likelihoods yields a trajectory-normalized retrospective factor that captures outcome-dependent changes in action plausibility, rather than hindsight plausibility alone. We use this factor to reweight discounted task returns and construct local advantages by comparing weighted returns among actions from equivalent states within a task. This couples retrospective relevance with observed reward, producing fine-grained credit that complements trajectory-level GRPO supervision. Temporal smoothing and success-protecting masking further stabilize the local signal. RELACE requires neither auxiliary value or reward models nor additional autoregressive rollouts for credit estimation. Experiments on ALFWorld and WebShop with Qwen2.5-1.5B-Instruct and Qwen2.5-7B-Instruct demonstrate substantial improvements over GRPO, GiGPO, and HCAPO. With the 1.5B model, RELACE achieves 96.35% success on ALFWorld and 79.43% on WebShop, surpassing GiGPO by and percentage points, respectively.

Primary area:
reinforcement learning
Keywords:
LLMs, Reinforcement Learning, GRPO, Agentic RL, Long-horizon RL

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZgEhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RELACE, a critic-free credit-assignment method for multi-turn language agents. After collecting trajectories, it scores each executed action under its original context and a hindsight context conditioned on the observed terminal state. The resulting score ratios are normalized and clipped within each trajectory, then used to reweight discounted terminal returns. These targets enter state-conditioned advantage estimation, which is combined with trajectory-level GRPO supervision. Experiments on ALFWorld and WebShop report improvements over several baselines.

Critical strengths
  1. The paper addresses an important problem: terminal rewards provide limited information about the contributions of individual actions in long-horizon interaction.

  2. Comparing original-context and hindsight-context scores is a reasonable attempt to distinguish outcome-dependent changes in action plausibility from baseline plausibility. The method uses teacher-forced scoring without requiring a separately trained value model or additional environment rollouts for credit estimation.

  3. Evaluation covers two environments and two model sizes. The authors also appropriately acknowledge that their scores represent retrospective relevance rather than causal contribution.

Critical weaknesses
  1. The central interpretation of the weights lacks direct empirical validation. RELACE measures how changing the context alters the likelihood of an executed action. However, compatibility with the observed terminal state does not necessarily indicate usefulness for the original task. In particular, the hindsight templates reformulate the goal as reaching the actual final observation. The paper does not directly establish that necessary or useful actions in successful trajectories receive higher weights than redundant actions. Table 1 measures downstream performance, Table 2 compares training variants, and Figure 3 reports aggregate scoring dynamics; none directly evaluates this correspondence. Appendix E states that per-step scores and weights are reported, but the examples provide observations and actions without those numerical values. Consequently, the reported performance gains do not establish that the method improves action-level credit accuracy.

  2. The retrospective weighting cannot distinguish actions within zero-reward failed trajectories. From Eq. (6), Ri=0R_i = 0 implies Qi,tH=0Q^H_{i,t} = 0 for every step, regardless of the retrospective weights. The macro advantage therefore still assigns the same negative signal to all actions in a failed trajectory when its task group contains successful trajectories. State-conditioned comparisons can provide different local corrections, but these remain based on downstream outcomes and need not identify where the failure actually occurred. If all trajectories for a task receive zero reward, both advantage components vanish. Thus, the proposed weighting does not resolve the motivating case in which a late mistake causes useful earlier actions to receive unfavorable credit. This is a substantive scope limitation rather than an implementation error and should be reflected more explicitly in the claims.

  3. The ablation does not isolate the main proposed addition. Table 2 compares hindsight-score weighting using pHQp^H Q with RELACE using a trajectory-normalized and clipped likelihood ratio. This changes the original-context denominator together with normalization and clipping. The improvement from 75.78% to 79.43% therefore does not establish the denominator's independent contribution. A comparison that removes only the denominator while preserving the other transformations would more directly test the central design choice. Moreover, the success-protecting mask removes all negative local corrections from successful trajectories, so identifying a relatively low-value action does not necessarily lead to its local suppression.

  4. Comparison settings and statistical reporting limit confidence in the gains. Appendix C.1 uses γ=0.6\gamma = 0.6 for RELACE and γ=0.95\gamma = 0.95 for GiGPO, directly affecting the step-level targets being compared. Table 1 also combines published baseline configurations without sufficiently establishing protocol equivalence. In addition, the cited HCAPO paper reports an ALFWorld 7B result of 96.9%±1.3%96.9\% \pm 1.3\% with smoothing, whereas Table 1 uses its 91.4%91.4\% result; the smoothed configuration is particularly relevant because RELACE also uses smoothing. Finally, the seed coverage and aggregation details referenced in the text and Figure 2 are not supplied in Appendix C. These omissions matter for interpreting the smaller performance margins.

Questions for authors
  1. Can the authors provide per-step weights for successful trajectories containing both necessary actions and clearly redundant actions, together with an independent assessment of action usefulness? A controlled comparison involving redundant steps or action removal would directly test the proposed interpretation.

  2. What specific benefit does retrospective weighting provide for zero-reward failed trajectories, beyond the existing macro and state-conditioned outcome comparisons? Please clarify the intended scope of the claim about protecting useful actions from late failures.

  3. Does the original-context denominator improve performance when normalization, clipping, smoothing, masking, and the discount factor are held fixed? Please also clarify seed counts, checkpoint selection, and the comparison with smoothed HCAPO.

  4. How sensitive are the retrospective weights to the hindsight prompt formulation? Since the current template replaces the original task with the observed final state, can the authors distinguish changes caused by outcome-relevant information from those caused by reformulating the goal? A comparison with a prompt that retains the original task and adds the final observation would help clarify this issue.

  5. How much of the reported improvement depends on the success-protecting mask? Since this mask removes all negative local advantages from successful trajectories, does RELACE mainly reinforce selected actions rather than suppress redundant ones? An otherwise matched comparison with and without the mask would help separate the contribution of retrospective weighting from that of asymmetric protection.

3 comments

We thank the reviewer for the detailed and constructive feedback. We have substantially strengthened the manuscript with two new action-level case studies, controlled ablations, additional theoretical analysis, and clearer experimental reporting. We address each concern below.

W1 and Q1. Direct evidence for action-level credit assignment

We agree that improvements in aggregate success rates alone do not establish the quality of action-level credit assignment. To directly address this concern, we have added two extensive WebShop case studies (Appendix E) reporting original-context likelihood scores, hindsight scores, retrospective weights, discounted returns, and final advantages.

Importantly, we independently identify necessary and redundant actions from their effects on the environment, rather than inferring their usefulness from RELACE's weights. We then compare RELACE against an otherwise identical estimator with retrospective weighting disabled (ρt=1\rho_t=1), using the exact same rollout batches and retaining all other components.

Case Study 1: Necessary purchase versus redundant action. Two trajectories successfully purchase the same product, but one performs an unnecessary color re-selection that leaves the environment unchanged.

  • The necessary click[buy now] receives ρt=1.053\rho_t=1.053, increasing its advantage from 2.946 to 3.034.
  • The redundant click[dark heather] receives ρt=0.961\rho_t=0.961, decreasing its advantage from 2.355 to 2.302.
  • Consequently, the advantage gap between the necessary and redundant actions increases from 0.591 to 0.732, a 24% increase.

Case Study 2: Identical actions with different usefulness. We compare the exact same action, click[5x-large tall], across two successful trajectories. In one trajectory, the action selects the required size; in the other, it redundantly re-selects an already selected size, leaving the environment unchanged.

Both actions have identical discounted returns (Gt=6G_t=6) and receive identical advantages (0.724) without retrospective weighting. However, RELACE distinguishes them:

  • Necessary action: ρt=1.081\rho_t=1.081, increasing its advantage to 0.824.
  • Redundant action: ρt=1.004\rho_t=1.004, decreasing its final advantage to 0.658 after temporal smoothing and state-conditioned comparison.

Thus, retrospective weighting changes the advantage gap from zero to 0.166, despite identical action text, identical discounted returns, and identical terminal success.

These examples directly demonstrate the mechanism the reviewer requested: RELACE can distinguish actions according to their usefulness in the observed environment, even when conventional discounted-return comparisons assign them identical credit. The distinction remains after all components of the implemented estimator, including success-protecting masking, are applied. Although these examples do not establish universally accurate causal attribution, they provide concrete action-level evidence beyond aggregate benchmark performance.

W2 and Q2. Zero reward trajectories.

We acknowledge that when all trajectories receive zero reward, RELACE's advantages vanish, as is also the case for GRPO and GiGPO. This is a general limitation of outcome-based learning, rather than a limitation specific to our retrospective weighting. Our contribution is to provide finer-grained credit assignment from available reward signals. Dynamic sampling and exploration-based methods are complementary approaches for handling all-failure groups.

Importantly, even when individual trajectories fail, RELACE can still provide informative local comparisons when the task group contains successful trajectories. The retrospectively weighted returns from successful trajectories influence the state-conditioned baselines, thereby affecting the advantages assigned to actions in failed trajectories. Unlike GiGPO, RELACE additionally incorporates hindsight information into these comparisons, allowing more informative credit allocation without requiring additional reward models.

W3 and Q3. Misunderstanding

We would like to clarify an important point about the ablation design. Our denominator-removal ablation changes only the original-context denominator, while retaining trajectory normalization, clipping, temporal smoothing, success-protecting masking, the discount factor, and all remaining training settings.

Specifically, the only modification is

\quad\longrightarrow\quad u_{i,t}=p^H_{i,t}.$$ The controlled WebShop results are: - **GiGPO:** 73.83% success. - **Without original-context denominator:** 75.78%. - **Full RELACE:** 79.43%. We have also strengthened the theoretical motivation in Appendix C by deriving the hindsight likelihood-ratio identity. This explains why hindsight plausibility alone is insufficient: an action may receive a high hindsight score simply because it was already probable in its original context. The denominator accounts for this baseline plausibility. Our practical likelihood scores approximate this principle rather than constituting exact hindsight posteriors. Regarding reporting, we have clarified that validation occurs **every five training updates**, and each run reports its highest validation success rate within the **first 200 updates**. For multi-seed experiments, we compute this maximum independently for each seed and then aggregate the resulting values, reporting the mean and standard deviation. Single-seed ablations are explicitly distinguished from multi-seed results. ### **W4. Stronger HCAPO baseline and comparison fairness** We thank the reviewer for identifying the stronger published HCAPO configuration. **We have updated Table 1 to include the published smoothed HCAPO results** and revised the corresponding comparisons. To investigate discount-factor sensitivity, we additionally evaluated RELACE at $\gamma=0.95$, obtaining **77.34% WebShop success**, compared with 79.43% at its default $\gamma=0.6$. This indicates that RELACE's performance is not solely dependent on choosing the smaller discount factor. We are currently performing similar ablation studies on GiGPO as well. For $\gamma=0.6$ on GiGPO, the accuracy is $74.22\%$ and the validation success rate curve is very instable, often fluctuating between high and low values. ### **Q4. Sensitivity to hindsight prompt formulation** Our hindsight prompt intentionally conditions on the **observed terminal state** to measure how the outcome changes the likelihood of an executed action. This is used only for retrospective scoring, without changing rollout generation. Retaining the original task alongside the final observation could introduce conflicting objectives, particularly in failed trajectories, where the intended goal differs from the observed outcome. Nevertheless, we agree that comparing prompt formulations is an interesting direction for future analysis. ### **Q5. Contribution of success-protecting masking** We have added component ablations in **Appendix D.3**. Removing the success-protecting mask reduces WebShop success from **79.43% to 75.00%**, while removing temporal smoothing reduces it to **75.78%**. Importantly, **retrospective weighting remains effective even with the mask fixed**, as demonstrated by our case studies. Although the mask prevents negative local advantages on successful trajectories, RELACE still reduces the positive advantages of redundant actions (2.355 to 2.302 and 0.724 to 0.658), while increasing those of necessary actions. Thus, the mask does not eliminate RELACE's ability to distinguish useful from redundant actions. ## **Summary** We thank the reviewer for the constructive suggestions. We have now added **two detailed case studies, controlled ablations, stronger baseline comparisons, and additional theoretical justification**, while clarifying the limitations of our method. We believe these additions directly address the main concerns and substantially strengthen the paper. We hope the reviewer finds the revised manuscript satisfactory and welcome any further questions.

Thank you very much for your response. I have read it carefully; however, my concerns mentioned earlier regarding the methodology and experiments remain, and therefore I have decided to maintain my score (2).

Reviewer_CKW5OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes RELACE, which weights the discounted rewards by comparing the likelihood ratios of actions in the original context and the subsequent context, and combines state-level advantage estimation to achieve credit assignment for long-term LLM agents. Experiments show that this method outperforms multiple existing methods on ALFWorld and WebShop.

Critical strengths

This paper focuses on the credit allocation issue in long-term agent reinforcement learning.

The proposed method does not require an additional value model and has relatively low computational cost.

The experimental results are satisfactory, especially outstanding on ALFWorld, and comparisons have been made with various existing methods under two model scales.

Critical weaknesses

W1. RELACE combines the post-event likelihood score of HCAPO with the state-level advantage estimation of GiGPO. The newly added original context likelihood correction is relatively simple, but it has limited overall innovation and lacks experimental verification of its independent contribution.

W2. The discount factor used by RELACE is 0.6, while GiGPO uses 0.95. This may affect the fairness of the experimental comparison. It is recommended to unify the discount factor or conduct relevant sensitivity experiments.

W3. The paper mainly proves the effectiveness of the method by improving the task success rate, but it lacks an analysis of whether the likelihood ratio can effectively distinguish critical, redundant, and incorrect actions. Additionally, according to formula 6, when the final reward is zero, the post-event weighted return is also always zero, making it difficult to distinguish effective actions in failed trajectories through this mechanism.

W4. Although Table 2 compares several method variants, the paper lacks independent ablation experiments on key components such as time smoothing, success masking, trajectory normalization, and likelihood ratio clipping. It is impossible to clearly determine the specific contributions of each component to the final performance.

Questions for authors

see above.

1 comment

We thank the reviewer for the feedback. We have strengthened the paper with new theoretical results, controlled ablations, and action-level case studies.

W1. Novelty and theoretical insights

Our novelty goes beyond combining HCAPO and GiGPO. We introduce an improved retrospective factor, analyze it empirically, and added new theoretical insights in Appendix C to explain the credit-assignment mechanisms.

We interpret GiGPO's state-conditioned averaging as an implicit least-squares critic, explaining how it estimates local values without a separate critic. We derive a hindsight likelihood-ratio identity that motivates correcting hindsight action scores using their original-context likelihoods. Unlike HCAPO's practical trajectory-level approximation, RELACE explicitly computes this correction for each executed action.

We provide a detailed empirical analysis of how retrospective factors change action-level credit, beyond aggregate performance:

  • Direct action-level analysis (Appendix E): We report original and hindsight likelihood scores, retrospective weights, discounted returns, and final advantages. We compare identical rollout batches with and without retrospective weighting, keeping all other components fixed.
  • Necessary versus redundant actions: In one case, RELACE increases the necessary purchase's advantage from 2.946 to 3.034, while decreasing a redundant action's advantage from 2.355 to 2.302, increasing their advantage gap by 24%.
  • Identical actions, different credit: In another case, two identical actions with identical discounted returns receive the same advantage (0.724). RELACE distinguishes them by context, increasing the necessary action's advantage to 0.824 and decreasing the redundant action's advantage to 0.658.
  • Training dynamics: We analyze how original-context scores, hindsight scores, and retrospective weights evolve during training. Their different trajectories demonstrate why hindsight scores alone do not capture the same information as the likelihood ratio.

Our controlled denominator ablation improves WebShop success from 75.78% to 79.43%, supporting the importance of original-context correction.

Thus, our contribution is not simply combining existing methods, but introducing an explicit retrospective likelihood-ratio mechanism, providing theoretical insight into its motivation, and demonstrating quantitatively how it improves the distinction between necessary and redundant actions.

W2. Discount factor and baseline comparison

We originally used the selected hyperparameters: γ=0.6\gamma=0.6 for RELACE and γ=0.95\gamma=0.95 for GiGPO following their paper. We have now evaluated both methods at both discount factors:

  • At γ=0.6\gamma=0.6: RELACE achieves 79.43%, compared with 74.22% for GiGPO.
  • At γ=0.95\gamma=0.95: RELACE achieves 77.34%, compared with 73.83% for GiGPO.

These results show that RELACE's improvement is not simply due to its choice of discount factor. We clarified the seed counts and evaluation protocol.

W3. Retrospective scoring dynamics and failed trajectories

We have added training-time analyses of original-context scores, hindsight scores, and retrospective weights. Hindsight scores increase steadily, while original-context scores initially decrease before recovering, demonstrating that the two scores capture different information and motivating their ratio.

We also observe that the normalized mean weight remains near one, while upper-threshold clipping becomes less frequent during training. These results provide further insight into how retrospective weighting evolves.

Regarding failed trajectories, when all rollouts receive zero reward, RELACE cannot produce a nonzero advantage, as is also true for GRPO and GiGPO. However, in mixed-outcome groups, successful trajectories contribute retrospectively weighted returns to the state-conditioned baseline, providing additional information for comparing failed actions. Dynamic sampling and other related techniques are complementary directions for handling all-failure groups that are beyond the scope of this work.

W4. Additional ablation studies

We have added component ablations in Appendix D.3 to isolate the contributions of RELACE's mechanisms.

On WebShop, removing the success-protecting mask reduces success from 79.43% to 75.00%, while removing temporal smoothing reduces it to 75.78%. We are currently performing similar ablation studies on GiGPO.

Together with our denominator ablation and action-level case studies, these results demonstrate that the main improvement comes from retrospective likelihood-ratio reweighting, rather than simply adding existing stabilization techniques.

We believe these directly address the reviewer's concerns and substantially clarify RELACE's novelty and effectiveness. We thank the reviewer again and welcome any further questions.

Reviewer_BJGdOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RELACE, a hindsight-based credit assignment framework for long-horizon LLM agent reinforcement learning. The key idea is to leverage the final outcome of a trajectory as privileged context to retrospectively evaluate the likelihood of each executed action. Specifically, RELACE computes a likelihood ratio between hindsight-conditioned and original action probabilities, which is then used to reweight discounted returns. Building upon GiGPO, the method constructs step-level advantages by comparing hindsight-weighted returns among actions originating from similar states, and combines them with episode-level advantages for policy optimization. Experiments on ALFWorld and WebShop with Qwen2.5-based models demonstrate consistent improvements over several existing RL baselines.

Critical strengths
  1. Important Problem. The paper addresses credit assignment in long-horizon agentic reinforcement learning, where sparse outcome rewards make it difficult to identify the contributions of individual actions. This is an important and challenging problem, particularly for multi-step decision-making tasks.

  2. Intuitive and Lightweight Design. The proposed method introduces an intuitive way to leverage hindsight information for action-level credit estimation. By comparing action likelihoods with and without the final outcome, RELACE provides an additional credit signal without requiring a separately trained critic or reward model, making it relatively straightforward to integrate into existing RL frameworks.

  3. Promising Empirical Results. The experiments demonstrate consistent improvements across two agentic benchmarks and different model scales. The authors also provide ablation studies to investigate the contributions of key components, supporting the effectiveness of the proposed design.

Critical weaknesses
  1. Outdated Models and Limited Evaluation Scenarios. The experiments are conducted on Qwen2.5-based models, which are approximately two years old, and relatively simple benchmarks such as ALFWorld and WebShop. These settings may not adequately reflect the complexity of modern agentic tasks, which often involve longer horizons, more diverse tool interactions, and complex reasoning. It remains unclear whether RELACE can generalize to stronger contemporary models and more realistic agentic scenarios, such as software engineering agents.

  2. Unclear Source of Improvement and Theoretical Justification. RELACE uses the final outcome generated by the agent itself as privileged information to retrospectively evaluate its actions. However, since this outcome is produced by the same model without introducing additional external knowledge or supervision, it is unclear why conditioning on it should yield more reliable credit estimates. In particular, an increase in an action's hindsight likelihood does not necessarily indicate a positive contribution to task success. The paper lacks a convincing explanation or empirical validation of why such likelihood shifts should correlate with meaningful action-level credit.

  3. Insufficient Ablation Studies. RELACE combines several design choices, including hindsight likelihood ratios, trajectory-level normalization and clipping, temporal smoothing, state-conditioned advantage estimation, and success-protecting masking. Although the authors provide some ablations, the contributions of these components are not sufficiently disentangled. In particular, the improvements may partly arise from additional stabilization techniques rather than the proposed hindsight-based credit estimation. More comprehensive ablations, such as applying the same smoothing and masking strategies to GiGPO, would help isolate the contribution of the core method.

Questions for authors

see weaknesses

1 comment

We thank the reviewer for the constructive feedback. We have strengthened the paper with new theoretical insights, controlled ablations, and detailed action-level case studies. We address each concern below.

W1. Models and evaluation benchmarks

ALFWorld and WebShop are established benchmarks for long-horizon agentic RL, widely used in previous works such as GiGPO, HCAPO, and GraphGPO. We follow these prior evaluations to provide direct comparisons with established baselines under the same benchmark settings, allowing us to assess the improvements from our proposed credit-assignment mechanism.

Furthermore, RELACE is model-agnostic, requiring only action likelihoods and terminal observations, without additional value models or environment rollouts. We agree that evaluation on larger models and software engineering agents would be interesting and we can add that for the camera-ready version..

W2. Why hindsight information improves credit assignment

The terminal observation provides new information that was unavailable when the action was taken. Although the trajectory is generated by the agent, the final observation is feedback from the environment. RELACE uses this information to retrospectively evaluate executed actions.

Crucially, RELACE does not simply reinforce actions with high hindsight likelihoods. We compare hindsight and original-context likelihoods to identify how much observing the outcome changes the model's assessment of an action. Our new theoretical analysis in Appendix C derives the hindsight likelihood-ratio identity, motivating this correction.

More importantly, we now provide direct empirical evidence that this mechanism improves action-level credit assignment (Appendix E):

  • Necessary versus redundant actions: RELACE increases the advantage of a necessary purchase from 2.946 to 3.034, while decreasing a redundant action's advantage from 2.355 to 2.302.
  • Identical actions, different usefulness: Two identical actions with identical discounted returns initially receive the same advantage (0.724). RELACE increases the necessary action's advantage to 0.824 and decreases the redundant action's advantage to 0.658.

These actions are identified as necessary or redundant independently from their effects on the environment, not from their likelihood scores. We also compare identical rollout batches with and without retrospective weighting, holding all other components fixed.

Thus, our contribution is not simply conditioning on the final outcome, but using the change in action likelihood to improve credit assignment, supported by both theoretical motivation and direct action-level evidence.

W3. Independent contributions of the components

We have expanded our ablation studies to isolate retrospective weighting from the stabilization components.

First, our controlled denominator ablation changes only the original-context correction, keeping normalization, clipping, smoothing, masking, and discounting fixed. Removing the denominator reduces WebShop success from 79.43% to 75.78%, directly demonstrating its independent contribution.

Second, Appendix D.3 isolates the stabilization components:

  • Without success-protecting masking: 75.00%, compared with 79.43% for RELACE.
  • Without temporal smoothing: 75.78%, compared with 79.43% for RELACE.

Third, the improvements persist under matched discount factors. RELACE achieves 79.43% versus GiGPO's 74.22% at γ=0.6\gamma=0.6, and 77.34% versus 73.83% at γ=0.95\gamma=0.95.

We are also evaluating GiGPO with smoothing and masking to further isolate their effects.

These results directly address the concern that RELACE's gains arise only from stabilization techniques: the denominator ablation retains those techniques, yet removing the retrospective correction still reduces performance. Together with our action-level case studies, this supports the independent contribution of retrospective likelihood-ratio reweighting.

Closing

We believe the new theoretical insights, controlled ablations, and direct action-level evidence substantially address the reviewer's concerns about RELACE's motivation, novelty, and effectiveness. We thank the reviewer again and welcome further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.