acceptodds
Under review as a conference paper at ICLR 2027

CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents

Abstract

Long-horizon agentic LLMs are increasingly limited by finite context windows, as extended interaction trajectories can exceed the maximum context length before a task is completed. Context compaction offers a natural solution by summarizing previous interaction states and continuing the rollout under a compressed context, but incorporating compaction into reinforcement learning remains underexplored. We propose CompactionRL, a reinforcement learning strategy to train long-horizon agentic LLMs with context compaction. Our approach jointly optimizes task execution and summary generation with token-level loss normalization and cross-segment generalized advantage estimation. This design enables the LLM agents to learn from compacted long-horizon trajectories. We train CompactionRL on top of open models and observe consistent performance gains on agentic coding tasks. CompactionRL enables the open GLM-4.5-Air model (106B-A12B) to achieve Pass@1 scores of 66.4% on SWE-bench Verified and 26.2% on Terminal-Bench 2.0, exceeding the base model under inference-time compaction by 6.6 and 4.9 points, respectively. Built upon GLM-4.7-Flash (30B-A3B), CompactionRL improves Pass@1 by 5.5 and 6.7 points against the base model, reaching 56.0% on SWE-bench Verified and 20.2% on Terminal-Bench 2.0.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agent, Context Management, Long-Horizon Agents, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_1BMLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CompactionRL, a PPO-based reinforcement learning framework for long-horizon LLM agents under a fixed context budget. When the context nears its limit during rollout, the policy generates a summary and resumes from the summary plus recent turns. Both execution and summarization tokens are optimized under the same task reward. The authors address compacted-rollout issues with single-rollout PPO, token-level loss normalization, and cross-segment GAE. Validation on SWE-bench Verified and Terminal-Bench 2.0 shows gains over base models with inference compaction, standard no-compaction RL, and a variant without summary loss. Results suggest that learning compaction during RL improves long-horizon agentic coding performance without increasing peak context length.

Critical strengths

1.Jointly optimizing execution and summarization under a shared task reward is a strong practical contribution. It turns context compaction from an inference-time heuristic into a trainable policy component. Ablations show that summary loss matters, which is consequential for long-horizon agentic RL under fixed context budgets.

2.The optimization methods for compacted rollouts are technically meaningful. Token-level loss normalization reduces segment-count bias, and cross-segment GAE improves credit assignment across compaction boundaries. Ablations support both, strengthening the framework beyond a simple application of PPO.

Critical weaknesses

The empirical evaluation is primarily conducted on coding and terminal-style tasks with models from a single family. While the results are promising, it remains open how well the approach generalizes to other long-horizon agentic settings and to models with different architectures or scales. A broader evaluation would help clarify the scope of the claimed benefits.

Questions for authors

In training, summary tokens and execution tokens share the same final task reward. How does the method prevent the policy from learning to produce longer or more detailed summaries simply to gain credit, rather than because they improve continuation? Is there any length, quality, or efficiency regularization?

2 comments

The empirical evaluation is primarily conducted on coding and terminal-style tasks with models from a single family. While the results are promising, it remains open how well the approach generalizes to other long-horizon agentic settings and to models with different architectures or scales. A broader evaluation would help clarify the scope of the claimed benefits.

This evaluation scope is consistent with prior RL studies: DAPO and VAPO study mathematical reasoning with a single Qwen2.5-32B backbone [1,2], while RLEF, Self-play SWE-RL, Agent-RLVR, and Seed-Coder etc.. primarily evaluate on coding tasks [3–6].

CompactionRL is task-agnostic and requires no GLM-specific architecture, nevertheless, it achieves improvements at both 30B and 106B scales, including on Terminal-Bench 2.0 after training only on SWE-Dev. These results provide evidence of effectiveness across model scales and generalization beyond the training benchmark.

[1] Yu et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv:2503.14476.

[2] Yue et al. VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks. arXiv:2504.05118.

[3] Gehring et al. RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning. arXiv:2410.02089.

[4] Wei et al. Toward Training Superintelligent Software Agents through Self-Play SWE-RL. arXiv:2512.18552.

[5] Da et al. Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards. arXiv:2506.11425.

[6] ByteDance Seed et al. Seed-Coder: Let the Code Model Curate Data for Itself. arXiv:2506.03524.

In training, summary tokens and execution tokens share the same final task reward. How does the method prevent the policy from learning to produce longer or more detailed summaries simply to gain credit, rather than because they improve continuation? Is there any length, quality, or efficiency regularization?

The policy has no incentive to generate longer summaries merely to gain credit, as the reward depends solely on task correctness, not length. If leading to failed trajectories, longer summaries would instead receive greater negative penalties. We use no explicit length penalty or auxiliary summary-quality reward. Appendix C in our paper evaluates summary quality using a rubric covering fidelity, failure history, continuation usefulness etc... CompactionRL achieves a higher overall quality score (3.52 vs. 3.30), supporting that the learned summaries are of better quality and better facilitate subsequent task execution.

Reviewer_1BMLOpenReview ↗

Reply to author

Thank you to the author for the response; my concerns have been addressed.

Reviewer_1hynOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the challenge of context length limitations during the reinforcement learning training of long-horizon large language model agents. When the interaction history approaches the context limit, the agent uses the same policy to generate a summary and resumes execution based on that summary and recent interactions. Built on PPO, the proposed method partitions a rollout into execution and summarization segments, applying cross-segment advantage correction to leverage the final task reward for optimizing generation behaviors across both segment types.

Critical strengths
  1. The proposed method introduces an effective RL training scheme for long-horizon agents, demonstrating significant performance gains by incorporating context compression into RL training.

  2. The key training designs are thoroughly validated, supported by comprehensive experimental coverage and multi dimensional analyses.

Critical weaknesses
  1. The novelty of this paper is relatively weak. SUPO, which is mentioned in the paper, has already incorporated compressed summaries into the RL training of long-horizon agents. This paper merely replaces GRPO with PPO and introduces several optimization measures, without proposing any significant conceptual or methodological innovations.

  2. The paper scales the local advantage of the current segment by the number of tokens to be optimized in subsequent segments to approximate credit assignment over the full trajectory. However, this section does not provide a direct comparison with full-trajectory GAE or an analysis of the approximation error.

Questions for authors
  1. How close is the cross-segment GAE estimate to the full-trajectory GAE result? Can the authors provide an empirical analysis?

  2. The paper compares the 80k×4 compressed setting with a 160k uncompressed context. Can the authors report the number of generated tokens for both settings?

3 comments

The paper scales the local advantage of the current segment by the number of tokens to be optimized in subsequent segments to approximate credit assignment over the full trajectory. However, this section does not provide a direct comparison with full-trajectory GAE or an analysis of the approximation error. How close is the cross-segment GAE estimate to the full-trajectory GAE result? Can the authors provide an empirical analysis?

To provide a comparison with exact GAE, we randomly sampled 840 compacted trajectories and evaluated all segments to which cross-segment correction was applied. MSE measures the mean squared difference from exact GAE advantages over optimized tokens. We group trajectories by their number of compactions (1, 2, or 3).

As shown in Table 1, cross-segment GAE reduces approximation error by 48.4%, with larger improvements for trajectories involving multiple compactions.

Table 1. Advantage approximation error(MSE) against exact GAE with length-adaptive alpha=1.5.

Compaction CountLocal GAECross-segment GAEMSE reduction
10.020470.0139531.8%
20.031370.0118562.2%
30.031110.0111764.1%
Overall0.025050.0129348.4%

In our implementation, exact GAE required a sequential backward pass across compaction segments, preventing independent parallel processing within a batch and making training impractically slow. Cross-segment GAE preserves segment-level parallelism while reducing approximation error by 48.4%, with larger gains under multiple compactions.

The novelty of this paper is relatively weak. SUPO, which is mentioned in the paper, has already incorporated compressed summaries into the RL training of long-horizon agents. This paper merely replaces GRPO with PPO and introduces several optimization measures, without proposing any significant conceptual or methodological innovations.

CompactionRL provides a practical framework for large-scale RL on long-horizon agentic tasks. CompactionRL outperforms SUPO by 4.6 percentage points on SWE-bench Verified and 4.9 points on Terminal-Bench 2.0 on GLM-4.5-Air,achieving 3.3× SUPO’s gain.

These improvements are supported by optimization tailored to compacted setting.

  1. Single-rollout PPO removes SUPO’s within-group waiting to reduce policy lag, while its critic accounts for different reconstructed context states.
  2. Token-level normalization addresses compaction-induced segment-count imbalance.
  3. Cross-segment GAE mitigates temporal overweighting of early segments, reducing approximation MSE against exact GAE by 48.4% while preserving parallel segment processing as stated above.

The paper compares the 80k×4 compressed setting with a 160k uncompressed context. Can the authors report the number of generated tokens for both settings?

For the same 200 SWE-bench Verified tasks, the table below reports generated token counts for CompactionRL (80k ×4) and RL (w/o compaction)-160k.

Table 2. Generated token counts on SWE-bench Verified.

MetricCompactionRL (80k ×4)RL (w/o compaction)-160k
Mean total generated tokens per task25,98615,607
Mean generated tokens per summarization request2,609—

However,despite generating more tokens, CompactionRL achieves faster inference by reducing context length therefore reducing attention computation and KV reads during decoding. We randomly sampled 40 compacted trajectories, and found compaction reduces model-request time by 42.0% as shown in Table 3.

Table 3. Model-request time comparison.

MetricCompactionRL (80k ×4)No compaction
Mean request time per trajectory372.4 s642.4 s

Thank you to the authors for the detailed response and the additional empirical results. Since my primary empirical concerns have been adequately addressed, I have decided to raise my score to a Weak Accept, though I maintain my previous reservations regarding the methodological novelty of the paper.

First, I acknowledge the value of the supplementary experiments. However, at the conceptual and methodological level, my concerns remain. The core framework of this paper largely relies on the integration and direct adaptation of existing methods. Although cross-segment GAE reduces MSE empirically, it remains essentially a heuristic reweighting strategy lacking strict theoretical bounds, rather than a fundamental algorithmic breakthrough in cross-boundary credit assignment.

While the algorithmic contribution is incremental, considering that the context bottleneck in long-horizon agents is a crucial and practically significant problem,and given the substantial performance gains demonstrated on 30B and 106B scale models,I believe the paper's practical value and empirical rigor are sufficient to warrant acceptance at the conference.

Thank you for your constructive feedback and recognition of the practical value of our work. We will further clarify the motivation and formulation of cross-segment GAE, including its relationship to exact GAE, and incorporate the additional experimental results into the camera-ready version if accepted. We would appreciate any further suggestions to help improve our work.

Reviewer_FS1NOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper proposes CompactionRL, a PPO-based framework for training long-horizon agentic LLMs with context compaction. When the remaining context budget falls below a threshold, the same policy writes a summary of the interaction history, and the rollout resumes from the summary plus the most recent kk turns. Summary segments and execution segments are optimized jointly under the shared final task reward; there is no separate summary reward. Following SAO, the method uses single-rollout PPO with a learned critic instead of group-based advantages. On top of this, it adds two modifications: (i) token-level loss normalization, to handle imbalance in segment length and count; and (ii) a “cross-segment GAE” correction, which rescales each segment’s local GAE advantage by (γλ)N>s(\gamma\lambda)^{N_{>s}}, where N>sN_{>s} is the number of trainable tokens in later segments. Models are GLM-4.7-Flash (30B-A3B) and GLM-4.5-Air (106B-A12B), trained on SWE-Dev. Evaluation uses a 200-task subset of SWE-bench Verified and Terminal-Bench 2.0. Over the base model with inference-time compaction, the paper reports gains of +5.5/+6.6 on SWE-bench Verified and +6.7/+4.9 on Terminal-Bench 2.0. It also includes ablations (no summary loss, a SUPO/GRPO baseline, no token-level loss, no cross-segment GAE, and 160k no-compaction RL), inference-time hyperparameter sensitivity, and an LLM-judge study of summary quality.

Critical strengths

S1. Timely and practically important problem. Context compaction is already a standard mechanism in deployed coding agents such as Claude Code and Codex, but it is mostly used as an inference-time heuristic. How to train an agent so that it works well under compaction is a relevant open question, and the paper tackles it at realistic model scales (30B and 106B MoE) on realistic agentic benchmarks.

S2. Comprehensive ablations. The paper ablates each component and adds an extended-context (160k) RL baseline, a compute comparison with SUPO, a hyperparameter-sensitivity study, and a summary-quality analysis. This is valuable because readers can check where the gains actually come from. As discussed below, however, these same ablations show that the gains come mostly from components other than the claimed contribution.

Critical weaknesses

W1. The evidence does not support the core claim that training the summarizer drives the gains.

CompactionRL vs. w/o sum. loss is 54.5±1.2 → 56.0±0.4 (30B) and 64.5±2.4 → 66.4±0.7 (106B) on SWE-bench Verified. These gaps are similar to the reported std, measured on a 200-task subset (1 point = 2 tasks), from a single training run.

An untrained summarizer matches the method. In Table 1, the untrained GLM-4.7-Flash paired with an off-the-shelf Qwen3.5-27B summarizer already reaches 55.5, vs. 56.0 for CompactionRL.

Most tasks never trigger compaction. Only 79/200 tasks trigger compaction (App. C), at 0.36 compactions per trace (Fig. 3). This benchmark mostly does not exercise the method; longer-horizon benchmarks such as SWE-bench Pro are needed.

The largest gains come from known techniques. The biggest ablation drop is from removing token-level loss (66.4 → 60.0), an existing technique (DAPO) unrelated to compaction.

The PPO vs. GRPO/SUPO gap (66.4 vs. 61.8) is confounded: the advantage estimator, the sampling scheme, and the extra critic all change at once. The paper does not explain why PPO should help here, and the comparison is not matched on wall-clock time or GPU-hours, including critic cost. The proper baseline is PPO + compaction without the proposed changes; against it, the remaining gain looks limited.

W2. Limited novelty.

Joint training is not new. Training summarization and execution together under a shared outcome reward already appears in SUPO, MemAgent, MEM1.

Two of the three changes are borrowed. Single-rollout PPO comes from VAPO, and token-level loss comes from DAPO.

The new piece is a heuristic. Cross-segment GAE rescales the whole advantage, including the critic baseline terms, so it is not an approximation of GAE; it simply down-weights early segments. There is no theoretical justification and no comparison with exact GAE, which only needs bootstrapping at segment boundaries.

Overall, the paper reads as a re-assembly of existing methods plus a heuristic, with few new insights.

Other concerns:

Compaction overhead is not analyzed. Each compaction invalidates the KV cache and forces a re-prefill, and in training, overlapping prefixes are re-encoded.

No decontamination is reported between SWE-Dev and SWE-bench Verified. This mainly affects the headline gains over the base model, not the relative comparisons, since the RL baselines use the same data.

Questions for authors

How does vanilla PPO + compaction perform (segment-level loss, local GAE)? Can you show learning curves vs. GPU-hours, including critic training and value pretraining?

How does cross-segment GAE compare with exact GAE and with λ=1\lambda=1? How large are the advantages on summary tokens after scaling?

How are the KV cache and re-prefill handled after compaction, and what is the overhead?

6 comments

W1. The evidence does not support the core claim that training the summarizer drives the gains. CompactionRL vs. w/o sum. loss is 54.5±1.2 → 56.0±0.4 (30B) and 64.5±2.4 → 66.4±0.7 (106B) on SWE-bench Verified. These gaps are similar to the reported std, measured on a 200-task subset (1 point = 2 tasks), from a single training run. An untrained summarizer matches the method. In Table 1, the untrained GLM-4.7-Flash paired with an off-the-shelf Qwen3.5-27B summarizer already reaches 55.5, vs. 56.0 for CompactionRL.

Training the summarizer yields statistically significant gains on both model scales. We increased the number of evaluation repeats to six and the improvement from summary training is statistically significant(p value <0.05) as shown in Table 1.

Table 1. Additional evaluations of the summary-loss ablation.

BackboneEval. runsCompactionRLw/o summary lossp-value
30B655.4 ± 1.253.3 ± 1.80.0343
106B666.4 ± 0.764.2 ± 1.70.0228

CompactionRL also shows consistent stable performance across independent training runs. Table 2 below reports the results of both runs. The result reported in our paper is the average of the two runs.

Table 2. Two independent CompactionRL training runs on GLM-4.5-Air (106B-A12B).

Training runSWE 80k ×1SWE Comp. 80k ×4SWE 160kTB 80k ×1TB Comp. 80k ×4TB 160k
Run 157.3 ± 1.466.8 ± 0.7562.1 ± 2.121.4 ± 0.924.5 ± 1.523.1 ± 0.6
Run 260.8 ± 0.766.0 ± 0.061.5 ± 0.722.1 ± 1.128.0 ± 0.925.1 ± 1.5

While a strong off-the-shelf summarizer can support good task performance, summarizer training enables the summarizer's adaptation to the evolving actor policy during RL. Our ablation measures the benefit of this adaptation by comparing the same framework with and without summarizer training.

Most tasks never trigger compaction. Only 79/200 tasks trigger compaction (App. C), at 0.36 compactions per trace (Fig. 3). This benchmark mostly does not exercise the method; longer-horizon benchmarks such as SWE-bench Pro are needed.

SWE-bench Verified provides a meaningful evaluation of CompactionRL. Without compaction, over one-third of trajectories would exhaust the context budget and fail. Disabling compaction on the same checkpoint reduces accuracy from 66.4% to 59.1% (as stated in Table 8 in appendix B), an 11.0% relative drop, demonstrating its impact on overall benchmark performance.

This level of compaction is also a bottleneck for RL training. We chose the 80k context budget to ensure stable RL training without compaction, as shorter budgets led to training collapse.

CompactionRL also remains robust when compaction becomes more frequent. As shown in Table 3, CompactionRL substantially outperforms the base model under more frequent compaction and exhibits a smaller performance degradation, demonstrating its effectiveness and robustness under high compaction frequency.

Table 3. Effect of compaction frequency on performance on SWE-bench Verified.

Context BudgetModelCompactions Per TrajAccuracy (%)
80kCompactionRL0.3666.4
60kCompactionRL0.5765.2
80kGLM-4.5-Air0.4759.8
60kGLM-4.5-Air0.6155.5

The largest gains come from known techniques. The biggest ablation drop is from removing token-level loss (66.4 → 60.0), an existing technique (DAPO) unrelated to compaction.

Instead of unrelated to compaction, token-level loss normalization addresses compaction-induced weighting bias. Sample-level loss assigns equal weight to each segment, making a trajectory’s contribution depend on its segment count and overweighting short segments such as summaries. Token-level loss instead assigns equal weight to each optimized token across segments. CompactionRL and DAPO apply token-level normalization for different purposes: DAPO addresses length imbalance across responses, whereas CompactionRL addresses segment-count and segment-length imbalance introduced by compaction within a trajectory. Correcting this bias improves accuracy from 60.0% to 66.4%.

The PPO vs. GRPO/SUPO gap (66.4 vs. 61.8) is confounded: the advantage estimator, the sampling scheme, and the extra critic all change at once. The paper does not explain why PPO should help here, and the comparison is not matched on wall-clock time or GPU-hours, including critic cost. The proper baseline is PPO + compaction without the proposed changes; against it, the remaining gain looks limited.

How does vanilla PPO + compaction perform (segment-level loss, local GAE)? Can you show learning curves vs. GPU-hours, including critic training and value pretraining?

PPO supports compaction in two ways: (1) differences in compaction frequency increase rollout-time variability; PPO enabled single-rollout sampling eliminates within-group waiting and reduces the associated policy lag in asynchronous training and (2) the critic estimates more accurate state-dependent value baselines for each segment’s reconstructed context, accounting for the state changes introduced by compaction.

The critic and the advantage estimation are integral to PPO thus are not considered as confounded independent changes. The rollout procedure remains the same, except that the group size is reduced to one.

The comparison of CompactionRL and GRPO is matched on a wall-clock time of approximately 24 hours. Our actor and critic share the same training GPUs while waiting for rollouts,thus the training costs of the two methods are comparable as is shown below. We note that the pretrained critic can be reused across subsequent RL runs thus value pretraining cost is therefore a one-time overhead rather than a recurring cost for each training run. Including the full preparation cost, the first CompactionRL run uses 1,777 GPU-hours:

Table 4. Training resources for CompactionRL and SUPO.

MethodTraining GPUsRollout GPUsValue pretrain GPU-hoursTotal GPU-hours
CompactionRL32 H800 (shared by actor + critic)32 H8003051777
SUPO32 H80032 H80001,600

SUPO is not vanilla GRPO with compaction, as it already incorporates compaction-specific optimizations. Thus, a more suitable comparison is when proposed optimizations are applied to both methods, CompactionRL achieves 66.4% versus 61.8% for SUPO. With respective optimizations disabled, vanilla PPO achieves 60.8%, compared with 59.3% for vanilla GRPO.

Table 5. Vanilla PPO performance.

PPO variantSWE-bench Verified
CompactionRL66.4 ± 0.7
Vanilla PPO + compaction60.8 ± 0.8

Figure below presents the training reward curve vs. GPU-hours. Costs include rollout generation, actor and critic training and an 305 GPU-hour for the 50-step value pretraining. Training curve url: https://hackmd.io/_uploads/B1bIIhQ0zl.png

How does cross-segment GAE compare with exact GAE and with λ=1\lambda=1? How large are the advantages on summary tokens after scaling?

To provide a comparison with exact GAE, we randomly sampled 840 compacted trajectories and evaluated all segments to which cross-segment correction was applied. MSE measures the mean squared difference from exact GAE advantages over optimized tokens. We group trajectories by their number of compactions. (1, 2, or 3).

As shown in Table 6, cross-segment GAE reduces approximation error by 48.4%, with larger improvements for trajectories involving multiple compactions.

Table 6. Advantage approximation error(MSE) against exact GAE with length-adaptive alpha=1.5.

Compaction CountLocal GAECross-segment GAEMSE reduction
10.020470.0139531.8%
20.031370.0118562.2%
30.031110.0111764.1%
Overall0.025050.0129348.4%

In our implementation, exact GAE required a sequential backward pass across compaction segments, preventing independent parallel processing within a batch and making training impractically slow. Cross-segment GAE preserves segment-level parallelism while reducing approximation error by 48.4%, with larger gains under multiple compactions.

When λ=1\lambda=1, cross-segment GAE equals exact GAE: both reduce to R-V(x).

After scaling, summary and execution tokens have comparable advantage magnitudes, with median ∣A∣|A| of 0.303 and 0.319, respectively.

W2. Limited novelty. Joint training is not new. Training summarization and execution together under a shared outcome reward already appears in SUPO, MemAgent, MEM1. Two of the three changes are borrowed. Single-rollout PPO comes from VAPO, and token-level loss comes from DAPO. The new piece is a heuristic. Cross-segment GAE rescales the whole advantage, including the critic baseline terms, so it is not an approximation of GAE; it simply down-weights early segments. There is no theoretical justification and no comparison with exact GAE, which only needs bootstrapping at segment boundaries. Overall, the paper reads as a re-assembly of existing methods plus a heuristic, with few new insights.

CompactionRL provides a practical framework for large-scale RL on long-horizon agentic tasks. CompactionRL outperforms SUPO by 4.6 percentage points on SWE-bench Verified and 4.9 points on Terminal-Bench 2.0 on GLM-4.5-Air,achieving 3.3× SUPO’s gain. It also outperforms Vanilla PPO + compaction by 5.6 points on SWE-bench Verified, demonstrating the benefit of our adaptations beyond adopting PPO.

These improvements are supported by optimization tailored to compacted setting.

  1. Single-rollout PPO removes SUPO’s within-group waiting to reduce policy lag, while its critic accounts for different reconstructed context states.
  2. Token-level normalization addresses compaction-induced segment-count imbalance, extending DAPO’s application of response-length imbalance.
  3. Cross-segment GAE mitigates temporal overweighting of early segments. As demonstrated above, cross-segment GAE reduces approximation MSE against exact GAE by 48.4% while preserving parallel segment processing, supporting its effectiveness in improving temporal credit assignment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.