acceptodds
Under review as a conference paper at ICLR 2027

DGPO: Directional Gain Policy Optimization for Credit Assignment in Reasoning

Abstract

Outcome-level rewards supervise entire reasoning trajectories, even when their success depends on a few intermediate segments. We present Directional Gain Policy Optimization (DGPO), which converts binary outcome rewards into information-weighted segment-level credit without process-level supervision. DGPO constructs uncertainty-triggered reasoning trees to expose alternative continuations under shared prefixes. Its core credit signal, Directional Information Gain (DIG), combines normalized information gain with a signed measure of success association, capturing both how strongly a segment predicts the outcome and whether it is associated with success or failure. Global DIG captures tree-wide associations, while local DIG contrasts sibling branches; their normalized scores augment trajectory-level advantages for policy optimization. Across four language-model backbones and six mathematical reasoning benchmarks, DGPO achieves higher average pass@1 accuracy than GRPO, TreePO, and TreeRL on every backbone. With the same tree-construction procedure, DIG also outperforms direct segment advantages and TreeRL advantages in a four-benchmark comparison on Qwen3-4B-Base. A separate comparison on this backbone shows higher final accuracy at comparable total training time.

Primary area:
reinforcement learning
Keywords:
LLM, RL, Reasoning Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gehjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DGPO, which constructs segment-level credit from binary outcome rewards. Sustained high entropy triggers tree branching, followed by semantic-similarity pruning. DIG multiplies normalized mutual information by a bounded log-odds polarity term; standardized global and same-prefix local scores augment trajectory advantages. Experiments cover four backbones and six mathematical reasoning benchmarks, with credit-estimator comparisons under a fixed branching procedure and a comparable-runtime comparison on Qwen3-4B-Base. The appendix expresses unsmoothed DIG as nonnegative reweighting of direct advantages and provides conditional improvement results for independent branch-policy updates on a fixed tree.

Critical strengths

Introducing mutual information into segment-level credit assignment is a meaningful and well-motivated contribution. By combining information gain with directional association, the proposed method offers a principled approach to extracting fine-grained learning signals from outcome-level rewards. The consistent improvements across multiple model backbones and benchmarks further highlight the potential of this direction.

Critical weaknesses
  1. Although Tables 2 and 5 provide evidence for the effectiveness of DIG, the contribution of the normalized information-gain factor remains unclear. Since the polarity term already captures both the direction and strength of segment–outcome associations, a controlled polarity-only ablation would help clarify the source of the improvements.
  2. As discussed in Appendix A.1, the availability of DIG corrections depends on the support threshold and the topology of the sampled reasoning trees. However, the paper does not report the proportion of supported edges or tokens receiving nonzero DIG corrections, nor does it examine sensitivity to the support threshold. These analyses would help clarify the practical coverage and robustness of the proposed credit assignment mechanism.
Questions for authors
  1. What is the independent contribution of the normalized information-gain factor?
  2. How frequently does DIG provide effective credit during training, and how sensitive is it to the support threshold?
2 comments

We thank the reviewer for recognizing the motivation behind DIG and for identifying the two questions about its information factor and practical coverage.

Q1. Contribution of the information factor

A1. The matched polarity-only ablation shows a 3.59-point contribution from the information factor. On Qwen3-4B-Base, seed 42, we remove GG while retaining the smoothed polarity, DGPO tree-construction configuration, base trajectory advantage, global/local comparison sets, support masks, normalization, and mixture weights. Constant-outcome comparison sets retain zero raw scores.

Segment scoreSix-benchmark pass@1 (%)
Polarity only46.01
Full DIG: information factor × polarity49.60

Polarity already captures the direction and magnitude of the odds contrast. The information factor supplies a different weighting: the empirical outcome information carried by the occurrence split, normalized by segment-occurrence entropy. It therefore changes the relative credit across supported segments. This ablation isolates its contribution within the same credit-construction and normalization procedure.

Q2. Credit coverage and support threshold

A2. At the default threshold, 31.29% of token occurrences have a nonzero normalized global or local component, and 53.13% of trees contain at least one such component. We measured this on 768 actual training trees from Qwen3-4B-Base, seed 42, with 256 trees at each of steps 50, 250, and 450, using B=8B=8, D=3D=3, and τmin⁡=2\tau_{\min}=2.

Coverage at τmin⁡=2\tau_{\min}=2GlobalLocal
Supported edges32.46%30.48%
Tokens on supported edges58.24%54.52%
Tokens with a nonzero normalized component30.75%25.67%

Edge statistics count each distinct edge once per tree. Token statistics count occurrences across complete paths, so a shared segment is counted once for each path containing it. Global and local coverage overlap. Unsupported edges remain zero after normalization.

For threshold sensitivity, we recomputed coverage on the same recorded trees at thresholds 1, 2, and 4. The accuracy measurements come from separate training runs using the corresponding threshold:

τmin⁡\tau_{\min}Tokens with either component nonzero (%)Six-benchmark pass@1 (%)
156.1446.07
2 (default)31.2949.60
45.5148.39

At τmin⁡=1\tau_{\min}=1, comparisons based on a single path are admitted. Coverage increases, but accuracy falls; these comparisons are more sensitive to individual binary outcomes. At τmin⁡=4\tau_{\min}=4, the eight-path budget permits only 4-versus-4 comparisons. The coverage audit leaves few tokens with segment corrections, and the corresponding training run scores close to the trajectory-only control under DGPO branching (48.29). The default threshold retains more credit coverage than four while requiring at least two paths in each comparison group.

We will incorporate the polarity-only ablation, coverage statistics, and threshold analysis into the revised manuscript.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer gehj, thank you again for your thoughtful review and for recognizing the motivation behind DIG. In our response, we isolate the independent contribution of the normalized information-gain factor through a matched polarity-only ablation that removes GG while keeping the polarity, support masks, normalization, and global/local weights fixed; the information factor accounts for a 3.59-point gain. We also report how frequently DIG provides effective credit, measured on actual training trees across three training stages, and analyze sensitivity to the support threshold τmin⁡∈1,2,4\tau_{\min} \in {1, 2, 4}, where the default threshold achieves the best accuracy by balancing credit coverage against the reliability of each comparison. As the discussion period is nearing its end, we would appreciate it if you could take another look and let us know whether these additions address your concerns. We would be happy to clarify any remaining questions.

Reviewer_D56ROpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Directional Gain Policy Optimization (DGPO), which derives segment-level credit from binary final-answer rewards. It collects reasoning trees using uncertainty-triggered branching and semantic-similarity pruning. Directional Information Gain (DIG) combines normalized mutual information with a signed association measure, computed globally across the tree and locally among sibling continuations. Standardized DIG scores are added to trajectory-level advantages for policy optimization. Experiments on four backbones and six mathematical benchmarks show higher average accuracy than GRPO, TreePO, and TreeRL. Additional experiments compare credit estimators under the same tree-construction procedure and performance at similar training runtimes.

Critical strengths
  1. The comparisons address relevant alternatives and important confounds. The paper compares with TreePO and TreeRL, rather than only GRPO. Table 5 keeps the DGPO branching procedure fixed while changing the credit estimator, and Appendix B.3 compares performance at similar total training times on Qwen3-4B-Base. These are useful controls, and improvements in average accuracy appear across all four backbones.
  2. The credit rule is explicit and its interpretation is carefully bounded. DIG separates association strength from direction and combines global and sibling-conditioned information without process labels. Appendix C characterizes its relationship to direct advantages and provides conditional improvement results. The authors also distinguish association from causality and evaluation variability from training variability, which helps clarify what the evidence establishes.
Critical weaknesses
  1. The main novelty is the information-based weighting, but its specific benefit is not isolated. Monte Carlo estimation of intermediate values and segment advantages is already established in methods such as VinePPO and SPO, while tree-based credit is also studied in the included baselines. Appendix C further shows that unsmoothed local DIG is a nonnegative reweighting of the direct population advantage. The key question is therefore why this particular information weighting improves learning. Table 5 provides relevant evidence: DIG reaches 61.59 compared with 59.42 for direct segment advantages and 60.26 for TreeRL advantages under the same branching procedure. However, this compares complete credit designs, including the information factor, polarity, global/local combination, and normalization. It does not isolate the information factor itself, as the paper acknowledges. A return-difference baseline with matched support masks, global/local structure, normalization, and correction scales would make the contribution more convincing. At present, the added value of the defining DIG ingredient remains uncertain.

  2. The effective amount and stability of the segment-level evidence are unclear. Each tree retains at most eight complete paths, and a DIG comparison requires at least two paths both containing and not containing an edge. This can leave very small comparison groups and excludes singleton terminal edges. Appendix A correctly notes that a flat eight-leaf tree provides no supported DIG correction. However, the paper does not report how often useful tree structures occur, the distribution of branch sample counts, or the proportion of tokens actually receiving corrections. These statistics are important for understanding how much credit depends on only a few sampled outcomes.

Besides, the global/local coefficients are fixed at α=1\alpha=1 and β=0.5\beta=0.5, without reported sensitivity analysis, which might even weaken the whole experimental result,

Questions for authors

1.Information factor: Do any existing ablations isolate the mutual-information factor while keeping support masks, global/local combination, normalization, and correction scales matched to a direct return-difference baseline? What evidence supports this factor specifically?

2.Effective sample support: What fraction of training trees produce supported and nonzero DIG scores? Please report the branch-support distribution and the proportion of tokens receiving nonzero global and local corrections, ideally by training stage or backbone.

3.Sensitivity: Is any sensitivity evidence available for α\alpha, β\beta, and the eight-leaf budget? How frequently do the corrections reverse the sign of the base trajectory advantage in the existing runs?

4.Training stability: Are any independent training repeats available beyond the eight stochastic evaluation repeats? If not, please keep the interpretation of the significance tests explicitly limited to evaluation variability.

3 comments

Response to Reviewer D56R(1/2)

We thank the reviewer for the careful assessment and the specific suggestions for isolating the information factor.

Q1. Independent contribution of information weighting

A1. In the matched ablation, removing the information factor reduces pass@1 from 49.60 to 46.01. The full score also outperforms an unweighted return-difference control.

These experiments use Qwen3-4B-Base, seed 42, and report the equally weighted average pass@1 across all six benchmarks. Unlike the original four-benchmark Table 5 comparison, the four variants below share the DGPO tree-construction configuration, base advantage, global/local comparison sets, support masks with τmin⁡=2\tau_{\min}=2, supported-edge normalization in Eq. (12), and coefficients (α,β)=(1,0.5)(\alpha,\beta)=(1,0.5). Only the raw segment-scoring rule changes.

Segment scorepass@1 (%)
Aligned direct advantage, m1−mm_1-m48.22
Cubic weighting, sign⁡(δ)∣δ∣3\operatorname{sign}(\delta)\lvert\delta\rvert^348.78
Polarity only46.01
Full DIG49.60

To isolate GG specifically, the polarity-only control retains the same smoothed polarity and preserves zero raw scores for constant-outcome comparisons:

=\mathbf{1}\left\lbrace 0<\sum_{i\in\Omega}Y_i<|\Omega|\right\rbrace \tanh\left(\frac{\Delta(z;\Omega)}{2}\right).$$ The 3.59-point gain over this control directly tests the information factor. Polarity measures the odds contrast; $G$ weights it by the empirical outcome information carried by the segment-present/absent split, relative to the entropy of segment occurrence. The 1.38-point gain over aligned direct advantages separately tests the complete weighting against an unweighted return difference. Our contribution concerns this credit weighting; Monte Carlo intermediate-value estimation is established in VinePPO and SPO. ### **Q2. Effective credit coverage during training** **A2.** We analyzed 768 trees from the reported Qwen3-4B-Base training run, seed 42: 256 trees at each of steps 50, 250, and 450. These are the actual training-time trees. Overall, 98.31% contain a supported global or local edge, 56.38% contain supported nonzero raw DIG, and 53.13% have at least one nonzero normalized component. | Coverage (%) | Step 50 | Step 250 | Step 450 | Overall | | --- | ---: | ---: | ---: | ---: | | Trees: nonzero supported raw DIG | 55.86 | 51.95 | 61.33 | 56.38 | | Trees: nonzero normalized component | 52.73 | 49.22 | 57.42 | 53.13 | | Tokens: nonzero global component | 30.48 | 28.04 | 33.77 | 30.75 | | Tokens: nonzero local component | 25.05 | 23.68 | 28.31 | 25.67 | | Tokens: either component nonzero | 30.95 | 28.70 | 34.24 | 31.29 | Token percentages count occurrences across complete paths, as in Eq. (14): a shared segment is counted once for each path containing it. Global and local coverage overlap. Centering in Eq. (12) can give a supported edge with zero raw DIG a nonzero normalized component; unsupported edges remain zero. At the edge level, 67.10% have only one descendant leaf and fail the default support check. Global and local support cover 32.46% and 30.48% of distinct edges, respectively. Among supported comparisons, the smaller group's size is distributed as follows: | Paths in the smaller group | Global (%) | Local (%) | | --- | ---: | ---: | | 2 | 57.87 | 61.95 | | 3 | 18.74 | 16.70 | | 4 | 23.38 | 21.35 | DIG therefore supplies selective corrections to supported shared segments. Tokens on unsupported edges retain their base trajectory advantage.

Response to Reviewer D56R(2/2)

Q3. Sensitivity and advantage sign changes

A3. The benefit persists across the tested coefficient settings. On Qwen3-4B-Base, seed 42, all settings below exceed the trajectory-only baseline under DGPO branching (48.29):

α\alphaβ\betaSix-benchmark pass@1 (%)
10.549.60
10.2549.30
1148.91
0.50.549.04
20.548.65

For the completed-path budget, B=4,8,16B=4,8,16 yields 46.87, 49.60, and 50.48, respectively. Accuracy improves as more paths become available for training.

On the same recorded trees, we count a sign change when sign⁡(A^i,tDGPO)≠sign⁡(A^ibase)\operatorname{sign}(\widehat A^{\mathrm{DGPO}}_{i,t})\ne\operatorname{sign}(\widehat A_i^{\mathrm{base}}), restricting the denominator to token occurrences with nonzero base advantage. The rate is 9.60% overall, with 10.36%, 9.36%, and 9.12% at the three sampled stages. Restricting further to occurrences with a nonzero combined correction gives 17.62% overall. Thus, the corrections change the base-advantage sign for a minority of eligible token occurrences.

Q4. Independent training repeats

A4. We now have three independent training runs per method on both backbones, using seeds 42, 5, and 28. Each entry is mean ± sample standard deviation across the seed-level six-benchmark averages, with eight stochastic decoding passes per benchmark.

MethodQwen3-4B-BaseLlama-3.2-3B-Instruct
GRPO46.54 ± 0.3122.82 ± 0.88
TreeRL47.60 ± 0.4523.77 ± 0.73
DGPO branching + TreeRL advantage48.52 ± 0.9024.31 ± 0.76
DGPO49.66 ± 0.6624.68 ± 0.84

DGPO exceeds TreeRL in all three seeds on both backbones. Under DGPO branching, replacing TreeRL advantages with DIG improves the mean by 1.14 points on Qwen and 0.37 on Llama. The new repeats measure training variability; the original Table 4 tests remain specific to decoding variability.

We will add the matched ablations, training-log diagnostics, sensitivity results, and training-seed summaries to the revised manuscript.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer D56R, thank you again for your careful assessment and specific suggestions. In our response, we provide a matched ablation that isolates the information factor, holding the support masks, global/local combination, normalization, and coefficients fixed; full DIG outperforms the polarity-only control by 3.59 points and the unweighted return-difference control by 1.38 points. We also report training-time coverage statistics (branch-support distribution and the fractions of trees and tokens receiving nonzero global and local corrections across training stages), sensitivity results for α\alpha, β\beta, and the path budget, and the rate at which corrections reverse the base-advantage sign. Finally, we add three independent training seeds on both backbones and limit the interpretation of the original Table 4 tests to decoding variability, as you suggested. As the discussion period is nearing its end, we would appreciate it if you could take another look and let us know whether these additions address your concerns. We would be happy to clarify any remaining questions.

Reviewer_anbVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper uses Directional Information Gain to address issues for outcome-based RLVR rewards by building a reasoning tree per prompt which branches at high-entropy positions to guide credit assignment.

Critical strengths

This paper includes sensible ablations where the branching rule and the credit term are separated, credit designs are also compared, and the overhead is calculated at matched wall-clock.

Critical weaknesses
  1. Training-run variability is not quantified, and the margins over TreeRL are not tested against it.
  2. The proposed weighting is not shown to beat a simpler reweighting under a matched tree.
Questions for authors
  1. W1: Table 4's Wilcoxon tests over eight decoding passes are significant on qwen but not on llama, and they do not cover training-run variability. Please provide ≥ 3 seeds for DGPO vs TreeRL vs "DGPO branching + TreeRL advantage".
  2. W2: DIG is a TreeRL-style return difference times a nonnegative weight, and Table 5 shows the margin over TreeRL advantages is the only direct evidence. Please compare advantages, a simple nonlinear sign-preserving weight, and DIG, with the global and local terms removed separately.
2 comments

We thank the reviewer for recognizing the controlled ablations and wall-clock comparisons.

W1 / Q1. Training-run variability

A1. DGPO outperforms TreeRL in all three independent training seeds on both backbones. We trained with seeds 42, 5, and 28; the table reports mean ± sample standard deviation across these runs. Each run is evaluated by equally averaging pass@1 across six benchmarks, with eight stochastic decoding passes per benchmark.

MethodQwen3-4B-BaseLlama-3.2-3B-Instruct
GRPO46.54 ± 0.3122.82 ± 0.88
TreeRL47.60 ± 0.4523.77 ± 0.73
DGPO branching + TreeRL advantage48.52 ± 0.9024.31 ± 0.76
DGPO49.66 ± 0.6624.68 ± 0.84

Replacing TreeRL advantages with DIG under the same DGPO branching configuration improves the three-seed mean by 1.14 points on Qwen and 0.37 on Llama. These training repeats complement Table 4, whose tests concern decoding variability at fixed checkpoints.

W2 / Q2. Matched credit ablations

A2. The reviewer is correct that unsmoothed raw DIG can be written as a nonnegative weighting of the return difference, DIG=w(m1−m)\mathrm{DIG}=w(m_1-m). This preserves its direction but changes the relative credit assigned to different segments. We tested whether that weighting improves learning beyond direct advantages and a simpler nonlinear alternative.

All rows below use the same DGPO tree-construction configuration on Qwen3-4B-Base, seed 42. For the aligned direct, cubic, polarity-only, and full DIG variants, we also hold fixed the base trajectory advantage, global/local comparison sets, support masks, normalization, and coefficients (α,β)=(1,0.5)(\alpha,\beta)=(1,0.5); only the raw segment score changes. Results are six-benchmark average pass@1 (%), extending the four-benchmark comparison in Table 5.

Credit under DGPO branchingpass@1
GRPO trajectory advantage48.29
TreeRL advantage48.38
Aligned direct advantage48.22
Cubic weighting48.78
Polarity only46.01
Local DIG only48.82
Global DIG only49.17
Full DIG49.60

The aligned direct control uses m1−mm_1-m, where m1m_1 is the segment-present success rate and mm is the overall success rate in the comparison set. The cubic control uses sign⁡(δ)∣δ∣3\operatorname{sign}(\delta)|\delta|^3, with δ=m1−m0\delta=m_1-m_0 and m0m_0 the segment-absent success rate. It matches the leading cubic dependence of unsmoothed DIG near independence (Appendix C).

Full DIG improves over the aligned direct and cubic controls by 1.38 and 0.82 points, respectively. Removing global DIG reduces accuracy by 0.78 points; removing local DIG reduces it by 0.43. Thus, both the weighting and the two comparison levels contribute in this ablation: global credit uses tree-wide outcome context, while local credit compares alternatives under the same prefix.

We will incorporate the training-seed results and matched credit ablations into the revised manuscript.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer anbV, thank you again for your thoughtful review. In our response, we report three independent training seeds for DGPO, TreeRL, and the requested "DGPO branching + TreeRL advantage" variant on both backbones, where DGPO achieves the highest mean on both Qwen and Llama. We also provide the requested matched credit ablations, comparing direct advantages, a sign-preserving cubic weighting, and full DIG, with global and local DIG removed separately; full DIG outperforms the aligned direct and cubic controls by 1.38 and 0.82 points, and removing either term reduces accuracy. As the discussion period is nearing its end, we would appreciate it if you could take another look and let us know whether these additions address your concerns. We would be happy to clarify any remaining questions.

Comments on the paper

3 comments

Dear reviewers,

The authors have submitted their rebuttals. I would be grateful if you could take a moment to review them carefully and respond as appropriate. Thank you for your ongoing engagement and support.

Your AC.

AuthorsOpenReview ↗

Summary of Rebuttal(1/2)

Dear Area Chair and Reviewers,

We thank all reviewers for their constructive feedback. Reviewers recognized the motivation behind DIG (gehj), the controlled ablations, and the wall-clock comparisons (anbV). The reviews raised three shared concerns, which we addressed with new experiments summarized below.

1. Independent contribution of the information weighting (anbV W2, D56R Q1, gehj Q1)

We ran a matched ablation on Qwen3-4B-Base that holds fixed the DGPO tree construction, base advantage, global/local comparison sets, support masks, normalization, and coefficients; only the raw segment score changes (six-benchmark pass@1, %):

Segment scorepass@1
GRPO trajectory advantage48.29
TreeRL advantage48.38
Polarity only (information factor removed)46.01
Aligned direct advantage (unweighted return difference)48.22
Cubic sign-preserving weighting48.78
Local DIG only48.82
Global DIG only49.17
Full DIG49.60

Removing the information factor costs 3.59 points. Full DIG outperforms the unweighted and simple nonlinear controls by 1.38 and 0.82 points, and the global and local terms each contribute.

2. Training-run variability (anbV W1, D56R Q4)

We added three independent training seeds per method on both backbones, including the requested "DGPO branching + TreeRL advantage" variant.

MethodQwen3-4B-BaseLlama-3.2-3B-Instruct
GRPO46.54 ± 0.3122.82 ± 0.88
TreeRL47.60 ± 0.4523.77 ± 0.73
DGPO branching + TreeRL advantage48.52 ± 0.9024.31 ± 0.76
DGPO49.66 ± 0.6624.68 ± 0.84

DGPO outperforms TreeRL in all three seeds on both backbones (Qwen: 49.66 ± 0.66 vs. 47.60 ± 0.45; Llama: 24.68 ± 0.84 vs. 23.77 ± 0.73). As D56R suggested, we now limit the interpretation of the original Table 4 tests to decoding variability.

AuthorsOpenReview ↗

Summary of Rebuttal(2/2)

3. Credit coverage and sensitivity (D56R Q2–Q3, gehj Q2)

  • Coverage: On 768 actual training trees across three training stages, 53.13% of trees and 31.29% of token occurrences receive a nonzero DIG correction. Coverage stays stable across training, and unsupported edges retain their base advantage.
Coverage (%)Step 50Step 250Step 450Overall
Trees: nonzero supported raw DIG55.8651.9561.3356.38
Trees: nonzero normalized component52.7349.2257.4253.13
Tokens: nonzero global component30.4828.0433.7730.75
Tokens: nonzero local component25.0523.6828.3125.67
Tokens: either component nonzero30.9528.7034.2431.29
  • Support threshold: The default threshold (2) achieves the best accuracy. It balances coverage against the reliability of each comparison.
$ \tau_{\min} $Tokens with either component nonzero (%)Six-benchmark pass@1 (%)
156.1446.07
2 (default)31.2949.60
45.5148.39
  • Coefficients and budget: All tested (α, β) settings exceed the trajectory-only baseline (48.29), and accuracy increases with the path budget (46.87 / 49.60 / 50.48 for 4 / 8 / 16 paths).
$ \alpha $$ \beta $Six-benchmark pass@1 (%)
10.549.60
10.2549.30
1148.91
0.50.549.04
20.548.65
  • Sign changes: we count a sign change when $ \operatorname{sign}(\widehat A^{\mathrm{DGPO}}_{i,t})\ne\operatorname{sign}(\widehat A_i^{\mathrm{base}}) $, restricting the denominator to token occurrences with nonzero base advantage. The rate is 9.60% overall, with 10.36%, 9.36%, and 9.12% at the three sampled stages. Restricting further to occurrences with a nonzero combined correction gives 17.62% overall. Thus, the corrections change the base-advantage sign for a minority of eligible token occurrences.

We have not yet received follow-up responses from the reviewers. We believe the new experiments directly address each of the questions raised, and we would be grateful if these additions could be taken into account in the subsequent discussion.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.