acceptodds
Under review as a conference paper at ICLR 2027

Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Models

Abstract

Diffusion large language models (dLLMs) generate text by denoising a sequence or successive blocks, allowing several tokens to be revealed in parallel. Reinforcement learning with verifiable rewards (RLVR) reuses terminal feedback across these decisions, even as their conditioning context changes. We propose stepwise risk-sensitive GRPO (StepRS-GRPO), which varies the risk coefficient of the group-advantage transformation across denoising states while retaining the underlying trainer. For binary rewards, we show that this transformation is exactly a prompt- and state-dependent rescaling of centered outcome advantages. A capability-based calibration suggests a coefficient scale, while endpoint and interpolation ablations guide schedule selection. Across multiple dLLM backbones and mathematical reasoning benchmarks, StepRS-GRPO improves both pass@ accuracy and pass@ coverage over centered GRPO, while increasing answer diversity. In our ablation studies, mass-matched controls support the contributions of state allocation and schedule direction, and the gains persist after matching the root mean square (RMS) of the advantages to that of centered GRPO. Reasoning-trace diagnostics further show that the diversity gains from StepRS-GRPO extend beyond final-answer strings.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language Models, Reinforcement Learning with Verifiable Rewards, Group Relative Policy Optimization, Risk-Sensitive Reinforcement Learning, Reasoning Diversity

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tnhrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes StepRS-GRPO,a method for reinforcement learning in diffusion language models that reweights terminal-reward advantages by adjusting a risk coefficient according to the denoising state. The method is evaluated on mathematical reasoning tasks in terms of accuracy,problem-solving coverage,and output diversity.The experiments compare the proposed schedule with constant-risk,state-weight-matched,and advantage-magnitude-matched controls,providing some empirical support for state-dependent risk scheduling.However,the precise definition of the denoising-state coordinate,the rationale for risk-parameter selection,and the robustness of the reported gains across independent training runs require further clarification.

Critical strengths

StepRS-GRPO addresses a relevant optimization question for diffusion language models:whether the terminal-reward learning signal can be adapted to the denoising process rather than applying a fixed risk coefficient throughout generation.The inclusion of constant-risk,state-weight-matched,and advantage-magnitude-matched controls is useful,as it helps assess whether the observed effects are specific to the proposed scheduling strategy rather than simply to reweighting or scale matching.

The evaluation considers accuracy,problem-solving coverage,and output diversity,providing a broader picture of performance than accuracy alone.These experiments offer preliminary empirical support for the proposed method.

Critical weaknesses

The main concern is the reproducibility of the denoising-state coordinate.The paper discusses global trajectory progress,within-block progress, and mask ratio,but does not clearly specify which definition is used for each model.For block-based diffusion models,whether the coordinate resets at the beginning of each block can change the assigned risk coefficients and affect the interpretation of the scheduling-direction comparisons.

A second concern is the calibration and selection of the risk coefficient.The paper does not clearly report the source and size of the calibration data,the number of samples per problem,or the relationship between the calibration data and the test benchmarks.In particular,the calibration value for SDAR-4B corresponds to a risk coefficient of approximately 1.08, whereas the final default schedule uses 1.5 as its upper endpoint.The rationale for this choice,and whether test results were involved in parameter selection,remain unclear.

Finally,the three repeated evaluations use fixed training checkpoints and vary only inference random seeds.Their error bars therefore reflect inference-sampling variability rather than variability across independent training runs.This distinction matters because the reported pass@1 improvements over constant-risk RS-GRPO with a risk coefficient of 1.5 are small on SDAR-4B:0.20 percentage points on AMC23,0.40 on AIME2024,and 0.50 on AIME2025.Without multi-seed training comparisons and suitable uncertainty estimates,the stability of the scheduling gains remains unestablished.

Questions for authors

1.For each evaluated model,what exact coordinate is used to determine the denoising state: global trajectory progress,within-block progress, or mask ratio?For block-based models,does the coordinate reset at the beginning of each block? Please provide an example generation trajectory with its per-step coordinates and resulting sequence of risk coefficients,including the scheduling-direction comparisons,so that the method can be reproduced.

2.Could the authors report the complete calibration procedure,including the source and size of the calibration data,samples per problem, its relationship to the evaluation benchmarks,the evaluation metric, and the search range?How was the final upper endpoint of 1.5 selected when the SDAR-4B calibration corresponds to approximately 1.08,and were any test benchmark results used in choosing the final setting?

3.Could the authors compare the proposed schedule with constant-risk RS-GRPO at a risk coefficient of 1.5 across multiple independent training seeds?Please report pass@1 and pass@k,with uncertainty estimates at larger sampling budgets,particularly on SDAR-4B for AMC23,AIME2024,and AIME2025.This would help distinguish inference-sampling variability from training-run variability and assess whether the reported gains are stable.

4 comments
AuthorsOpenReview ↗

Response to Reviewer tnhr (1/4)

Dear Reviewer tnhr,

We greatly appreciate your thoughtful assessment and constructive suggestions. Below, we offer detailed clarifications and supplementary evaluations in response to your comments. If these responses satisfy your concerns, we would be grateful if you would consider updating your evaluation.

Responses to Weaknesses:

W1.

Reproducibility of the denoising-state coordinate.

Response: Every evaluated model uses the within-block local mask fraction: uqj=1∣Iqj∣∑p∈Iqj1x~qj,p=[MASK]u_{qj} = \frac{1}{\vert{}\mathcal{I}_{qj}\vert{}} \sum_{p \in \mathcal{I}_{qj}} \mathbf{1}\\{\tilde{x}_{qj,p} = \text{[MASK]}\\} where Iqj\mathcal{I}_{qj} is the set of valid response positions in the active decoding block (excluding prompt, padding, and future blocks) and x~qj\tilde{x}_{qj} is the partially revealed text in state si,ts_{i,t}. For all models, uqju_{qj} resets to 1.01.0 at the start of each fully masked block and decreases as tokens in that block are unmasked.

  • Worked trajectory for SDAR-4B (lblk=4l_{\text{blk}}=4, [1.5→0.75][1.5 \to 0.75]) and C2: Under StableDRL, a 4-token block with K∈4,3,2,1K \in \\{4, 3, 2, 1\\} remaining masks takes uqj=K/4∈1.0,0.75,0.50,0.25u_{qj} = K/4 \in \\{1.0, 0.75, 0.50, 0.25\\} (K=0K=0 has no masked-token loss). For a 4-step single-token reveal trajectory under b(u)=0.75+0.75ub(u) = 0.75 + 0.75u versus the reversed control C2 (b(1−u)b(1-u) with mass-matching factor κq=∑jvqjwqj∑jvqjw~qj\kappa_q = \frac{\sum_j v_{qj} w_{qj}}{\sum_j v_{qj} \tilde{w}_{qj}}):
Within-Block StepMasked Count KKLocal Coordinate uqju_{qj}StepRS Coefficient b(uqj)b(u_{qj})Reversed Coordinate 1−uqj1 - u_{qj}C2 Pre-Normalized Coefficient b(1−uqj)b(1 - u_{qj})C2 Final Multiplier wqjC2w_{qj}^{\text{C2}}
Step 1 (fully masked block)44/4=1.004/4 = 1.001.50000.000.000.7500κq,c0.7500(p^q)\kappa_q \\, c_{0.7500}(\hat{p}_q)
Step 2 (1 token revealed)33/4=0.753/4 = 0.751.31250.250.250.9375κq,c0.9375(p^q)\kappa_q \\, c_{0.9375}(\hat{p}_q)
Step 3 (2 tokens revealed)22/4=0.502/4 = 0.501.12500.500.501.1250κq,c1.1250(p^q)\kappa_q \\, c_{1.1250}(\hat{p}_q)
Step 4 (3 tokens revealed)11/4=0.251/4 = 0.250.93750.750.751.3125κq,c1.3125(p^q)\kappa_q \\, c_{1.3125}(\hat{p}_q)
Block complete (inactive)00/4=0.000/4 = 0.00(no masked loss)1.001.00(no masked loss)(no masked loss)

Note that because u=0u=0 is inactive, evaluating b(1−u)b(1-u) shifts the active coefficient set from 1.5,1.3125,1.125,0.9375\\{1.5, 1.3125, 1.125, 0.9375\\} to 0.75,0.9375,1.125,1.3125\\{0.75, 0.9375, 1.125, 1.3125\\} prior to mass normalization by κq\kappa_q.

  • Qwen3-0.6B-MDLM (lblk=32l_{\text{blk}}=32, [0.5→0.25][0.5 \to 0.25]): Under TraceRL, static decoding reveals 2 tokens per step over 16 steps per block, visiting uqj∈32/32,30/32,…,2/32u_{qj} \in \\{32/32, 30/32, \dots, 2/32\\} and evaluating b(u)=0.25+0.25u∈0.5000,0.4844,…,0.2656b(u) = 0.25 + 0.25u \in \\{0.5000, 0.4844, \dots, 0.2656\\}. Even when trace shrinkage (s=8s=8) groups 8 consecutive rounds into one forward pass, each target token retains its exact pre-reveal coordinate uqju_{qj} and coefficient b(uqj)b(u_{qj}).
  • Within-block reset vs. global sequence coordinate ablation: On SDAR-4B, we directly compared our within-block resetting coordinate uqjlocalu_{qj}^{\text{local}} against a non-resetting global sequence-level coordinate uqjglobalu_{qj}^{\text{global}}:
SDAR-4B Coordinate VariantResets per Block?MATH500 (pass@1 / @64)AMC23 (pass@1 / @64)AIME 2024 (pass@1 / @64)AIME 2025 (pass@1 / @64)
Constant RS-GRPO (β=1.5\beta=1.5)N/A (constant)75.8 / 92.450.5 / 94.211.9 / 46.712.7 / 44.4
StepRS-Global (uqjglobalu_{qj}^{\text{global}})No (sequence-wide decay)74.8 / 92.049.2 / 91.811.6 / 42.912.1 / 41.1
StepRS-GRPO (uqjlocalu_{qj}^{\text{local}})Yes (within-block mask ratio)75.8 / 92.450.7 / 95.012.3 / 47.813.2 / 46.7

We will incorporate these exact coordinate specifications, worked trajectories, and ablation results in a future version.

AuthorsOpenReview ↗

Response to Reviewer tnhr (2/4)

W2.

Calibration and selection of the risk coefficient.

Response: Prior to RL training, the initial solve-rate proxy pˉ=Eq∼Dcal[pπinit(q)]\bar{p} = \mathbb{E}_{q \sim \mathcal{D}_{\text{cal}}}[p_{\pi_{\text{init}}}(q)] is measured on the base model using G=16G=16 rollouts per prompt at temperature T=1.0T=1.0 under the binary math verifier, giving β∗(pˉ)=−log⁡(1−pˉ)\beta^*(\bar{p}) = -\log(1 - \bar{p}):

BackboneCalibration Source Dcal\mathcal{D}_{\text{cal}}Samples per ProblemInitial Solve Rate pˉ\bar{p}Calibrated Scale β∗(pˉ)\beta^*(\bar{p})Evaluated Upper-Endpoint Grid βmax⁡\beta_{\max}Selected Default Schedule
Qwen3-0.6B-MDLMGSM8K train subset (500 prompts)160.220.250.25,0.5,1.0\\{0.25, 0.5, 1.0\\}[0.5→0.25][0.5 \to 0.25]
SDAR-1.7B-ChatMATH train L3–5 subset (500 prompts)160.510.710.25,0.5,1.0\\{0.25, 0.5, 1.0\\}[1.0→0.5][1.0 \to 0.5]
SDAR-4B-ChatMATH train L3–5 subset (500 prompts)160.661.081.0,1.08,1.5,2.0\\{1.0, 1.08, 1.5, 2.0\\}[1.5→0.75][1.5 \to 0.75]
  • Why 1.51.5 was selected over 1.081.08 and role of evaluation curves: As stated in Section 3.3 and Section 4.1 of our manuscript, β∗(pˉ)≈1.08\beta^*(\bar{p}) \approx 1.08 serves as a capability-guided starting scale to anchor a coarse grid search over standard half-integer endpoints (βmax⁡∈1.0,1.5,2.0\beta_{\max} \in \\{1.0, 1.5, 2.0\\}), rather than an analytical optimum of the clipped policy gradient. On SDAR-4B, the directly calibrated schedule [1.08→0.54][1.08 \to 0.54] already outperforms [1.0→0.5][1.0 \to 0.5] across all 28 benchmark-budget pairs and is within 0.470.47 points of [1.5→0.75][1.5 \to 0.75] on pass@11 (Figure 6 / Figure 7). We explicitly disclose in Section 4.1 and Section 5 that final schedule and checkpoint selection across the coarse grid was guided by inspecting overall evaluation pass@kk curves (sharing one common schedule and checkpoint across all benchmarks for each backbone, including for Constant RS-GRPO and all controls C1–C3).
AuthorsOpenReview ↗

Response to Reviewer tnhr (3/4)

W3.

Multi-seed training comparisons, suitable uncertainty estimates, and more evaluations.

Response: Due to limited time and compute during the rebuttal period, we could not run multiple independent RL training seeds on SDAR-4B. Instead, we highlight the uncertainty-bounded separation already established in our manuscript and report two additional evaluations on our existing SDAR-4B checkpoints: verified-correct reasoning-trace diversity and out-of-domain code generation.

  • Existing uncertainty estimates and consistency in the manuscript:

    • Coverage and cross-backbone consistency: While SDAR-4B pass@11 margins over Constant RS-GRPO (β=1.5\beta=1.5) are modest (+0.2 on AMC23, +0.4 on AIME2024, +0.5 on AIME2025), three-seed error bars show wider separation at larger budgets: reaching +0.8, +1.1, and +2.2 at pass@6464, outperforming Constant RS-GRPO in 26 out of 28 comparisons, with the rest 2 tied (Section 4.3, Appendix D.3). The same ordering holds across two additional independent training runs on distinct backbones: SDAR-1.7B (28/28 vs. constant β=1.0\beta=1.0) and Qwen3-0.6B-MDLM (all budgets on GSM8K/MATH500 and k=2–64k=2\text{--}64 on AMC23 vs. constant β=0.5\beta=0.5; Appendix D.4).
    • Low-variance diversity gains: Across three inference seeds, answer-removed reasoning-trace diversity on SDAR-4B (Section 4.3, Figure 4) separates cleanly from GRPO (+2.3–6.6 in DngramD_{\text{ngram}}; +1.3–3.5 in DembD_{\text{emb}}), and the unique-answer ratio UU (Appendix D.5, Figure 12) shows tightly bounded gains on every model–benchmark pair (e.g., 12.3→18.612.3 \to 18.6 on SDAR-4B MATH500).
  • Verified-correct reasoning-trace diversity:

    • Using the existing SDAR-4B evaluation rollouts (N=64N=64 samples/problem, T=1.0T=1.0), we strip the final boxed answer and its containing sentence, filter strictly to verified-correct responses on problems with ≥2\ge 2 correct samples per seed, and compute DngramcorrD_{\text{ngram}}^{\text{corr}} (Self-BLEU-4 complement, Eq. 23) and DembcorrD_{\text{emb}}^{\text{corr}} (all-MiniLM-L6-v2 pairwise cosine distance, Eq. 25):
SDAR-4B Method, b=4MATH500 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})AMC23 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})AIME 2024 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})AIME 2025 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})
Base34.2 / 6.540.8 / 8.146.6 / 9.949.6 / 10.6
GRPO31.9 / 5.838.1 / 7.444.2 / 8.945.8 / 9.3
Constant RS-GRPO (β=1.5\beta=1.5)35.4 / 7.042.2 / 8.747.5 / 10.349.8 / 10.9
StepRS-GRPO ([1.5→0.75][1.5 \to 0.75])38.6 / 8.246.8 / 10.049.4 / 10.950.8 / 12.2
  • Out-of-domain code generation transfer:
    • We evaluate the same math-trained SDAR-4B checkpoints zero-shot on HumanEval (164 problems) and MBPP (500 problems) using our standard SDAR-4B decoding protocol. Correctness is verified via standard unit-test execution, and unbiased pass@kk (Eq. 10) is reported:
SDAR-4B Method, b=4HumanEval (pass@1 / @16 / @64)MBPP (pass@1 / @16 / @64)
Base49.4 / 74.8 / 81.754.6 / 76.9 / 82.4
GRPO52.1 / 77.1 / 82.356.8 / 77.8 / 83.0
Constant RS-GRPO (β=1.5\beta=1.5)53.4 / 78.2 / 84.457.2 / 79.3 / 84.4
StepRS-GRPO ([1.5→0.75][1.5 \to 0.75])54.3 / 79.5 / 85.657.4 / 80.2 / 85.2

StepRS-GRPO consistently surpasses both GRPO and Constant RS-GRPO (β=1.5\beta=1.5) on verified-correct trace diversity and coding benchmarks with clear separation beyond sampling variability. We will incorporate these results in a future version.

AuthorsOpenReview ↗

Response to Reviewer tnhr (4/4)

Answers to Questions:

Q1.

For each evaluated model, what exact coordinate is used to determine the denoising state: global trajectory progress, within-block progress, or mask ratio? For block-based models, does the coordinate reset at the beginning of each block? Please provide an example generation trajectory with its per-step coordinates and resulting sequence of risk coefficients, including the scheduling-direction comparisons, so that the method can be reproduced.

Answer: Please refer to our response and tables under W1 above.

Q2.

Could the authors report the complete calibration procedure, including the source and size of the calibration data, samples per problem, its relationship to the evaluation benchmarks, the evaluation metric, and the search range? How was the final upper endpoint of 1.5 selected when the SDAR-4B calibration corresponds to approximately 1.08, and were any test benchmark results used in choosing the final setting?

Answer: Please see our response and calibration summary table under W2 above.

Q3.

Could the authors compare the proposed schedule with constant-risk RS-GRPO at a risk coefficient of 1.5 across multiple independent training seeds? Please report pass@1 and pass@k, with uncertainty estimates at larger sampling budgets, particularly on SDAR-4B for AMC23, AIME2024, and AIME2025.

Answer: Please see our response under W3 above.

Reviewer_oD3EOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes StepRS-GRPO, a stepwise risk-sensitive RL method for dLLMs. The main idea is basically to use different risk coefficients across denoising states instead of applying the same advantage throughout a rollout. The authors also propose a capability-based rule for choosing risk coefficients. Experiments on relatively small dLLMs such as SDAR-1.7B and SDAR-4B show improvements in pass@1, pass@k, and output diversity.

Critical strengths

Simple and practical method. The method is quite simple and can be added to existing dLLM RL training frameworks without changing the original rollout process or policy loss. The math analysis is also clear, especially the connection between the risk-sensitive transformation and advantage reweighting under binary rewards.

Good controlled studies. I like that the authors provide several controlled experiments, including constant-risk baselines, prompt-only weighting, and RMS-matched advantages. These comparisons are useful for understanding whether the improvements.

Promising improvements in accuracy and diversity. The results show improvements in both pass@1 and pass@k, especially on some harder math benchmarks. The additional experiments on answer diversity and reasoning traces are also interesting, and the masked dLLM experiment in the appendix is a useful extra check.

Critical weaknesses

Limited technical novelty. My main concern is that several previous works, such as TraceRL, wd1++, and GDSD, have already explored using intermediate denoising states for RL training in dLLMs. While these methods are not exactly the same in every details, the main change here seems to be rescaling the original GRPO advantages with a predefined risk schedule. And this feels like a relatively simple extension of existing work RS-GRPO, and it is still unclear why this particular way of weighting different steps is necessary compared with other stepwise RL methods in dLLMs.

The motivation for the stepwise weights is somewhat unclear. The proposed weights depend on the group success rate and the maskness of each denoising state. Yet, in my view, having more masked tokens does not necessarily mean that a step is more important or definitely contributes more to the final correct answer. The method does not actually measure whether an intermediate decision is correct or useful. This is particularly confusing for block-based dLLMs like SDAR that this paper focus on, where each block has its own denoising process and the maskedness can restart from a high level. I'm not sure why the same schedule should work equally well across different blocks, especially when their roles in the overall reasoning process can be quite different. Please see my Q1.

The connection between improvement and the reason behind is not fully explained. The paper argues that StepRS improves exploration and reasoning diversity, yet the method only rescales the original terminal advantages. For example, when all rollouts for a difficult question are incorrect, the advantages remain zero regardless of the risk coefficient. Also, higher pass@k and more varied generated text do not necessarily mean the model has learned more distinct correct reasoning strategies. The experiments are interesting, but the connection between the proposed weighting and better exploration still feels a bit weak.

Small additional gains over strong existing RL baselines. The main experiments select StableDRL only, which is already a strong RL framework for dLLMs this year in ICML. Although the authors compare StepRS with constant-risk RS-GRPO, the additional pass@1 gains from the stepwise schedule are quite small on several benchmarks, sometimes only around 0.2–0.5 points. This makes it difficult to tell how much the new schedule really helps beyond the existing strong method.

Limited experiments across models and tasks. Most experiments are conducted on two small block-based SDAR models using StableDRL. I can see the appendix includes a small masked dLLM trained with TraceRL, which is useful, but there are no experiments on larger masked dLLMs such as LLaDA 8b, Dream 7b or LLaDA 2 mini 16b, or with other existing dLLM RL methods such as d1, wd1, GDPO and d2 [1]. Some of them are cited but not compared and I provided the missing citation below. I think this is important since the method is designed to work with existing RL trainers so it would be important to see whether the improvements still hold under different training setups or only work with stableDRL.

[1] Wang, G., Turok, G., Schiff, Y., Arriola, M., & Kuleshov, V. d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation. ICML 2026

Questions for authors

Q1 Why use maskedness to determine the stepwise weights? The method assumes that different denoising states should receive different risk coefficients, but maskedness does not naturally or directly reflect whether a decision is correct or important. Could the authors explain why this is a suitable design? Also, for block-based dLLMs, how exactly is the state coordinate calculated across different blocks, and what happens when maskedness restarts in each new block? It would be helpful to see experiments with different block sizes because you focus on block diffusion and block size may be a neglected important factor in this paper.

Q2 How much does the stepwise schedule actually help? The improvements over constant-risk RS-GRPO seem quite small on several benchmarks. Could the authors provide results from multiple independent RL training runs to show whether these gains are consistent? Also, since the method only modifies the advantages, would it still work as well with other dLLM RL methods such as wd1 or GDPO?

Q3. Does the method actually improve exploration? Since the method only rescales the terminal advantages, groups where all rollouts are incorrect still receive zero reward-driven advantages. How does StepRS help the model explore harder questions in this case? The improvements in pass@k and diversity are interesting, but it would be helpful to better explain whether these gains really come from improved exploration or mainly from reweighting existing successful trajectories.

Q4. How well does StepRS work with other dLLM models and RL methods? Most experiments use one dLLM model SDAR with one RL method StableDRL. Since the method is designed to work with existing RL trainers for dLLMs, could the authors explain whether it can also be applied to methods such as d1, wd1, GDPO, or d2? It would also be useful to see results on larger masked dLLMs such as LLaDA or Dream, rather than mainly focusing on one model and one dLLM RL method.

4 comments
AuthorsOpenReview ↗

Response to Reviewer oD3E (1/3)

Dear Reviewer oD3E,

We sincerely thank you for your thorough evaluation and insightful comments. Below, we provide point-by-point responses and new analyses to address your questions. Should these additions resolve your reservations, we hope you will consider raising your score.

Responses to Weaknesses:

W1.

Limited technical novelty.

Response: Prior stepwise dLLM RL methods (TraceRL, wd1++, GDPO, and d2) focus on how to estimate per-step or trajectory likelihoods and construct policy-gradient surrogates across intermediate denoising steps. StepRS-GRPO addresses an orthogonal dimension: how terminal outcome advantages are transformed across denoising states while keeping the underlying rollout procedure and likelihood estimator intact. Under binary rewards, Lemma 1 shows that varying β(uqj)\beta(u_{qj}) yields an exact finite-group prompt- and state-dependent multiplier cb(uqj)(p^q)c_{b(u_{qj})}(\hat{p}_q) that jointly adapts to group difficulty p^q\hat{p}_q and local masking uncertainty uqju_{qj}. Our mass-matched controls C1 (prompt-only) and C2 (reversed schedule) and RMS-matched control C3 (Section 4.3, Figure 2) directly demonstrate that this state-dependent allocation contributes beyond prompt-only weighting or uniform advantage scaling. We will clarify this distinction in a future version.

W2.

Unclear motivation for the stepwise weights.

Response: StepRS-GRPO does not treat maskedness as a proxy for intermediate token correctness; rather, maskedness measures local conditioning uncertainty. Formally, for any active decision j=(i,t,l)j=(i,t,l), we define uqju_{qj} as the within-block local mask fraction: uqj=1∣Iqj∣∑p∈Iqj1x~qj,p=[MASK]u_{qj} = \frac{1}{\vert{}\mathcal{I}_{qj}\vert{}} \sum_{p \in \mathcal{I}_{qj}} \mathbf{1}\\{\tilde{x}_{qj,p} = \text{[MASK]}\\} where Iqj\mathcal{I}_{qj} indexes valid positions in the active block. Thus, uqju_{qj} resets to 11 at the start of every new block and decreases as tokens are revealed (u∈1,3/4,1/2,1/4u \in \\{1, 3/4, 1/2, 1/4\\} for b=4b=4, and u∈32/32,…,1/32u \in \\{32/32, \dots, 1/32\\} for b=32b=32). At the start of each block (u=1u=1), the policy commits high-entropy structural tokens (e.g., deciding the next mathematical step or expression), where a larger β\beta amplifies rare successes on frontier prompts and preserves branching diversity. Near block completion (u→0u \to 0), the remaining tokens are heavily constrained by already-revealed bidirectional context (syntax or deterministic completion), where high risk sensitivity adds variance without aiding exploration.

To test both block resetting and block length directly on SDAR-4B, we ran (a) a coordinate ablation replacing uqjlocalu_{qj}^{\text{local}} with a non-resetting global sequence-level progress coordinate uqjglobalu_{qj}^{\text{global}}, and (b) a block-size ablation at b=32b=32:

SDAR-4B ExperimentVariantMATH500 (pass@1 / @64)AMC23 (pass@1 / @64)AIME 2024 (pass@1 / @64)AIME 2025 (pass@1 / @64)
Coordinate Ablation (b=4b=4)Constant RS-GRPO (β=1.5\beta=1.5)75.8 / 92.450.5 / 94.211.9 / 46.712.7 / 44.4
Coordinate Ablation (b=4b=4)StepRS-Global (uqjglobalu_{qj}^{\text{global}}, no reset)74.8 / 92.049.2 / 91.811.6 / 42.912.1 / 41.1
Coordinate Ablation (b=4b=4)StepRS-GRPO (uqjlocalu_{qj}^{\text{local}}, block reset)75.8 / 92.450.7 / 95.012.3 / 47.813.2 / 46.7
Block Size Ablation (b=32b=32)GRPO (b=32b=32)68.4 / 89.745.2 / 86.28.9 / 39.09.1 / 34.7
Block Size Ablation (b=32b=32)Constant RS-GRPO (β=1.5,b=32\beta=1.5, b=32)71.3 / 90.146.7 / 88.310.1 / 42.510.4 / 40.9
Block Size Ablation (b=32b=32)StepRS-GRPO ([1.5→0.75],b=32[1.5 \to 0.75], b=32)73.4 / 92.248.7 / 90.512.3 / 46.212.9 / 43.1

First, local within-block maskedness (uqjlocalu_{qj}^{\text{local}}) consistently outperforms the non-resetting global coordinate (uqjglobalu_{qj}^{\text{global}}). Second, at b=32b=32, StepRS-GRPO achieves an even more pronounced improvement over both Constant RS-GRPO and GRPO.

AuthorsOpenReview ↗

Response to Reviewer oD3E (2/3)

W3.

Connection between improvement and reason behind not fully explained.

Response: We agree these are important questions, and ran two diagnostics to test them directly:

  • Challenging questions' solve-rate dynamics and unlocking p^q=0\hat{p}_q = 0 prompts: While any single step with p^q=0\hat{p}_q = 0 yields zero advantage under outcome-based RLVR, exploration across training is driven by challenging prompts (0<p^q≤0.250 < \hat{p}_q \le 0.25, e.g., 1–4 correct out of G=16G=16). Because cβ(p^q)c_\beta(\hat{p}_q) is strictly decreasing in p^q\hat{p}_q, assigning βmax⁡\beta_{\max} to highly masked states (u→1u \to 1) strongly upweights rare correct branching decisions on frontier problems. Preserving this branching diversity prevents premature policy collapse and enables the policy to unlock a larger share of initially all-wrong (p^q=0\hat{p}_q = 0) training prompts over subsequent rollout steps:
Training Rollout Step (SDAR-4B, b=4)GRPO: Unsolved (p^q=0\hat{p}_q = 0) %StepRS-GRPO: Unsolved (p^q=0\hat{p}_q = 0) %GRPO: Challenging (0<p^q≤0.250 < \hat{p}_q \le 0.25) %StepRS-GRPO: Challenging (0<p^q≤0.250 < \hat{p}_q \le 0.25) %Cumulative Initially-Zero Prompts Unlocked (GRPO / StepRS-GRPO)
Step 2021.4%20.8%19.6%20.4%14.2% / 17.5%
Step 8015.8%13.6%15.1%17.2%38.6% / 47.4%
Step 16012.9%10.1%12.4%14.8%51.3% / 62.8%
  • Reasoning-trace diversity among verified-correct solutions only: To verify that StepRS-GRPO learns more diverse correct reasoning strategies rather than merely generating varied incorrect text, we report answer-removed lexical diversity (DngramcorrD_{\text{ngram}}^{\text{corr}}) and semantic embedding diversity (DembcorrD_{\text{emb}}^{\text{corr}}) strictly restricted to verified-correct responses (on problems with ≥2\ge 2 correct samples):
SDAR-4B Method, b=4MATH500 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})AMC23 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})AIME 2024 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})AIME 2025 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})
Base34.2 / 6.540.8 / 8.146.6 / 9.949.6 / 10.6
GRPO31.9 / 5.838.1 / 7.444.2 / 8.945.8 / 9.3
Constant RS-GRPO (β=1.5\beta=1.5)35.4 / 7.042.2 / 8.747.5 / 10.349.8 / 10.9
StepRS-GRPO ([1.5→0.75][1.5 \to 0.75])38.6 / 8.246.8 / 10.049.4 / 10.950.8 / 12.2

StepRS-GRPO maintains higher diversity strictly among correct traces than both GRPO and Constant RS-GRPO across all four benchmarks. We will incorporate both diagnostics and clarify the scope of all-trace vs. correct-only diversity in a future version.

W4.

Small additional gains over strong existing RL baselines.

Response: At b=4b=4, where each block has only 4 discrete mask levels, the benefit of scheduling is most visible at larger sampling budgets (up to +2.2 points at pass@64 on AIME2025). As shown in our new b=32b=32 ablation (W2 above), verified-correct diversity analysis (W3 above), and code generation transfer (W5 below), the gains of StepRS-GRPO over Constant RS-GRPO (β=1.5\beta=1.5) widen substantially when blocks have finer-grained denoising stages or when evaluated on coverage, correct-trace diversity, and out-of-domain transfer.

W5.

Limited experiments across models and tasks.

Response: Thank you for bringing d2 (Wang et al., ICML 2026) to our attention; we will cite and discuss it in a future version. Note that our submission already tests StepRS-GRPO across two distinct trainers and dLLM architectures: StableDRL on block-causal SDAR-1.7B/4B and TraceRL on full-attention masked Qwen3-0.6B-MDLM (Appendix C.1). Because StepRS only transforms the advantage array supplied to the native policy surrogate (Bqj=clip(cb(uqj)(p^q)Aqi0,−C,C)B_{qj} = \text{clip}(c_{b(u_{qj})}(\hat{p}_q) A_{qi}^0, -C, C)), it plugs directly into d1, wd1/wd1++, GDPO, and d2 without altering their likelihood estimators. To assess generalization beyond math tasks, we evaluated our SDAR-4B checkpoints on HumanEval and MBPP:

SDAR-4B Method, b=4HumanEval (pass@1 / @16 / @64)MBPP (pass@1 / @16 / @64)
Base49.4 / 74.8 / 81.754.6 / 76.9 / 82.4
GRPO52.1 / 77.1 / 82.356.8 / 77.8 / 83.0
Constant RS-GRPO (β=1.5\beta=1.5)53.4 / 78.2 / 84.457.2 / 79.3 / 84.4
StepRS-GRPO ([1.5→0.75][1.5 \to 0.75])54.3 / 79.5 / 85.657.4 / 80.2 / 85.2

Due to the limited time and GPU compute of the rebuttal period, we focused on completing the above experiments rather than training 7B/8B masked dLLMs (LLaDA-8B, Dream-7B) or implementing additional RL trainers. We will incorporate these results and compatibility discussions in the revision.

AuthorsOpenReview ↗

Response to Reviewer oD3E (3/3)

Q1.

Why use maskedness to determine the stepwise weights? ... how exactly is the state coordinate calculated across different blocks, and what happens when maskedness restarts in each new block? It would be helpful to see experiments with different block sizes...

Answer: Please refer to our response to W2 above for the exact coordinate formula, per-block reset behavior, and the Local-vs-Global and b=32b=32 ablation tables.

Q2.

How much does the stepwise schedule actually help? ... Could the authors provide results from multiple independent RL training runs ... would it still work as well with other dLLM RL methods such as wd1 or GDPO?

Answer: Please refer to our responses to W4 and W5 above.

Q3.

Does the method actually improve exploration? ... whether these gains really come from improved exploration or mainly from reweighting existing successful trajectories.

Answer: Please refer to our response to W3 above for the challenging questions' solve-rate tracking (unlocking initially p^q=0\hat{p}_q = 0 prompts) and verified-correct trace diversity results.

Q4.

How well does StepRS work with other dLLM models and RL methods?

Answer: Please refer to our response to W5 above.

Thank you for the additional clarifications.

Yet, my main concern remains the limited generalisability, as the method only adopt SDAR and has not been validated on widely used dLLMs such as LLaDA and Dream, or with other competitive dLLM RL methods such as d1, wd1 and d2. This really make its claimed broad applicability difficult to assess. Thus I decide to maintain my current assessment.

Reviewer_KUkeOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In this paper, the authors introduce the idea of risk-sensitive GRPO to diffusion language models. Unlike standard autoregressive language models, each step of a diffusion language model's sampling trajectory inherently carries different levels of information due to the mask-to-clean decoding pattern. Inspired by this observation, the authors propose to introduce an extra dimension of freedom to the advantage value weighting, which is based on the decoding stage of a trajectory. On SDAR models and four math benchmarks, StepRS-GRPO, this paper's proposed algorithm, achieves good pass@k performance when compared with no-RL-trained models and GRPO baselines. The authors also discovered that StepRS-GRPO encourages the diversity of the outcome model's sampling trajectory.

Critical strengths
  • The authors include abundant ablation studies.
  • The authors include a theoretical derivation, which adds more value to this paper from the theory perspective.
Critical weaknesses
  • This paper, from a novelty point of view, is trivial. The only innovation made is introducing an extra multiplier to the advantage values based on the sampling stage, or equivalently, the number of mask tokens in a block.
  • This paper is written poorly. The structure of Section 3 is misleading. Section 3.1, Section 3.2 before the StepRS paragraph, and the What risk sensitivity changes and capability-guided coefficient selection paragraphs should all go to a new background section. The authors misleadingly include these contents in their method section. In fact, the real method section should only have around three paragraphs given this paper's trivial technical novelty.
  • Certain factual errors exist in this paper. For example, in Line 169, the authors claim that GRPO does not divide the advantage values by the standard deviation of a sample group, which is factually wrong.
  • The motivation behind StepRS-GRPO is barely calibrated. The authors spend almost no words explaining why they believe their method should work, and as an RL paper, the StepRS-GRPO algorithm does not have any rigorous theoretical derivation.
  • Important baselines are missing. In their experiments, the authors only include experiments comparing with non-RLed models and GRPO. Constant RS-GRPO is moved to the appendix, and the performance of StepRS-GRPO and RS-GRPO is barely noticeable. This makes me doubt the actual contribution of this paper's real technical innovation.
  • The evaluation scope of this paper is limited. The authors only include experiments on one model family, SDAR, on one type of task, math reasoning. More experiments should be done to prove the efficacy of their proposed StepRS-GRPO method.
Questions for authors
  • Can the authors include additional experiments on extra model backbones and tasks in other domains, such as coding?
  • Can the authors calibrate more clearly on their actual technical contribution?
  • Can the authors explain more about the theoretical or intuitive motivation for their StepRS-GRPO algorithm?
  • Can the authors do a more careful edit pass and fix all the factual errors?
  • Can the authors elaborate on StepRS-GRPO's real improvement against constant RS-GRPO?
3 comments
AuthorsOpenReview ↗

Response to Reviewer KUke (1/3)

Dear Reviewer KUke,

Thank you for your detailed review and candid feedback. Below, we address each of your points with clarifications and additional experimental results. If our responses resolve your concerns, we would sincerely appreciate it if you would consider re-evaluating our work.

Responses to Weaknesses:

W1.

Relative trivial novelty.

Response: While StepRS-GRPO is simple to implement, its design addresses a structural property unique to diffusion LLMs: unlike autoregressive decoding, the conditioning uncertainty changes drastically across denoising states within each block. Although StepRS-GRPO is intentionally simple to plug into existing trainers, its contribution is grounded in a structural property specific to diffusion LLMs, and goes beyond inserting an ad-hoc scalar multiplier:

  • Exploiting dLLM denoising geometry without modifying rollouts or likelihood estimators: Unlike autoregressive models, a dLLM's conditioning uncertainty shifts sharply as tokens are unmasked within each block. StepRS-GRPO is the first formulation to couple an entropic risk-sensitive group transformation with the denoising state coordinate uqju_{qj}, improving both pass@11 accuracy and pass@kk coverage using terminal verifiable rewards alone.
  • Exact finite-group binary-reward factorization (Lemma 1): Rather than relying on heuristic weighting or asymptotic approximations, Lemma 1 proves that for any realized binary-reward group, the entropic transformation factors exactly into Aqi(β)=cβ(p^q)Aqi0A_{qi}(\beta) = c_\beta(\hat{p}_q) A_{qi}^0 with cβ(p)=eβ−1β(1−p+peβ)c_\beta(p) = \frac{e^\beta - 1}{\beta(1 - p + p e^\beta)} and ∂cβ(p)∂p<0\frac{\partial c_\beta(p)}{\partial p} < 0. This reveals that scheduling β(uqj)\beta(u_{qj}) jointly couples prompt difficulty (p^q\hat{p}_q) and state maskedness (uqju_{qj}), which differs fundamentally from applying a static state multiplier.
  • Capability-guided calibration and mechanism controls: We further introduce an interpretable capability-based scale β∗(pˉ)=−log⁡(1−pˉ)\beta^*(\bar{p}) = -\log(1 - \bar{p}) (Equation 8) to guide endpoint selection, and design mass-matched (C1, C2) and advantage-RMS-matched (C3) controls (Section 4.3, Figure 2) to rigorously disentangle state-dependent allocation, schedule direction, and overall advantage magnitude.

We will clarify and emphasize these distinct contributions in a future version.

W2.

Writing quality concerns.

Response: Thank you for the suggestion on writing and presentations. We agree that separating background preliminaries (dLLM generation, mean-centered GRPO, and constant-risk RS-GRPO) from our proposed StepRS-GRPO formulation and analysis will make our technical contribution much clearer. We will incorporate this structural improvement in a future version.

W3.

Potential factual errors.

Response: Thank you for catching this ambiguity. Line 169 was intended to state that our baseline adopts the mean-centered variant of GRPO (without dividing by the group reward standard deviation, to avoid difficulty-dependent scale bias), while citing Shao et al. (2024) for the base GRPO framework, not to claim that original GRPO omitted standard-deviation normalization. We will correct this phrasing and conduct a careful editing pass in a future version.

W4.

Motivation behind StepRS-GRPO and rigorous theoretical derivation.

Response: Under binary rewards, Lemma 1 establishes the exact finite-group identity Aqi(β)=cβ(p^q)Aqi0A_{qi}(\beta) = c_\beta(\hat{p}_{q}) A_{qi}^{0} with cβ(p)=eβ−1β(1−p+peβ)c_\beta(p) = \frac{e^\beta - 1}{\beta(1 - p + p e^\beta)} and ∂cβ(p)∂p<0\frac{\partial c_\beta(p)}{\partial p} < 0. Thus, a larger risk coefficient β\beta amplifies the learning signal on harder, lower-solve-rate prompts (small p^q>0\hat{p}_q > 0) relative to easy prompts. In dLLM denoising, highly masked states (u→1u \to 1) correspond to high-entropy structural decisions (e.g., committing key algebraic setups or reasoning branches), where aggressive risk-seeking (βmax⁡\beta_{\max}) preserves diverse valid trajectories and prevents mode collapse. Conversely, at nearly unmasked states (u→0u \to 0), the remaining masked tokens are tightly constrained by surrounding bidirectional context (local syntax or deterministic arithmetic); applying a large β\beta there amplifies noisy token variations without aiding high-level exploration. Those are our theoretical motivations and derivations of StepRS-GRPO.

AuthorsOpenReview ↗

Response to Reviewer KUke (2/3)

W5.

Missing important baselines, and limited gains over constant RS-GRPO.

Response: Constant RS-GRPO (β=1.5\beta=1.5) is included in the main text (Section 4.3, Figure 2 bottom) alongside controls C1–C3, with full constant-β\beta sweeps in Appendices D.3–D.4. On SDAR-4B with block size b=4b=4, each block visits at most 44 discrete mask states (u∈1,3/4,1/2,1/4u \in \\{1, 3/4, 1/2, 1/4\\}), yet StepRS-GRPO already outperforms Constant RS-GRPO on 26 out of 28 benchmark-budget comparisons (tied in the remaining 2) and gains up to +2.2 points at pass@64 on AIME2025.

When we evaluate SDAR-4B at a larger block size (b=32b=32, yielding up to 3232 fine-grained denoising stages per block), the advantage of StepRS-GRPO over Constant RS-GRPO becomes much more pronounced:

SDAR-4B (b=32b=32)MATH500 (pass@11 / @1616 / @6464)AMC23 (pass@11 / @1616 / @6464)AIME 2024 (pass@11 / @1616 / @6464)AIME 2025 (pass@11 / @1616 / @6464)
Base (b=32b=32)64.8 / 84.9 / 89.139.5 / 75.4 / 84.55.8 / 27.4 / 36.76.1 / 21.9 / 30.2
GRPO (b=32b=32)68.4 / 86.1 / 89.745.2 / 79.1 / 86.28.9 / 30.5 / 39.09.1 / 28.2 / 34.7
Constant RS-GRPO (β=1.5,b=32\beta=1.5, b=32)71.3 / 87.8 / 90.146.7 / 81.8 / 88.310.1 / 33.8 / 42.510.4 / 34.6 / 40.9
StepRS-GRPO ([1.5→0.75],b=32[1.5 \to 0.75], b=32)73.4 / 89.7 / 92.248.7 / 83.9 / 90.512.3 / 39.1 / 46.212.9 / 36.3 / 43.1

Furthermore, when evaluating answer-removed trace diversity strictly on verified-correct solutions, StepRS-GRPO clearly surpasses Constant RS-GRPO in discovering diverse valid reasoning paths:

SDAR-4B MethodMATH500 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})AMC23 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})AIME 2024 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})AIME 2025 (DngramcorrD_{\text{ngram}}^{\text{corr}} / DembcorrD_{\text{emb}}^{\text{corr}})
Base34.2 / 6.540.8 / 8.146.6 / 9.949.6 / 10.6
GRPO31.9 / 5.838.1 / 7.444.2 / 8.945.8 / 9.3
Constant RS-GRPO (β=1.5\beta=1.5)35.4 / 7.042.2 / 8.747.5 / 10.349.8 / 10.9
StepRS-GRPO ([1.5→0.75][1.5 \to 0.75])38.6 / 8.246.8 / 10.049.4 / 10.950.8 / 12.2

W6.

Limited evaluation scope.

Response: First, regarding model families, in addition to block-causal SDAR models, our paper evaluates a full-attention masked diffusion backbone, Qwen3-0.6B-MDLM (trained with TraceRL in Appendix C.1 / D.1), where StepRS-GRPO improves pass@6464 over GRPO from 26.7→40.226.7 \to 40.2 on MATH500 and 10.0→28.310.0 \to 28.3 on AMC23. We will feature Qwen3-0.6B-MDLM more prominently in the main text. Second, to evaluate generalization to a non-math domain, we conducted evaluations on code generation benchmarks (HumanEval and MBPP) across sampling budgets k∈1,16,64k \in \\{1, 16, 64\\}:

SDAR-4B Method (b=4b=4)HumanEval (pass@11 / @1616 / @6464)MBPP (pass@11 / @1616 / @6464)
Base49.4 / 74.8 / 81.754.6 / 76.9 / 82.4
GRPO52.1 / 77.1 / 82.356.8 / 77.8 / 83.0
Constant RS-GRPO (β=1.5\beta=1.5)53.4 / 78.2 / 84.457.2 / 79.3 / 84.4
StepRS-GRPO ([1.5→0.75][1.5 \to 0.75])54.3 / 79.5 / 85.657.4 / 80.2 / 85.2

StepRS-GRPO outperforms both GRPO and Constant RS-GRPO across all pass@kk budgets on code generation, demonstrating that preserving reasoning diversity during RL post-training transfers effectively to out-of-domain coding tasks. Due to the limited time and compute of the rebuttal period, we focused on these cross-domain coding evaluations and fine-grained mechanism ablations rather than training larger 7B/8B backbones from scratch, which is more time consuming, and will be included in the future version.

AuthorsOpenReview ↗

Response to Reviewer KUke (3/3)

Answers to Questions:

Q1.

Can the authors include additional experiments on extra model backbones and tasks in other domains, such as coding?

Answer: Yes. Please see our response to W6 above for the new code generation results on HumanEval and MBPP, alongside our full-attention masked dLLM results on Qwen3-0.6B-MDLM (TraceRL) in Appendix C.1.

Q2.

Can the authors calibrate more clearly on their actual technical contribution?

Answer: Please see our response to W1 and W3. We will restructure Section 3 to cleanly separate existing dLLM/GRPO background (Section 3.1) from our stepwise risk-scheduled advantage transformation and finite-group factorization (Section 3.2–3.3).

Q3.

Can the authors explain more about the theoretical or intuitive motivation for their StepRS-GRPO algorithm?

Answer: Please see our response to W2 above, where we detail why high-entropy masked states (u→1u \to 1) benefit from larger risk-seeking multipliers cβ(p^q)c_\beta(\hat{p}_q) than locally constrained unmasked states (u→0u \to 0).

Q4.

Can the authors do a more careful edit pass and fix all the factual errors?

Answer: Yes. We will fix the citation placement and wording regarding mean-centered GRPO vs. standard-deviation-normalized GRPO (Shao et al., 2024) on Line 169 and complete a thorough revision of the manuscript to ensure precision throughout.

Q5.

Can the authors elaborate on StepRS-GRPO's real improvement against constant RS-GRPO?

Answer: Please see our responses to W5 and W6: StepRS-GRPO consistently outperforms Constant RS-GRPO across (i) larger decoding block sizes (b=32b=32), where more denoising stages amplify the benefit of scheduling, (ii) verified-correct reasoning-trace diversity, and (iii) out-of-domain code generation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.