acceptodds
Under review as a conference paper at ICLR 2027

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation

Abstract

We introduce Nemotron-Cascade 2, an open 30B MoE model with 3B activated parameters that delivers best-in-class reasoning and strong agentic capabilities. Despite its compact size, its mathematical and coding reasoning performance approaches that of frontier open models. It is the second open-weight LLM, after DeepSeek-V3.2-Speciale-671B-A37B, to achieve Gold Medal-level performance in the 2025 International Mathematical Olympiad (IMO), the International Olympiad in Informatics (IOI), and the ICPC World Finals, demonstrating remarkably high intelligence density with 20× fewer parameters. In contrast to Nemotron-Cascade 1 (Wang et al., 2026), the key technical advancements are as follows. After SFT on a more meticulously curated dataset, we substantially scale Cascade RL to cover a broader spectrum of reasoning and agentic domains, grouping compatible domains with similar response lengths and verification costs. Furthermore, we incorporate multi-domain on-policy distillation from the strongest intermediate checkpoints in each domain to efficiently recover from potential benchmark regressions and sustain performance gains throughout the Cascade RL process. We release model checkpoint and training data for research community.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Post-Training, LLMs, RL, On-Policy Distillation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LzQSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents Nemotron-Cascade 2, a 30B MoE model with 3B active parameters post-trained from Nemotron-3-Nano-Base. It extends Cascade RL with a multi-domain RL stage that jointly trains domains with similar response lengths and verification costs, and adds multi-domain on-policy distillation (MOPD) from intermediate checkpoints to recover regressions. The model reports strong results on code reasoning, alignment, and instruction following, reaches gold-level scores on IMO 2025, IOI 2025, and ICPC World Finals 2025 with test-time scaling, and is released with its training data.

Critical strengths
  1. The work provides a strong and openly available research artifact. A 3B-active model reaching 87.2 on LiveCodeBench v6 and 82.9 on IFBench, released with its SFT and RL data, is valuable to the community.
  2. The work provides a transparent stage-wise analysis of model performance. Table 1 tracks benchmark performance after each stage, while the math length-budget control (Fig. 2b) and grader calibration on IMO-ProofBench provide careful validation.
Critical weaknesses
  1. The SFT checkpoint already reaches 92.9 on AIME25, above Nemotron-3-Nano (89.1) and the final model (92.4) (Table 1). Because the SFT data also differ from Nano's, this comparison does not isolate the RL pipeline as Section 4 suggests. RL appears to contribute mainly to instruction following, alignment, long-context, and agentic tasks.
  2. Multi-domain RL is not compared with sequential or fully joint training, and MOPD is not compared with simpler recovery options such as checkpoint merging. The "6× fewer steps" comparison (Fig. 2c) also leaves out the cost of training the RLHF teacher.
  3. The medal results rely on large test-time scaling (e.g., 8,192 samples per round for IMO and up to 1,000 solutions per problem for ICPC), which the abstract does not mention. IMO P2 was scored by an LLM grader, and the gold-level total depends on that score.
  4. τ²-Bench and HLE use non-official protocols worth several points, while baselines use official numbers. Some details differ between sections, e.g., RLHF steps (30 vs. 25), MOPD learning rate (2e-6 vs. 3e-6), and KL usage. The 13K GPU-hour total also excludes SWE RL.
Questions for authors
  1. How does the model perform if the Cascade-1 recipe, or the pipeline without the multi-domain RL stage, is run from the same SFT checkpoint?
  2. Could the authors report the inference budget and official submission counts for each competition, explain how ICPC penalty time was computed, and provide human verification for IMO P2?
  3. Could the baselines be evaluated under the same τ²-Bench and HLE settings? Which benchmarks were used for decontamination, and were IMO 2025 problems filtered from the AoPS proof data?
2 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

Thank you for those valuable comments and we would like address your concerns as below:

Because the SFT data also differ from Nano's, this comparison does not isolate the RL pipeline as Section 4 suggests. RL appears to contribute mainly to instruction following, alignment, long-context, and agentic tasks.

We would like to clarify that the primary goal of this paper is neither to claim superior performance on every benchmark nor to show that our model outperforms all other open-source models. Rather, we propose an efficient and effective training pipeline for enhancing multi-domain model capabilities, supported by comprehensive ablations of each training component. Notably, our RL approach yields the largest gains on reasoning tasks, where our model substantially outperforms models of comparable size and even approaches open-source models with roughly 10× more parameters.

Multi-domain RL is not compared with sequential or fully joint training, and MOPD is not compared with simpler recovery options such as checkpoint merging. The "6× fewer steps" comparison (Fig. 2c) also leaves out the cost of training the RLHF teacher.

Our multi-domain RL is fully joint RL training over multiple tasks from different domains that share similar characteristics.

We did not run a separate sequential comparison for this stage for two reasons. First, multi-domain RL is designed as a lightweight complement to the main reasoning stages (math, code, RLHF, IF, and SWE). It covers short-response capabilities that share the same length budget and similar verification cost, and it runs for only about 70 steps. For such compatible domains, the main motivation for grouping is training efficiency, and the blend gives consistent gains on MMLU-Pro, τ²-Bench, and IFBench. Second, our goal is to scale Cascade RL to more domains, and giving every new domain its own sequential stage would make the pipeline grow linearly with the number of environments. The more relevant question is therefore which domains can be safely grouped together. Here we did observe a clear boundary: adding math data to multi-domain RL lowers math accuracy, even with a longer math length budget (Fig. 2b), so we remove math from this stage.

Regarding the comparison between MOPD and checkpoint merging: we conducted preliminary experiments with checkpoint merging and found that it significantly degraded performance on reasoning benchmarks, so we dropped this approach early in development. Specifically, we used weight averaging to combine the math, code, RLHF, and IF teacher models. Compared with the individual teachers, AIME2025 dropped by 3.2%, LiveCodeBench v6 by 16.5%, and IFEval by 9.2%. We found weight merging difficult to tune, and its outcome hard to predict when combining capabilities from different domains, so we did not pursue it further.

Regarding the RLHF teacher cost in Fig. 2c: this is a good point, and we will clarify it in the revision. The main advantage of combining Cascade RL with MOPD is that checkpoints trained while exploring different Cascade RL paths are no longer wasted. Initially, we experimented with two branches, and we were then able to reuse a previously trained checkpoint as the teacher to fix certain aspects of the other path. This is what yields the 6× reduction in steps. We acknowledge, however, that this saving also depends on the cost of training the RLHF teacher, and we will state this explicitly.

The medal results rely on large test-time scaling (e.g., 8,192 samples per round for IMO and up to 1,000 solutions per problem for ICPC), which the abstract does not mention.

We agree that the abstract should state explicitly that our medal-level results rely on test-time scaling, and we will revise it accordingly. We note that this practice is consistent with other open models reporting medal-level results; for example, DeepSeek-V3.2-Speciale samples 500 solutions per problem for IOI and uses a similar generate–verify–refine pipeline for IMO to achieve gold medals. Specifically, we estimated the compute required to achieve gold-medal performance on IMO and IOI: DeepSeek-V3.2-Speciale requires approximately 1,500 and 291 H100 GPU hours, respectively, whereas our pipeline requires only 800 and 174 H100 GPU hours, making it substantially more efficient.

τ²-Bench and HLE use non-official protocols worth several points, while baselines use official numbers. Could the baselines be evaluated under the same τ²-Bench and HLE settings?

We apologize for the confusion. We adopt a non-official evaluation protocol in order to match the format of our training data. To ensure a fair comparison, we re-evaluated our baseline model, Nemotron-Nano-v3, under the same updated setting: its τ²-Bench score improved from 49.0 to 54.1, and its HLE score from 10.6 to 11.4. Both remain lower than our model’s performance under the same evaluation protocol. We will update these numbers and clarify the evaluation setting in the revision.

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

Some details differ between sections, e.g., RLHF steps (30 vs. 25), MOPD learning rate (2e-6 vs. 3e-6), and KL usage. The 13K GPU-hour total also excludes SWE RL.

Sorry for the confusion .The RLHF stage runs for 25 steps, and MOPD uses the learning rate as 2e-6 without the KL penalty. We also thank the reviewer for noting the missing SWE-RL cost: this stage requires approximately 564 H100 GPU hours. We will double-check all training details and costs and update them in the revision.

How does the model perform if the Cascade-1 recipe, or the pipeline without the multi-domain RL stage, is run from the same SFT checkpoint?

Without multi-domain RL, capabilities on certain tasks, such as STEM MCQA, agentic tool calling in the Workplace Assistant setup, and structured output, would be compromised. We found that the improvement from multi-domain RL is mainly on τ²-Bench (51 to 57), but it hurts math performance. Therefore, if the model is trained with the Cascade-1 recipe, without multi-domain RL and MOPD, we expect its basic performance to be maintained, while its τ²-Bench score would likely be lower. The other main difference from the Cascade-1 recipe is the Math RL stage, which we removed in Cascade-2. Starting from the SFT checkpoint, which already reaches 92.9 on AIME25, we found that Math RL consistently failed to improve AIME25 further, so we dropped this stage from the pipeline.

Could the authors report the inference budget and official submission counts for each competition, explain how ICPC penalty time was computed, and provide human verification for IMO P2?

We thank the reviewer for pointing this out. The inference budgets for our IMO and IOI test-time scaling pipelines are ~800 H100 GPU hours and ~174 H100 GPU hours, respectively. For IMO, we submit one final solution per problem; for IOI, we submit at most 50 solutions per problem, strictly following the official competition rules. Regarding ICPC, we do not explicitly account for penalty time, because the 4th-ranked human team solved only 9 problems. Since teams are ranked first by the number of problems solved, with penalty time used only to break ties, solving 10 problems is sufficient to secure a gold medal that year regardless of penalty.

Regarding IMO 2025 Problem 2, our model’s solution relies on computation-heavy analytic methods rather than a pure-geometry solution. While such proofs are valid, they are tedious to verify by humans. To assess correctness, we submitted the proof and every formula to multiple frontier LLMs (GPT, Claude etc.) for independent verification over several trials, and it was judged correct in all cases. We will also release our full IMO 2025 proofs for verification by the community.

Which benchmarks were used for decontamination, and were IMO 2025 problems filtered from the AoPS proof data?

We thank the reviewer for raising this point. We performed decontamination checks between our training data and all reported benchmarks, covering knowledge, chat, and reasoning tasks etc.. Specifically, we applied strict filtering based on n-gram overlap detection to remove potentially contaminated samples. For competitive programming, our training data was collected before August 2024, which excludes all possible contamination. For competitive math, our proof training data is strictly separate from IMO problems. We will clarify these details in the revision.

Reviewer_pcUTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Nemotron-Cascade 2, a 30B MoE model with 3B activated parameters. It extends Cascade RL by jointly training compatible domains and using multi-domain on-policy distillation (MOPD) to consolidate strengths from selected teacher checkpoints. Stage-wise evaluations and ablations show that MOPD recovers degraded capabilities and improves subsequent RLHF. The final model achieves strong mathematical and coding performance, including reported gold-medal-level results on IMO, IOI, and ICPC with substantial test-time scaling.

Critical strengths
  1. Strong results reporting competition-level reasoning at a relatively compact model scale. Achieving a gold-medal-level score of 35/42 on IMO 2025 with a model containing 30B total parameters and only 3B activated parameters is a particularly compelling empirical result. The gold-medal-level results on IOI 2025 and the ICPC World Finals further strengthen the evidence of its reasoning capabilities across mathematics and programming. With substantial test-time scaling, the proposed post-training recipe produces a compact model capable of solving exceptionally challenging reasoning tasks.
  2. The paper offers a targeted solution to observed capability regressions, supported by ablation evidence. Table 1 shows that after IF-RL and multi-domain RL, AIME25 decreases from 92.9 to 89.4 and ArenaHard Creative Writing from 65.7 to 39.7. MOPD restores or improves these scores to 92.3 and 71.0, respectively, while retaining most of the instruction-following gains: IFBench reaches 81.7, compared with 56.0 after SFT and 84.4 after the multi-domain stage. Figure 2 additionally examines GRPO, an increased math response-length budget, and subsequent RLHF without MOPD. These experiments provide concrete evidence for the role of MOPD beyond simply reporting final model scores.
Critical weaknesses
  1. Methodological novelty is incremental, and the gains are not sufficiently isolated. The recipe combines existing methods while changing SFT data and multiple training stages, making the contributions of domain grouping and MOPD difficult to isolate. Step-count comparisons also do not establish compute efficiency, as the reported costs do not clearly account for SWE RL and additional teacher training. As suggestions for further discussion, I recommend discussing Open-MOPD [1] on token-budget imbalance and Xie et al. [2] on controlled comparisons of multi-teacher distillation methods. The latter finds that well-tuned off-policy baselines can match MOPD at lower computational cost in its settings, providing useful context for the efficiency discussion here.
  2. The relationship between competition scores and the efficiency of a small model needs more experimental qualification. The paper explicitly discloses that competition results rely on substantial test-time compute, which is welcome. However, Table 3 reports an IMO score of 5/42 with single-pass sampling versus 35/42 with test-time scaling. Appendix D.1 reports 8,192 generated responses per round, with multiple rounds used for individual problems. A smaller parameter count alone does not establish a lower total inference cost.
Questions for authors
  1. Could you clarify what is included in the approximately 13K H100 GPU hours? Table 8 does not separately list SWE RL. Does the total include training the RLHF teacher from SFT? When teacher training, teacher scoring, and the MOPD stage itself are included, what is the actual difference in GPU hours or FLOPs between the MOPD-based approach and direct RLHF? This would affect the assessment of the efficiency claim.
  2. How do the baseline models perform when given the same iterative refinement pipeline and comparable test-time compute budgets? In particular, could Nemotron-3-Nano-30B-A3B and other models of comparable size achieve similar IMO/IOI results using the same generate–verify–refine or generate–select–submit procedure, with matched access to verification feedback and comparable generation budgets? Reporting both single-pass and iteratively refined scores would help distinguish the contribution of the proposed post-training recipe from that of the inference-time procedure, and establish whether Cascade-2 retains an advantage under comparable evaluation conditions.
  3. Could you clarify whether the decontamination described in Appendix C.9 explicitly covers IMO 2025 and IOI 2025 across all SFT, RL, and MOPD data? Please specify the matching procedure and thresholds, whether both problem statements and solutions were checked, and the number of matches removed. It would also be helpful to distinguish the guarantees provided by post-training data filtering from any unresolved exposure in the pretrained base or teacher models.
1 comment
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for the appreciation and we try our best to resolve the valuable comments as below:

Methodological novelty is incremental, and the gains are not sufficiently isolated.

We appreciate the reviewer’s concerns regarding novelty. Our main contribution is a multi-domain cascaded RL framework that improves training efficiency while limiting performance degradation across domains, combined with MOPD to recover performance on specific domains. To our knowledge, we are the first to extend cascaded RL to multiple domains with a principled stage ordering. In addition, Section 3.5 presents a detailed analysis comparing MOPD and GRPO across domains, offering new insights into their respective effects. We hope these points help address the reviewer’s concerns.

The relationship between competition scores and the efficiency of a small model needs more experimental qualification.

We thank the reviewer for the comment and we fully understand the concern. We would like to provide further details on our test-time scaling approach for IMO. Our model achieves scores of 21, 28, and 35 out of 42 with 2, 3, and 4 rounds of test-time scaling, respectively. The original purpose in presenting these results was to demonstrate that, with a well-designed test-time scaling pipeline, a small LLM can also achieve strong results on challenging reasoning tasks, competitive with frontier LLMs that have far more parameters, while also being more efficient. Specifically, our full test-time scaling pipeline consumes approximately 800 H100 GPU hours for scoring 35 in IMO 2025, whereas DeepSeek-V3.2-Speciale requires approximately 1,500 H100 GPU hours to achieve the same score, according to its official report methodology. We hope this addresses the reviewer’s concerns regarding efficiency.

Could you clarify what is included in the approximately 13K H100 GPU hours? Table 8 does not separately list SWE RL. Does the total include training the RLHF teacher from SFT? When teacher training, teacher scoring, and the MOPD stage itself are included, what is the actual difference in GPU hours or FLOPs between the MOPD-based approach and direct RLHF? This would affect the assessment of the efficiency claim.

We apologize for the confusion. The reported 13K H100 GPU hours covers the full training pipeline, from SFT through all subsequent stages, including the cost of training the RLHF teacher, as shown in the table. We also thank the reviewer for noting that the SWE-RL cost was missing from the table: this stage requires approximately 564 H100 GPU hours, and we will add it in the revision. Furthermore, we would like to clarify that MOPD plays an important role in our pipeline as an efficient way to recover the performance degradation introduced by specific Cascade RL stages. Without MOPD, applying RLHF alone does not achieve better overall performance than MOPD + RLHF (see Figure 2(c) and Table 6). Table 8 accounts for all of these costs, including RLHF teacher training and MOPD training.

How do the baseline models perform when given the same iterative refinement pipeline and comparable test-time compute budgets?

We thank the reviewer for this valuable suggestion. Following it, we applied our test-time scaling approach to our baseline model, Nemotron-Nano-v3, on IOI 2025. The baseline achieves a score of only 262.74, far below the gold-medal-level performance of our final model. We will include this experiment in the revised version.

Could you clarify whether the decontamination described in Appendix C.9 explicitly covers IMO 2025 and IOI 2025 across all SFT, RL, and MOPD data?

Sorry for the confusion. For the competitive coding data used in our training (SFT, RL and MOPD), the knowledge cutoff is 2024/08, so contamination is not possible. For competitive math, we use proof data drawn from sources separated from the IMO. We also applied strict filtering and n-gram overlap detection to rule out any remaining contamination. We will make this clear in our revision.

Reviewer_E7zaOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Nemotron-Cascade 2, an MoE model built on the Nemotron-3-Nano pretrained base, with 30B total parameters and 3B active parameters. It addresses a practical challenge in post-training: improving math, coding, instruction following, preference alignment, long-context processing, and agent capabilities without sacrificing one capability to improve another.

The authors make two changes to the existing Cascade RL framework. First, they train tasks with similar response lengths and verification costs together in the same stage. Second, they introduce multi-domain on-policy distillation (MOPD) after multi-domain RL, using earlier checkpoints that perform well in each domain as teachers to recover capabilities lost during training. The pipeline consists of SFT, IF-RL, multi-domain RL, MOPD, RLHF, long-context RL, Code RL, and SWE RL.

The jointly trained tasks include STEM multiple-choice questions, tool calling, and structured output, accounting for approximately 55%, 30%, and 15% of the mixture. MOPD uses three teachers: the SFT checkpoint for math, a separate RLHF checkpoint trained from SFT, and a checkpoint selected from the IF-RL and multi-domain RL stages. The student generates its own responses, and the assigned teacher provides token-level probability feedback. Teacher assignment depends on the prompt's data source.

The model performs well in math, competitive programming, instruction following, and preference alignment. However, these strengths do not extend to every task: it still trails other compared models on some knowledge, multilingual, long-context, and agent benchmarks.

Critical strengths
  1. The model has strong math and coding performance among similarly sized MoE models. With only 3B active parameters, it scores 87.2 on LiveCodeBench v6 and 92.4 on AIME25.

  2. The analysis of cross-domain interference and capability recovery through MOPD provides useful empirical evidence:

    • Cross-domain interference: IF-RL and multi-domain RL improve IFBench's prompt-level score from 56.0 to 84.4, but reduce AIME25 from 92.9 to 89.4 and ArenaHard Creative Writing from 65.7 to 39.7. These results show that gains across domains do not simply accumulate; training different capabilities can create conflicts.
    • Capability recovery: After MOPD, AIME25 recovers to 92.3 and ArenaHard Hard / Creative reaches 85.5 / 71.0, while LCB v5 remains at 88.2.
Critical weaknesses

W1. The novelty needs to be stated more clearly.

The approach appears to be a sensible integration of existing methods. The authors should explain more clearly how it differs from prior work. For example, what new, reusable insights does it offer beyond existing multi-teacher distillation and Cascade RL?

Cascade RL, GRPO, and on-policy distillation are established techniques. The useful aspects here are jointly training compatible domains and selecting teachers from different stages to recover and combine capabilities through MOPD. However, it is unclear whether these changes constitute a distinct methodological innovation.

W2. The sources of the overall improvement are not sufficiently isolated.

Although the comparison with Nemotron-3-Nano uses the same pretrained backbone, the SFT data, training scale, and post-training pipeline differ. This raises concerns about the fairness of the comparison, since the performance gap may not be attributable solely to the proposed method.

W3. The principles behind domain grouping and stage ordering still lack direct validation.

Section 3.1 argues that domains with similar response lengths and verification costs should be grouped together, and that IF-RL and RLHF should precede long-reasoning stages. These choices are reasonable from an engineering perspective, but the paper mainly offers empirical explanations without enough controlled comparisons to establish their effects.

Questions for authors

Please address the concerns raised in the Weaknesses section.

1 comment
AuthorsOpenReview ↗

Official Comment by Authors

Thanks for your insightful comments. We address your concerns below:

these strengths do not extend to every task: it still trails other compared models on some knowledge, multilingual, long-context, and agent benchmarks.

We would like to clarify that the primary goal of this paper is not to claim state-of-the-art performance on every benchmark, but to propose an efficient and effective training pipeline for enhancing multi-domain model capabilities. The largest gap between our model and models of comparable size (e.g., Qwen3.5-35B-A3B) is mainly on knowledge-intensive tasks, which benefit from broader and more diverse general-knowledge data. On reasoning tasks, by contrast, our model substantially outperforms these baselines and even approaches open-source models with roughly 10× more parameters.

The novelty needs to be stated more clearly …. For example, what new, reusable insights does it offer beyond existing multi-teacher distillation and Cascade RL? …. it is unclear whether these changes constitute a distinct methodological innovation.

We appreciate the reviewer’s concerns regarding novelty. Our main contribution is a multi-domain cascaded RL framework that improves training efficiency while limiting performance degradation across domains, combined with MOPD to recover performance on specific domains. To our knowledge, we are the first to extend cascaded RL to multiple domains with a principled stage ordering. In addition, Section 3.5 presents a detailed analysis comparing MOPD and GRPO across domains, offering new insights into their respective effects. We hope these points help address the reviewer’s concerns.

The sources of the overall improvement are not sufficiently isolated …. This raises concerns about the fairness of the comparison.

We understand the reviewer’s concern, and would like to clarify that improved benchmark performance relative to other open-source models is an outcome of our work rather than its main contribution. Our primary contributions are: (1) Cascade RL + MOPD training pipeline with a principled stage ordering, which effectively mitigates the performance degradation that typically follows each stage of domain-sequential training; and (2) comprehensive analysis and ablations of each component of the training pipeline.

The principles behind domain grouping and stage ordering still lack direct validation …. the paper mainly offers empirical explanations without enough controlled comparisons to establish their effects.

We apologize for the confusion. First, we would like to clarify that the ordering of the Cascade RL stages is purely from empirical explanations, but is grounded in our experimental observations. For example, we place the IFRL/RLHF stage at the beginning of the pipeline because, when it follows the reasoning RL stages, it sharply constrains the model’s reasoning length and leads to a substantial drop in reasoning benchmark performance. This observation motivated us to place IFRL/RLHF first, so that the model’s responses remain concise and accurate without compromising its reasoning ability. Second, we believe these empirical findings are themselves a valuable contribution: we systematically analyze how the ordering of Cascade RL stages affects the full training pipeline, providing practical guidance for others training general-purpose models.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.