acceptodds
Under review as a conference paper at ICLR 2027

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation

Abstract

We introduce Nemotron-Cascade 2, an open 30B MoE model with 3B activated parameters that delivers best-in-class reasoning and strong agentic capabilities. Despite its compact size, its mathematical and coding reasoning performance approaches that of frontier open models. It is the second open-weight LLM, after DeepSeek-V3.2-Speciale-671B-A37B, to achieve Gold Medal-level performance in the 2025 International Mathematical Olympiad (IMO), the International Olympiad in Informatics (IOI), and the ICPC World Finals, demonstrating remarkably high intelligence density with 20× fewer parameters. In contrast to Nemotron-Cascade 1 (Wang et al., 2026), the key technical advancements are as follows. After SFT on a more meticulously curated dataset, we substantially scale Cascade RL to cover a broader spectrum of reasoning and agentic domains, grouping compatible domains with similar response lengths and verification costs. Furthermore, we incorporate multi-domain on-policy distillation from the strongest intermediate checkpoints in each domain to efficiently recover from potential benchmark regressions and sustain performance gains throughout the Cascade RL process. We release model checkpoint and training data for research community.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.