- Rating
- 3
- Confidence
- 2(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces CASCADE, which synthesizes complex instructions by incrementally adding constraints, subtasks, and conditional branches, including branches determined by the model’s intermediate output. It co-constructs check graphs for verification and instruction versions for difficulty-adaptive RL through SPIRAL. Experiments cover SFT and RL on three models, seven benchmarks, and a new benchmark, CASCADEEval, evaluated on 48 models.
Critical strengths
-
CASCADE addresses a meaningful aspect of instruction following: executing dependent subtasks with output-dependent branching. Jointly constructing instructions, verification graphs, and intermediate versions provides a coherent resource for both training and evaluation.
-
The corpus shows substantial benefits beyond the newly introduced benchmark, supported by both SFT and RL experiments. For example, on Qwen3.5-9B, CASCADE-hard achieves 67.96 on ComplexBench versus 52.30 for the strongest existing-corpus result in that column. SPIRAL also reduces measured training time by approximately 40% relative to hardest-only training.
Critical weaknesses
-
SPIRAL’s incremental accuracy gains need stronger validation. Improvements over hardest-only training are 0.22, 0.62, and 1.88 IF-Avg points across the three models, but training uses a single random seed. There is also no direct comparison with an established difficulty-aware selection method; removing the offset does not substitute for such a comparison. Consequently, the robustness and distinct benefit of the proposed scheduler remain uncertain.
-
The complete training pipeline has a substantial reasoning tradeoff. On Qwen3.5-9B, CASCADE-adaptive achieves IF-Avg 54.98 and OOD-Avg 61.73, whereas CASCADE-SFT achieves 54.16 and 73.80, respectively (Tables 1 and 12). Moreover, the cold-start stage reduces OOD-Avg from 77.53 to 52.09. The shared initialization supports the controlled RL comparisons, but improvements over this weakened checkpoint do not establish that the overall recipe preserves reasoning or offers a better tradeoff than SFT.
Questions for authors
-
Do SPIRAL’s gains persist across training seeds and against an established difficulty-aware selector operating on the same instruction-version pool under matched budgets?
-
What causes the substantial reasoning degradation during cold start? Is there evidence that SPIRAL improves the instruction-following–reasoning tradeoff when initialized from the original Qwen3.5 model or the CASCADE-SFT checkpoint?
5 comments
Part 2/3
Against DAPO-style dynamic sampling (Yu et al., 2025) and AdaRFT (Shi et al., 2025), on the same instruction pool and under a matched update budget, SPIRAL has the highest IF-Avg in the new Qwen3.5-9B comparison. Since budgets are matched in optimization updates rather than total rollout compute, DAPO-DS's extra sampling and verification are counted in its runtime. SPIRAL reduces RL training time by 40.3% relative to Hardest and 36.1% relative to DAPO-DS.
Qwen3.5-9B. is relative to Flatten. Times are wall-clock hours on 16 H200 GPUs. AdaRFT's additional offline annotation costs 16.05 hours.
| Configuration | IFEval | IFBench | Complex | AdvIF | CasEval | IF-Avg | Δ | IF-Len | GPQA | AIME26 | OOD-Avg | Train (h) | Preproc. (h) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Flatten | 80.04 | 41.00 | 47.43 | 27.42 | 26.40 | 44.46 | — | 622 | 69.95 | 49.17 | 59.56 | 12.89 | 0.00 |
| AdaRFT b=0.5 | 83.64 | 44.83 | 57.09 | 34.23 | 19.20 | 47.80 | +3.34 | 1019 | 72.47 | 56.88 | 64.67 | 7.67 | 16.05 |
| AdaRFT b=0.3 | 85.58 | 50.33 | 66.35 | 36.16 | 18.80 | 51.44 | +6.98 | 2575 | 72.98 | 50.00 | 61.49 | 8.83 | 16.05 |
| Hardest | 84.94 | 51.50 | 67.96 | 33.92 | 27.20 | 53.10 | +8.64 | 2647 | 67.93 | 48.12 | 58.03 | 17.44 | 0.00 |
| DAPO-DS | 86.60 | 52.00 | 68.96 | 39.28 | 25.20 | 54.41 | +9.95 | 2747 | 72.73 | 48.12 | 60.42 | 16.30 | 0.00 |
| SPIRAL | 85.67 | 52.67 | 67.78 | 36.78 | 32.00 | 54.98 | +10.52 | 2078 | 70.96 | 52.50 | 61.73 | 10.42 | 0.00 |
The comparison isolates where the scheduler helps. SPIRAL's lead over the established selectors concentrates on the hardest, most branch-heavy data, with the best CasEval at 32.00 (+6.80 over DAPO-DS), while it stays close to them on the rule-style benchmarks. On the accuracy–reasoning balance, AdaRFT () is the only run with higher OOD-Avg (64.67) but pays about 7 IF-Avg points for it (47.80 vs. 54.98); no baseline is strong on both axes at once, whereas SPIRAL pairs the best IF-Avg with the second-best OOD-Avg at the lowest training cost of any method that needs no offline annotation.
Part 3/3
W2 / Q2. The pipeline shows a reasoning tradeoff: cold start lowers OOD-Avg from 77.53 to 52.09, and CASCADE-SFT reaches higher OOD-Avg than CASCADE-adaptive. What causes the cold-start degradation, and does SPIRAL improve the instruction-following vs. reasoning tradeoff when initialized from the original model or CASCADE-SFT?
Starting RL from CASCADE-SFT improves both instruction following and reasoning. On Qwen3.5-9B, SPIRAL raises IF-Avg from 54.16 to 55.58 (+1.42) and OOD-Avg from 73.80 to 75.55 (+1.75), with 32.4% shorter responses. Compared with Hardest from the same checkpoint, IF-Avg is comparable (+0.05), while OOD-Avg improves by 1.60.
The cold-start degradation reflects the shift toward instruction-following data without dedicated reasoning data. We constructed this checkpoint using queries from all comparison corpora, CASCADE, and general instruction sources, with a common response-generation and rejection-sampling protocol. The SFT experiments evaluate CASCADE as a training corpus; the original RL experiments use this shared initialization to compare training data and scheduling strategies.
Qwen3.5-9B, initialized from CASCADE-SFT. is relative to the SFT checkpoint; each “+” row denotes an RL continuation.
| Configuration | IFEval | IFBench | Complex | AdvIF | CasEval | IF-Avg | Δ | IF-Len | GPQA | AIME26 | OOD-Avg |
|---|---|---|---|---|---|---|---|---|---|---|---|
| CASCADE-SFT | 82.90 | 50.00 | 63.61 | 35.90 | 38.40 | 54.16 | — | 3223 | 71.97 | 75.62 | 73.80 |
| + Flatten | 76.99 | 31.50 | 48.74 | 21.05 | 33.20 | 42.30 | -11.86 | 880 | 72.22 | 67.29 | 69.75 |
| + Hardest | 85.58 | 52.33 | 66.57 | 35.98 | 37.20 | 55.53 | +1.37 | 2294 | 73.74 | 74.17 | 73.95 |
| + SPIRAL | 85.12 | 51.17 | 66.35 | 40.86 | 34.40 | 55.58 | +1.42 | 2179 | 72.98 | 78.12 | 75.55 |
SPIRAL therefore improves both IF and OOD scores over CASCADE-SFT, although OOD-Avg remains 1.98 points below the original model's 77.53.
References. Shi et al. (2025), AdaRFT, arXiv:2504.05520; Yu et al. (2025), DAPO, arXiv:2503.14476.
Part 1/3
We thank the reviewer for highlighting the need for stronger validation of the scheduler and a clearer assessment of the reasoning tradeoff. We added multi-seed runs, adaptive sampling baselines, and RL experiments initialized from CASCADE-SFT, and respond to each concern below; we will incorporate the resulting revisions into the paper and upload a revised version shortly.
SPIRAL improves IF-Avg on both tested seeds, and its OOD and length benefits are consistent across them. We repeated Hardest and SPIRAL on Qwen3.5-4B with seeds 0 and 42. SPIRAL improves IF-Avg on both (paired gains +0.41 and +0.62; mean +0.51), improves OOD-Avg by a mean of +3.62, and produces 12.1% shorter responses.