acceptodds
Under review as a conference paper at ICLR 2027

CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning

Abstract

Large Language Models (LLMs) have recently exhibited remarkable reasoning capabilities, largely enabled by supervised fine-tuning (SFT)- and reinforcement learning (RL)-based post-training on high-quality reasoning data. However, reproducing and extending these capabilities in open and scalable settings is hindered by three fundamental data-centric challenges: (1) the cold-start problem, arising from the lack of seed datasets with detailed, long Chain-of-Thought (CoT) trajectories needed to initialize reasoning policies; (2) limited domain coverage, as most existing open-source reasoning datasets are concentrated in mathematics, with limited coverage of broader scientific disciplines; and (3) the annotation bottleneck, where the difficulty of frontier-level reasoning tasks makes reliable human annotation prohibitively expensive or infeasible. To address these challenges, we introduce CHIMERA, a compact synthetic reasoning dataset comprising 9K samples designed to support generalizable reasoning across domains. CHIMERA is constructed with three key properties: (1) it provides rich, long CoT reasoning trajectories synthesized by state-of-the-art reasoning models; (2) it has broad and structured coverage, spanning 8 major scientific disciplines and over 1K fine-grained topics organized via a model-generated hierarchical taxonomy; and (3) it employs a fully automated, scalable evaluation pipeline that uses strong reasoning models to cross-validate both problem validity and answer correctness, removing the reliance on human annotations. We use CHIMERA to post-train a 4B Qwen3 model using a combination of SFT and RL. Despite the dataset's modest size, the resulting model achieves strong performance on a suite of challenging reasoning benchmarks, including GPQA-Diamond, AIME 24/25/26, HMMT 25, and Humanity's Last Exam, approaching or matching the reasoning performance of substantially larger models such as DeepSeek-R1 and Qwen3-235B.

Primary area:
datasets and benchmarks
Keywords:
Synthetic Data, LLM Reasoning, Post-training

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_naTSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CHIMERA, a synthetic reasoning dataset with 9,225 problems across eight subjects and 1,179 topics. The authors use LLMs to expand topics, generate problems, and check the problems and answers, then use an open reasoning model to produce long reasoning traces. They report gains on several reasoning benchmarks after training Qwen3-4B with SFT and RL.

Critical strengths
  1. The paper addresses the need for reasoning data beyond mathematics and coding. CHIMERA combines broad subject coverage with long reasoning traces, and its automated pipeline reduces reliance on expert annotation.

  2. The results across several models and benchmarks show that a relatively small synthetic dataset can improve existing reasoning models. This gives the work practical value as a data resource for further training.

Critical weaknesses
  1. I find the methodological novelty limited. CHIMERA combines topic expansion, problem generation, verification with LLMs, and reasoning distillation. Earlier work such as s1 and LIMO has already explored the value of small, carefully curated reasoning datasets. The paper should clarify what new ideas or design choices it adds.

  2. The added benefit of subject diversity is not yet clear. In Table 3, full CHIMERA exceeds training on mathematics alone by 0.7 points on GPQA Diamond and 0.9 on HLE. Mathematics alone already improves these scores over the base model by 2.3 and 0.8 points. This is insufficient to demonstrate CHIMERA’s advantage in terms of broader subject coverage. Evaluations on OOD tasks (such as code and general reasoning) should be added.

Questions for authors
  1. Can you evaluate the trained models on additional code and other general reasoning benchmarks?

  2. Can you report the number of samples retained after each filtering stage, including problem validation, correctness verification, and decontamination, as well as the final dataset sizes used for SFT and RL?

Reviewer_RnrjOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Chimera, a compact reasoning dataset designed to teach generalizable reasoning patterns. The Chimera pipeline has 3 stages: topic expansion in which a model generates a variety of scientific topics, problem generation in which problems are generated, and finally solution synthesis. The authors show that fine-tuning on these datasets can significantly improve aime, hle, hmmt, and gpqa on qwen 4b models.

Critical strengths
  1. The authors show strong gains over baselines, and produce a model that is strong for its size.
  2. The presentation of the method is easy to understand.
  3. The paper contains detailed ablations.
Critical weaknesses
  1. It is not clear what the different training budgets are for each SFT experiment, which makes it hard to understand the performance gain.
  2. The token efficiency of the trained models is not considered, which could significantly impact the utility of their models.
Questions for authors
  1. How many tokens are each model fine-tuned upon in table 2?
  2. What is the output generated reasoning length of the Chimera models? If Aime improves by 1% but the generated tokens increase by 10%, that may not be a worthwhile trade-off.
Reviewer_vEvgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents CHIMERA, a synthetic reasoning dataset of about 9K problems spanning 8 scientific disciplines and 1,179 fine-grained topics. The data is built in three stages. GPT-5 first expands a hierarchical topic taxonomy, then GPT-5 generates problems and answers that are cross-checked by GPT-5 and o4-mini, and finally Qwen3-235B-A22B-Thinking writes long-form reasoning trajectories for the retained problems. The authors post-train Qwen3-4B-Thinking on this data with SFT followed by CISPO-based RL. The resulting model improves over the base model on several reasoning benchmarks and on some tasks gets close to much larger models. The central claim is that a small but carefully constructed synthetic dataset is enough to improve reasoning in a way that generalizes.

Critical strengths
  1. The dataset covers 8 disciplines and 1,179 topics, and the hierarchical topic expansion gives a systematic way to steer generation. This moves away from the math-heavy distributions that dominate many existing reasoning datasets.
  2. The gains on several hard benchmarks are encouraging. The comparison against an OpenThoughts subset of the same size, with solutions produced by the same synthesizer, is a useful controlled experiment and suggests that the generated problems themselves carry real training value.
  3. The synthesis procedure is easy to follow. The appendix includes the generation and verification prompts, the topic taxonomy, sample reasoning traces and extra results, which helps both in understanding the method and in reproducing it.
Critical weaknesses
  1. The main concern is novelty. The pipeline is mostly a combination of known components: LLM-based topic expansion, problem generation, cross-model verification and distillation of reasoning traces. Generating problems from scratch is a reasonable practical choice, but the paper does not point to a specific technical idea that explains why the approach works.
  2. The title stresses compact data, yet compactness here seems to just mean a modest number of examples, not something the method is designed to optimize. s1: Simple test-time scaling (Muennighoff et al., 2025) already showed strong reasoning results with 1K curated examples, selected with explicit attention to quality, difficulty and diversity. CHIMERA synthesizes new problems rather than selecting existing ones, but the paper does not make clear what we learn from its synthesis strategy beyond what prior work already showed.
  3. There are also no data scaling experiments, so it is unclear why roughly 9K examples is a good or near-optimal budget. The solutions are also very long, 11,121 words on average, so I am not convinced the dataset is compact in terms of total training tokens or compute. Comparisons at matched token budgets, together with ablations of the main pipeline components, would make the data efficiency argument much stronger.
  4. Problem validity and answer correctness are checked mainly through agreement between GPT-5 and o4-mini. GPT-5 is also the generator, and both verifiers come from the same provider, so correlated errors and self-verification bias are a real risk, particularly for specialized PhD-level questions.
  5. The LLM-as-a-Judge study rates clarity, well-posedness and reasoning depth, but it says nothing direct about whether the reference answers are correct.
  6. Several statistics about generation and filtering are missing. Algorithm 1 has a parameter for the number of samples per topic, but its value is never given. The number of initially generated problems, the verification acceptance rate and the per-topic retention are not reported either.
  7. I have concerns about whether the RL stage provides informative rewards, whether it is worth its cost, and whether it can be reproduced. According to Section 3.1, the RL set mixes problems already used in SFT with problems that Qwen3-235B failed to solve during trajectory synthesis but that the SFT-trained 4B model solved at least once in eight attempts. This could give a very uneven difficulty distribution. Problems seen during SFT may be easy for the student, while some of the originally unsolved ones may still be very hard even after the solvability filter. With group-relative RL and eight rollouts per prompt, both cases tend to produce groups that are all correct or all wrong, which means zero reward variance within the group and no useful advantage signal.
  8. The solvability filter helps with the hard problems to some extent, but it does not guarantee that they keep producing informative groups as training proceeds. The paper does not report prompt-level reward statistics, the fraction of zero-variance groups, difficulty distributions or RL training curves, so there is no evidence that this data mixture keeps providing a learning signal.
  9. The gains from RL on top of SFT are also small. GPQA-Diamond goes from 68.8 to 70.1, and HLE stays at 9.0. Since the RL stage reportedly takes about seven days on eight H200 GPUs, its contribution relative to its cost needs a better justification.
  10. With average solutions longer than 11K words, the paper should state the maximum SFT sequence length, the truncation strategy and how very long trajectories are handled. These details matter for reproducibility and for understanding how well the long-CoT supervision actually works.
Questions for authors
  1. How many problems are generated per topic, and how many remain after cross-model verification? Could you report acceptance rates and the subject and topic distributions before and after filtering, to show whether the intended diversity is preserved?
  2. Why were GPT-5 and o4-mini chosen as verifiers when GPT-5 is also the generator? Did you consider verifiers from another provider, or an independent audit of a sample of accepted problems to estimate the actual answer accuracy?
  3. What is the main methodological contribution of CHIMERA relative to earlier sample-efficient approaches such as s1? Could you add data scaling experiments and comparisons at matched token budgets to back up the compactness claim?
  4. What fraction of the RL set consists of SFT-seen problems versus initially unsolved ones? Could you report pass rates for each subset, within-group reward variance, and the fraction of all-correct and all-wrong rollout groups during training? How much of the RL improvement comes from each subset?
  5. What maximum sequence lengths are used for SFT and RL? Are long trajectories truncated, filtered or processed in some other way? What fraction of the data exceeds the context limit, and how does reasoning length affect downstream performance?
1 comment

Since the authors have not given any response to my comments, I will maintain my original score here.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Response to All Reviewers

We thank all reviewers for their constructive feedback. We summarize our main responses and additional results below.

  • Pipeline contributions and ablations, raised by Reviewers vEvg and naTS. We clarified CHIMERA’s contribution in automatically constructing new, challenging problems with explicit subject and topic coverage. We added ablations of subject expansion, problem generation, and solution synthesis. These ablations support the effectiveness of topic expansion, stronger problem generation, and detailed reasoning supervision in our pipeline.

  • Training budgets and token efficiency, raised by Reviewers vEvg and Rnrj. We reported training-token counts and corresponding performance, including a comparison with s1 at 0.75B tokens. At this reduced budget, CHIMERA outperforms all compared SFT baselines on all six benchmarks. We also reported generated-token lengths and comparisons with majority voting to characterize the inference-cost trade-off.

  • Broader generalization, raised by Reviewer naTS. Additional evaluations show gains on LiveCodeBench v6 (+1.6 points) and MMLU-Pro (+1.0 point). Our comparisons also include multi-domain datasets, OpenScience and OpenThoughts-114K, and Table 9 reports training improvements across three models.

  • Data quality and RL training, raised by Reviewers vEvg and naTS. We added an independent cross-provider evaluation of 100 problems, with 92 judged correct or having only minor issues. We reported filtering statistics, final dataset sizes, sequence limits, and truncation handling. For RL, we provided subset-level pass rates and reward variance, together with rollout-group statistics showing that 49.2% of groups have nonzero reward variance.

Thank you again for your time and effort!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.