acceptodds
Under review as a conference paper at ICLR 2027

How Does Atomic Data Composition Shape LLM Mathematical Reasoning?

Abstract

Reinforcement learning (RL) has substantially advanced the mathematical reasoning capabilities of Large Language Models (LLMs). However, how the domain composition of training data shapes target-capability amplification remains unclear. Motivated by the field-atomic perspective, we investigate how supervised fine-tuning (SFT) data from algebra, geometry, analysis, and topology affects subsequent target-domain GRPO. Beyond final performance, we annotate reasoning chains using ThinkARM's eight reasoning behaviors and analyze behavior distributions, transition dynamics, and their conditional associations with correctness. Our results show that domain-aligned SFT does not consistently provide the strongest initialization, largest GRPO gain, and best final performance, with the most effective source domain varying across target capabilities. Different target domains exhibit distinct behavioral demands, while models initialized with different field data retain different reasoning profiles after the same target-domain GRPO. Moreover, the strength and direction of transition–correctness associations vary across **domains capability** and **training compositions** and Path-reward experiments show that reinforcing high-value transitions can improve target-domain reasoning. These findings highlight the importance of behavior-aware data selection and post-training for more effective mathematical capability amplification.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Large language Model, Reinforcement Learning, Domain Data Compositons, Mathematical Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4imrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper categorizes mathematical training data into four atomic data, as algebra, geometry, analysis, and topology. It assigns eight types of behavioral labels to the thought chains. It demonstrates that the transfer effects vary depending on the combination of SFT and GRPO used for different atomic data. The study finds that SFT from different mathematical domains alters the distribution of thinking behaviors generated by GRPO, and the conclusions indicate that transfer across mathematical domains exhibits asymmetry.

Critical strengths
  1. This paper represents a further combination and generalization of the conclusions from the references “Atomic Thinking of LLM” and “ThinkARM,” and the experimental setup is relatively fair.
Critical weaknesses
  1. The improvement brought by path reward is limited, and the current findings regarding domain transfer and reasoning behavior in SFT+GRPO are unlikely to guide future LLM training .
Questions for authors
  1. Why does ThinkARM emphasize the role of Explore, while this paper observes this phenomenon only rarely?
  2. A minor type error: the final of the introduction states “four key insights,” but actually lists five points.
Reviewer_yCMzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how field-specific SFT initialization affects subsequent target-domain GRPO. It compares initializations using algebra, geometry, analysis, and topology data, then characterizes generated reasoning through ThinkARM’s eight behavioral categories. It analyzes behavior proportions and transitions, fits associations between these features and correctness, and introduces a path reward that reinforces selected transitions. The main claims are that domain-aligned SFT is not consistently the best initialization, that different training compositions induce different reasoning profiles, and that rewarding selected transitions can improve performance.

Critical strengths

The experimental design includes both the original model without additional field-specific SFT and four SFT initializations, allowing comparisons before and after target-domain RL. The paper also includes a partial cross-backbone replication on MetaMath-Mistral-7B for geometry and topology. These are useful controls, and the setup could serve as a worked example of an RL study worth posting on arXiv or GitHub for others to reference.

The behavioral analysis is not entirely qualitative. The authors evaluate candidate annotators against a human-annotated set and conduct a path-reward intervention rather than relying exclusively on descriptive examples. These efforts are positive, although they do not resolve the main concerns below.

Critical weaknesses

The paper does not establish a sufficiently substantive new contribution beyond dataset-specific transfer observations.

The manuscript explicitly builds on a field-atomic framework that already studies asymmetric transfer among mathematical domains. Its extension is to examine these initializations after target-domain GRPO and describe the resulting reasoning behaviors. However, the study does not turn these observations into a validated principle for selecting data in a new setting. It remains unclear how a practitioner should use the analysis without first performing essentially the same exhaustive comparison of candidate initializations.

The broader relationship between post-training and generalization has also received substantial study. For example, Chu et al., SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training (ICML 2025), explicitly evaluates generalization to unseen rule and visual variants. That work does not settle the present initialization question, but it illustrates why another transfer study needs a clearly differentiated and well-supported finding. My concern is not that the research area is closed; it is that this manuscript adds too little reliable, reusable knowledge to it. Proceedings of Machine Learning Research

The improvements are too small and may not be significant.

Using the rounded average accuracies in Table 1, the best non-domain-aligned SFT initialization exceeds the domain-aligned initialization by approximately 0.1, 1.1, 0.2, and 1.4 percentage points across the four targets. Table 1 does not report uncertainty across independent training runs. The models studied here are 7B; differences between algorithms usually shrink for larger models, so this result may not generalize to frontier models.

The path-reward experiment is a useful intervention, and its largest reported improvement—3.7 percentage points on Topology-low—is larger than the initialization differences discussed above. Nevertheless, rewarding transitions into Answer does not by itself validate the proposed explanation of domain transfer. The intervention needs controls that distinguish the value of the selected transitions from simpler effects involving response length, answer formation, or termination behavior.

More generally, the manuscript has not shown that its behavioral descriptions can predict an effective initialization on held-out data or transfer into a reliable training decision. Annotation agreement establishes that the labels can be applied consistently; it does not establish that the labels capture the causal or practically decisive factors behind the reported improvements.

Questions for authors
  1. Can you verify that the differences produced by this paper’s data choices (the datasets used, the Answer-transition reward) are larger than the effects produced by algorithmic choices (output-length limits, replacing GRPO with another policy-optimization method, using more data, or the seed)? I would especially like to see the effect of the seed and of the data volume verified.
Reviewer_dupROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates how mathematical-domain SFT initialization affects subsequent target-domain GRPO. It compares a base model and initializations trained on algebra, geometry, analysis, or topology, primarily using Qwen2.5-Math, with additional Geometry/Topology results on MetaMath-Mistral. Beyond accuracy, generated reasoning is segmented and annotated with eight ThinkARM behavior categories. Token shares and behavior transitions are related to answer correctness through regularized logistic regression, including composition-specific models and pooled models controlling for initialization. An annotation pilot compares several LLM annotators against a human-labeled set of 400 responses. Finally, GRPO is augmented with rewards for selected transitions into Answer. The reported results suggest that domain-aligned SFT is not consistently the best initialization and that behavior-based reward shaping can improve some target-domain scores, although the evidence does not yet isolate the proposed explanatory mechanism.

Critical strengths
  1. The study separates initialization, RL gain, and final performance. Table 1 compares all four SFT sources and the base initialization before and after each target-domain GRPO run, rather than equating a better starting point with a larger RL benefit. Appendix C.1 provides a second-backbone check for two domains. This design exposes meaningful differences that aggregate post-training accuracy would hide.

  2. The behavioral analysis includes a useful measurement check and an intervention. Appendix E.1/Table 7 validates candidate annotators against 400 human-labeled responses, while Sections 3.3 and 5.3 control for length and initialization in parts of the association analysis. Table 4 goes beyond visualization by testing reward shaping. These are useful steps toward making behavior-based observations testable, even though important controls remain missing.

Critical weaknesses
  1. The magnitude and attribution of the source-domain effects are insufficiently established. Table 5 contains different source-set sizes, for example 8,330 Algebra versus 6,512 Topology training problems, but the manuscript does not report a matched-example/token SFT comparison or sufficient SFT optimization details to isolate domain composition from training exposure and source difficulty. Many final differences in Table 1 are small, without uncertainty across training runs or test problems: Algebra-aligned reaches 80.1 versus 80.2 for the best non-aligned SFT source. Figure 1(a), however, displays a 2.1-point Algebra advantage and other values that do not directly match Table 1's final averages. Until these results and budgets are reconciled, the central conclusion is an interesting observation rather than a controlled demonstration of composition-driven transfer.

  2. The transition analysis and reward intervention do not yet establish the claimed reasoning mechanism. The regression controls for length and composition but not problem identity/difficulty; transitions into Answer may partly reflect problems the model already knows how to finish. Table 4 lacks a length-matched or generic completion-reward control, so improvement after rewarding these transitions does not distinguish the identified paths from a simpler completion/verbosity effect. The source of the responses used to select paths, relative to the final evaluation set, is also not specified clearly. Finally, Eq. (6)/Appendix D define a transition proportion, whereas Figure 9 defines a binary indicator of whether a target pair occurs. These produce different incentives and leave the actual intervention ambiguous.

Questions for authors
  1. [Clarification] What exact metric, difficulty aggregation, and experiment produce Figure 1(a)? Please reconcile its Algebra/Geometry/Analysis advantages with Table 1 and specify the SFT data counts, token exposure, and optimization budgets actually used for each source. This determines whether the headline domain-composition comparison is reproducible and controlled.

  2. [Clarification] Were transition coefficients and the rewarded path set selected using training/validation traces independent of the final test questions? Was the implemented path reward the proportion in Eq. (6), or the binary indicator in Figure 9? These answers directly affect both validity and reproducibility of Table 4.

  3. [Evidence request] Can you compare the selected-path reward with a completion/length control under the same GRPO budget, and assess the transition associations within matched problem/difficulty groups? If the selected paths remain helpful under these controls, the proposed mechanism would be substantially more convincing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.