acceptodds
Under review as a conference paper at ICLR 2027

Reason in Style: Discovering and Controlling Style in Language Models

Abstract

Language models learn content and style jointly, making stylistic variation in their outputs difficult to identify and control. We study whether recurring styles in model responses can be discovered without supervision and explicitly controlled. We design an algorithm that learns to separate representations of content and style from language models' outputs and validate its effectiveness on math questions in a controlled setting. By applying this method to over 100K verified traces from nine distinct teacher models, we discover six recurring yet imbalanced styles. We then fine-tune smaller student models to follow these styles when explicitly conditioned on them, using importance weighting to balance the contribution of the styles represented in the corpus. This approach improves Pass@ over standard fine-tuning on the same data across six math reasoning benchmarks, demonstrating that we can diversify the style of answers effectively. We confirm that this also results in strong correspondence between requested and realized styles. We find that style affects correctness: the probability of solving a problem depends on the style we condition on, and different problems benefit from different styles. In summary, our results show that stylistic variation in model-generated data can be discovered in an unsupervised way, and made explicit, providing a source of both control and improved reasoning performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
style, representation learning, reasoning, supervised fine-tuning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cp4WOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Reason in Style, a framework for discovering and controlling stylistic variation in mathematical reasoning traces. It learns disentangled content and style representations, identifies recurring styles through clustering, and trains student models using style-conditioned fine-tuning with importance weighting. Experiments across multiple mathematical reasoning benchmarks demonstrate improvements in Pass@k.

Critical strengths
  • The paper addresses an important and relevant problem in LLM research, particularly regarding reasoning diversity and controllability.
  • The paper is clearly written and easy to follow, with a well-structured presentation of the motivation and proposed method.
  • The empirical evaluation is reasonably comprehensive, covering multiple benchmarks and model sizes. The additional analyses provide interesting findings on the relationship between reasoning styles and problem-solving performance.
Critical weaknesses
  1. The distinct motivation for style-specific control could be clearer. The paper offers an interesting perspective on stylistic variation, but its practical motivation overlaps with existing work on preserving reasoning diversity and improving mode coverage, such as ModC [1] and SSFT [2]. Although discovering stylistic structure is potentially valuable, the main downstream benefit is demonstrated through mixed-style Pass@k, which also measures the effectiveness of covering complementary solution modes (§5–§6). A clearer explanation of what style discovery and control enable beyond general reasoning diversity (and in which settings this distinction matters) would make the significance of the problem formulation more compelling.

  2. The comparisons could better reflect alternative approaches to exploiting reasoning diversity. The baselines primarily cover vanilla SFT and random-, teacher-, or discovered-style conditioning (§5.1). Comparisons with ModC’s gradient-based mode discovery [1] and SSFT’s mode-preserving training [2] would help position the method against closely related alternatives. In my opinion, another potentially relevant direction is gradient-based data diversification, as explored in Prismatic Synthesis [3], which identifies underrepresented modes through gradient-based clustering and selectively incorporates newly generated samples to improve training-data diversity. Although its data-expansion setting differs from the fixed-corpus setting considered here, a baseline adapting its core idea of gradient-based clustering and diversity-aware rebalancing to the same experimental setting could provide a useful comparison. Even if such an empirical comparison is not feasible due to differences in their settings, a more explicit conceptual comparison or discussion of these approaches would offer additional insights into how the proposed method relates to broader approaches for exploiting reasoning diversity.

  3. The interpretation of the discovered clusters as styles could be more precisely justified. The method encourages content representations to agree across traces answering the same question, while style representations capture complementary trace-level variation (§3). However, solutions to the same question can differ in mathematical strategy as well as presentation, and the paper includes verification and backtracking within its notion of style. This leaves some ambiguity about whether the discovered clusters capture expressive styles, broader reasoning modes, or a mixture of both. The successful recovery of four prompted writing styles provides useful evidence in the controlled setting (§4.1). A more explicit operational definition of style, together with a clearer justification for extending this interpretation to the natural multi-teacher clusters, would strengthen the paper’s central claims about content–style separation.

[1] Wu et al., Mode-Conditioning Unlocks Superior Test-Time Compute Scaling, ICLR 2026.

[2] Jia et al., Training Large Language Models to Reason in Parallel with Global Forking Tokens, ICLR 2026.

[3] Jung et al., Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning, NeurIPS 2025.

Questions for authors
  • I would be interested in the authors' perspective on whether style discovery and conditioning could be further developed to enhance the model's exploration capabilities during reasoning, rather than merely increasing diversity across independently generated solutions.

  • Given that the current study focuses on mathematical reasoning with verifiable answers, I would also be interested in the authors' perspective on whether this approach could be effectively extended to domains where correctness cannot be objectively verified, and what challenges might arise in such settings.

Reviewer_tm9mOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates stylistic variation in reasoning traces and its implications for student-model performance. It learns representations intended to separate content from style, clusters teacher traces into six groups, and uses the resulting labels for style-conditioned, reweighted supervised fine-tuning. The experiments evaluate generation control and reasoning performance, including Pass@k under a fixed total sampling budget.

Critical strengths
  • The empirical gains in Pass@k are convincing and demonstrate the practical value of conditioning and reweighting teacher trajectories using the discovered categories. These results provide a useful perspective on how variation across training traces can affect the diversity and success of sampled solutions.
Critical weaknesses
  • The paper does not sufficiently clarify the scope of “style” or establish how the discovered categories correspond to this concept. Separating information shared across solutions to the same question from residual trace-level variation need not separate content from style, and the controlled recovery of prompted writing styles does not fully resolve this ambiguity for the six clusters found in the teacher corpus.
Questions for authors
  • Could the authors further clarify what “style” means in this work and how it is distinguished from solution content and reasoning strategy? For example, which differences in wording, solution steps, or verification procedures count as stylistic variation, and could concrete examples help illustrate this boundary?

  • The method learns style representations by separating information shared across solutions to the same question from the remaining trace-level variation. Could the authors explain how this design corresponds to their intended notion of style, and how potential differences in solution strategies within that remaining variation should be understood?

  • How should the four prompted writing styles in the pilot and the six categories discovered in the teacher corpus be understood under the same notion of style? Could additional analysis or representative examples help clarify the differences captured by these categories?

Reviewer_BX6uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors study the problem of discovering and controlling styles in generated reasoning traces. They use an autoencoder to learn separate content and style representations, then cluster the style representations to identify recurring styles across nine teacher models. Using the resulting labels, they fine-tune students with style prefixes and apply importance weighting to balance the contribution of different styles. Experiments show improved Pass@k across three student models on six mathematical reasoning benchmarks and provide evidence that different problems benefit from different styles.

Critical strengths

Please see questions.

Critical weaknesses

The method is intuitive and straightforward to use in training.

The authors study an important question, where they aim to separate style within training data.

The paper structure is easy to follow.

Questions for authors

For Section 4.1, I would suggest using more models to generate solutions and different style definitions to improve the claims. It could be that c can distinguish styles between solutions, but those styles are just different from the four defined in Section 4.1.

It seems that the main results are based on the same student family of models Qwen3, which might share similar trends (as observed in Table 2). I wonder if different family of models would show similar trends as well or not?

How is style related to difficulty of the problem and length of the solution? It would be great to analyze in these perspectives.

Have you tried experiments without adding style prefix but with reweighting?

While the paper studies multiple math benchmarks, it does not explore other domains which is hard to know how generalizable the proposed method is.

What are computational costs? How long did it take to train autoencoder with SCAS dataset? What GPUs were being used?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank all reviewers for their thoughtful feedback! We have incorporated their suggestions in our revised manuscript:

In the introduction, we clarify that we view style as how a solution is expressed, including its presentation and the reasoning strategy it uses. We also strengthen the motivation for our work in the third paragraph of the introduction.

We have expanded the experiments and analysis in response to the reviewers’ questions. These additions include:

  • mentioning the importance weighting without style prefix experiment in Section 6.1,
  • results with a new fine-tuned student model Llama 3.2 1B in Appendix G,
  • analyses of style in relation to solution problem difficulty in Appendix C.2,
  • examples of the four prompted styles from Section 4.1 in Appendix B.1 ,
  • reporting the computational resources and training costs in Appendix A.3.

We have added the following discussion points in Section 7:

  • relationship to work on reasoning diversity and discovery of complementary reasoning modes
  • possible extensions to data diversification and reasoning exploration
  • generalizability to other domains.

We are grateful for the reviewers' feedback, which helped us improve the manuscript’s clarity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.