acceptodds
Under review as a conference paper at ICLR 2027

Steerable by Design: Conditional Pretraining for Robust Control of Language Models

Abstract

Our goal is to build language models that are steerable by design. Today's language models are pretrained with next-token prediction, which relegates alignment and control to the final 1% of training tokens or inference-time interventions. These models repurpose a single input channel for both instructions and data, leading to common failure modes where the model mistakenly treats data as instructions such as intentional jailbreaks and unintentional in-context alignment drift. To correct this, we propose a pretraining method and architecture that separates the two from the very start, producing Mancelle-3B. An encoder processes high-level semantic features of upcoming text with cross-attention that prevents the token stream from writing back into this control channel. During pretraining, the model learns to rely on these features to predict the next token, turning the channel into an effective handle for steering the model. Mancelle-3B steers more strongly than prompt- and activation-based methods on comparable open models while maintaining fluency, and it withstands adversarial prompts and activation-space counter-steering that defeat those baselines. We reproduce this against a more controlled baseline with Mancelle-400M. Our results show that robust control can be built into language models if we do so early and intentionally.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI safety, LLM steering, pretraining, conditional pretraining, alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2ssDOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose training language models with a separate control channel for SAE-derived semantic features. During pretraining, features from the target segment teach the decoder to use that channel; a prediction head supplies features at inference time. The Mancelle models are evaluated for concept steering and resistance to prompt and latent counter-steering. The 400M experiments include controls matched by parameter count and by decoder architecture. The results show a promising control mechanism, alongside a substantial reduction in general capability.

The training signal is richer than a single good/bad label: it comprises positive and negative units derived from a curated SAE vocabulary. The decoder reads these through cross-attention, and a separate head learns the next segment’s features so generation can proceed without future-derived annotations. The natural-language interface maps a description into the control vocabulary. The main evaluation distinguishes concept adherence, instruction following and fluency, and then measures what remains under prompt and latent counter-pressure. This is an interesting proof of concept for a learned control interface, rather than a direct demonstration of reliable behavior in arbitrary downstream applications.

Critical strengths
  1. I find the central design insightful and creative. Conditional pretraining has precedents, which the paper acknowledges, but the combination of a rich SAE-derived vocabulary, a separate encoder channel, and a training objective that rewards using the signal is a coherent contribution. The splicing procedure is intended to make the continuation harder to predict from context alone, strengthening the incentive to read the control channel. This connects the data construction, objective and architecture in a useful way. The information decomposition clarifies that incentive; its value here is the design insight rather than a new entropy identity. The paper also recognizes the difficult inference step: predicting the conditioning signal when the future text is unavailable.

    The choice to make the channel valuable to prediction is the most compelling part of the paper for me. It gives the decoder a reason to learn the interface, and the feature vocabulary aims to make that interface useful for more than a few predetermined labels. This is a more interesting contribution than simply adding another auxiliary tag to the token stream. The combination also exposes a meaningful research question: how much control can be obtained while the model remains able to predict the signals it needs for ordinary generation? I would encourage the authors to emphasize this mechanism and the question it opens.

  2. The small-model controls are helpful. Matching total parameters and matching the decoder answer different questions, and the paper includes both. I also appreciate the reporting of absolute scores, fluency, and capability loss alongside steering retention. Normalizing the latent perturbation by residual norm makes that comparison more interpretable.

    The two 400M comparisons deserve explicit credit because neither matching choice is perfect on its own. Matching total parameters changes the decoder resources; matching the decoder leaves the conditioned model with additional components. Showing both makes it harder to attribute the entire result to one favorable choice. The full steering curves and the reporting of absolute scores also help avoid treating high retention from a weak initial steering level as a decisive success. These are thoughtful evaluation choices that strengthen the case for the idea.

Critical weaknesses
  1. My main reservation is what “robust control” means here. The experiments test concept steering against a suffix search with up to eight proposals and two fitted latent directions. That is useful evidence, but it is a fairly limited attack setting. Preventing writes into the encoder does not establish that the decoder’s behavior, or its context-dependent feature predictions, cannot be redirected. An LM-judged concept score is also a different target from factual correctness or reliable tool use. The judge’s agreement with independent human ratings and its repeated-rating consistency are not established here. Attacking the steered model also does not test whether the evaluator itself can be redirected by adversarial text in the response.

    The architecture gives a concrete separation property: decoder tokens cannot write back into the encoder representation. The broader claim about behavior requires additional evidence, because the decoder still reads untrusted context and predicts future conditioning from that context. I would distinguish direct changes to the control state from changes in how the decoder responds to a fixed state. The present tests support persistence under specified counter-pressure, but not an invariant that arbitrary contextual instructions cannot change useful behavior.

    The evaluator also deserves a separate validation argument. An automated concept score is useful for a broad sweep, yet it can favor responses that repeatedly invoke the concept without answering the question well. The paper’s fluency and instruction-following axes partly address this, and the response examples help. A small blinded human check on representative successes and failures, together with repeated ratings and a targeted evaluator challenge, would give clearer evidence that the observed control is not peculiar to one judge or scoring prompt.

  2. The capability cost is large enough to affect the practical interpretation. ChatCORE falls from 0.3454 to 0.1998 at 3B and from 0.2212 to 0.1543 at 400M. The oracle result uses benchmark answers and an earlier model variant. It shows that the channel can carry useful information, but it does not establish that feature prediction alone causes the loss or that improving it will recover the missing capability. I would frame the current result as a control–capability tradeoff.

    The oracle-conditioning experiment is informative but must remain a diagnosis, not a capability result: its signal is derived from benchmark answers and is applied to an earlier variant. It establishes that an informative channel can change prediction, while leaving open how much of the ordinary-generation loss comes from feature prediction, the training distribution, splicing or resource allocation. The paper’s proposed explanation is plausible, but the experiment does not identify it uniquely.

    I would also like the cost comparison to include the data-processing burden as well as inference. The authors already report a large annotated dataset footprint and substantial preprocessing compute. That disclosure is valuable. It makes the practical question more precise: for a given training and deployment budget, how much useful control is gained after accounting for the encoder, signal construction and capability loss? This need not be resolved at frontier scale to merit publication, but it should bound the claims about scalability and inexpensive steering.

Overall Assessment

Presentation: The architecture and evaluation are clearly explained.

I particularly appreciate the explanation of why training with target-derived features creates a useful dependency while context-derived features at inference do not provide the same information advantage. That makes the central design easier to assess.

Informativeness: The pretraining approach provides an interesting alternative to post-hoc steering.

Soundness: The threat model and the interpretation of the capability loss need a narrower account.

I lean toward acceptance because the training design is creative and conceptually coherent, and the controlled small-model results provide a useful proof of concept. My main concern is the strength of the practical claim. The model is more controllable on these tests, but also less capable. I would be more convinced by an advantage at comparable useful quality and by a clearer boundary around the attacks the paper covers.

I would not require this first demonstration to match the strongest available model on every capability benchmark. The creative mechanism and the controlled small-scale evidence can merit acceptance even with a substantial limitation. What matters is presenting that limitation honestly and showing that the steering effect remains useful within a clearly defined regime. I encourage the authors to distinguish that credible contribution from a general claim of tamper-resistant or universally robust control.

Questions for authors
  1. How does the advantage change under a stronger adaptive suffix search and repeated generation and attack seeds? Can blinded human ratings, repeated judge evaluations and a targeted attack on the evaluator confirm the result? Please specify what the attacker can access, especially the control channel.

  2. At comparable useful-task accuracy or response quality, how much control advantage remains? What do the existing ablations tell us about the separate contributions of conditioning, feature prediction, and splicing?

  3. How does this approach differ conceptually from a mixture-of-experts model whose router is conditioned on the same semantic features? Conditioning appears to encourage specialization across kinds of text, although your model shares decoder parameters rather than routing among separate expert subnetworks. Is the important distinction the shared representation, the separate control channel, or the training incentive to use it? Would a feature-conditioned MoE be expected to have similar control properties, or is there a mechanism specific to your design?

  4. How sensitive are the main conclusions to the selected steering strength and to the quality threshold at which the models are compared? Can the existing sweeps identify a region where control remains stronger without relying on a substantially less capable or less fluent operating point?

  5. What evidence separates the effects of next-segment feature prediction, splicing and the separate-channel architecture? For the MoE analogy, I am asking for a conceptual distinction rather than requiring a new large-scale MoE training experiment: what property of the shared conditioned decoder is essential to the claimed control mechanism?

2 comments

We thank the reviewer for their time and detailed feedback. We appreciate that the reviewer found the central design “insightful and creative” and the choice to make the channel valuable to prediction “the most compelling part of the paper,” and that they credited the two 400M comparisons and our reporting of absolute scores, fluency, and capability loss. We agree with the reviewer’s view of this work as a proof of concept for a learned control interface.

On capabilities, the response to all reviewers describes what we found and the updated results. We agree with the reviewer that the oracle result is a diagnosis rather than a capability result. We emphasize that this is the first implementation of this approach. We believe it has demonstrated the potential of the method, and we expect that further experimentation and attention, which this paper justifies, will determine how much of a capabilities drop is necessary. Our results have already shown substantial improvement, and we believe there is more to come to close the gap.

RE: Our threat model for robustness evaluations and the effect of stronger attacks

We apologize that our threat model was not clear in the initial submission and value this feedback. We will update the paper to fix this issue. As we stated in our response to all reviewers, our threat model is that the attacker controls input tokens but not internal activations, weights, or the steering signal. We have also added additional attacks to our evaluation. Beyond the original GEPA suffix search, we now train an attacker LM to write a 64-token suffix (RL red-teaming, Sec. 5.2, App. E) and give the attacker gradient access to optimize 4 continuous suffix tokens (soft-prompt attacks, Sec. 5.3, App. F). Mancelle-3B retains 97% [92, 102] of its concept score under the RL attack and 81% under the strongest white-box attack; the comparison to baselines is in the response to all reviewers. We do not claim robustness to attacks that write directly into the model’s internals (App. F.4, G).

RE: Attacks through the context-dependent feature predictions

This is a good observation and we agree it is a potential source of vulnerability: during self-conditioned generation, the features for the next segment are selected from αfsteer+(1−α)fnext\alpha f_{steer} + (1 - \alpha)f_{next}, where fnextf_{next} is predicted from context, so this is a potential failure mode. Our current evaluations do not directly evaluate this. We will clearly discuss it in the limitations section of our paper.

As for the strength and rigor of the adversarial robustness evaluations, we highlight the soft-prompting and RL red-teaming attacks we have added to the paper (Sec. 5.2-5.3). Our robustness evaluations were relatively expensive due to the amount of API calls and inference steps needed (especially the RL red-teaming experiment), so we rely on our population of steering concepts to provide us with confidence intervals.

RE: Validation of LLM judge annotations

Thank you for the feedback. We agree that this is important to address and will update the paper accordingly. Since the initial submission, we had three human annotators label 150 samples from our experiments according to concept relevance, instruction following, and fluency to measure LLM judge agreement. We find agreement rates of 60%, 53%, and 42% between the LLM judge and human annotators. For comparison, the inter-annotator agreement rate for humans was 54.4%, 35.3%, and 58.8%, respectively. Additionally, we observe a monotonic increase in the average human annotator rating as LLM judge ratings increase—increased LLM judge ratings correspond to increased human judge ratings. This gives us confidence that this methodology highlights semantically meaningful differences between steering methods. We will update the paper to discuss these results.

RE: Control at useful-task accuracy, ablations to isolate the contributions of conditioning, feature prediction, and splicing

We see competitive steerability against the much stronger Qwen2.5 and SmolLM3 baselines and much stronger steering robustness, as indicated in the answer to the previous question. For splicing, we have added an ablation to Appendix K where we train a 400M model using the existing recipe except for pretraining data splicing, showing a moderate but significant effect. For conditioning, we point to our baseline decoder-only models, which were trained with no conditioning signals, matching our model in number of total parameters and decoder parameters. For feature prediction, we do not have an ablation available, as this would require another sizable full training run.

RE: Signal-conditioned mixture-of-experts, and whether the important distinction is the shared representation, the separate control channel, or the training incentive

Our view is that the most important distinction between our method and feature-conditioned MoE routing is the usage of mutual information to bake steering into the next-token prediction objective. The conditioning information is passed explicitly rather than being reinforced implicitly via targeted weight updates to specific experts. However, we also believe the other two differences to be important mechanisms in our design. The separate control channel operates through one-way cross attention that ensures a consistent, unchanging set of conditioning KV vectors. We also have a larger space to optimize for these signals, as the conditioning information is passed to the whole network rather than a specific conditional weight subset.

RE: Sensitivity of model quality to steering strength

We find a drop-off at steering strengths below 23\frac{2}{3} for our method. Despite the high steering strength that our model is effective in, it retains strong fluency even at a steering strength of 1. Pythia is unable to steer effectively without losing capabilities; LoRA and prompting retain fluency but are weaker than our method, while steering vectors are stronger (but still significantly weaker than our method) and completely destroy its language modelling ability.

Reviewer_R57COpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores whether language models can be made more robustly steerable by incorporating control mechanisms directly into pretraining. The authors develop Mancelle that learns to generate text conditioned on SAE features. They show that this approach achieves strong steering results and substantially more robust to adversarial prompts.

Critical strengths
  1. The method is novel and well motivated. The design of SAE-derived semantic conditioning, architectural separation, and training objectives is coherent.
  2. The evaluation considers both steering effectiveness and robustness and the empirical results are strong.
Critical weaknesses
  1. Tradeoff with general model capabilities. Mancelle-3B achieves a ChatCORE score of 0.1998 compared to 0.3454 for Pythia-2.8B-SFT. When switching to ground truth conditioning, the performance improves a lot, which may suggest that the next segment prediction is the bottleneck.
  2. Dependence on SAE feature quality, and the preprocessing effort is high. The method relies on a curated vocabulary derived from pretrained SAE features, which may limit its coverage of behaviors. These makes scalability questionable.
Questions for authors

How sensitive is steering effectiveness to the coverage and quality of this vocabulary (844 in total)? Can model reliably steer toward concepts that are not well represented by those selected features?

2 comments

We thank the reviewer for their positive comments on how novel and well-motivated our method is, and on the strength of our evaluation of both steering effectiveness and robustness.

RE: Capabilities tradeoff from our method and next segment prediction

We thank the reviewer for this observation, which our investigation confirmed: Mancelle’s performance dropped when generating from its own predicted conditioning features. As described in the response to all reviewers, this was one of three issues behind the capability drop. To address it, we replaced 50% of ground-truth conditioning signals with features predicted by the base model during SFT. This trained the model for generation from its own predicted conditioning features. Our interventions recovered most of the prior performance drop (numbers in the response to all reviewers) and we take this as evidence that the decrease in capabilities is not due to fundamental limitations of conditional pretraining but rather an artifact of a new architecture/training recipe with substantial room for improvement.

RE: Dependence on SAE feature quality/vocabulary and scalability

The amortized data labelling cost for a given document using the 65k-feature SAE on Gemma3-4B is about half of the compute required for a training pass (forward + backward) on our 3B model. With our multi-epoch pretraining, data annotation ended up amounting to about 5-10% of the actual pretraining cost, or about 0.01% per final feature. The labelling itself is also reusable for future runs. In future work we intend to explore methods to distill the annotation into an efficient model to further reduce this overhead. We will update the paper to more clearly discuss this cost and frame it appropriately.

RE: Connection between vocabulary coverage and steering effectiveness

Our vocabulary is quite expressive — in fact, we chose a feature space derived from unsupervised SAE features to support a wide range of steering signals. Our feature space has high cardinality and variety (helped by our de-duplication/clustering) compared to existing conditional generation methods (the largest, CTRL, has 55 control codes) and this provides substantial expressivity through composition. For example, none concepts in our steering evaluations come are explicitly represented in our vocabulary. Instead, we take our steering directions from the benchmark and steer towards these concepts via vectors in our feature space. Our strong steering performance is evidence of the flexibility of our vocabulary.

I appreciate the authors' thoughtful response. After consideration, I will keep my score at 3.

Reviewer_MtpGOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Mancelle, a conditional pretraining framework for improving the steerability and robustness of language models. It introduces a separate semantic control channel using SAE-derived features, which are processed by an encoder and incorporated into the decoder through cross-attention. The method also uses data splicing and auxiliary feature prediction to encourage reliance on control signals and support autoregressive generation. Experiments on 3B and 400M models demonstrate improved concept steering and robustness against adversarial prompting and activation-based counter-steering.

Critical strengths
  1. Learning a dedicated control interface during pretraining offers an alternative perspective to conventional post-training and inference-time steering methods.

  2. The paper evaluates steering effectiveness and robustness across multiple baselines and includes a controlled 400M comparison to support its findings.

  3. Comprehensive analysis.

Critical weaknesses
  1. My main concern is that it is unclear why this paper focuses on steering rather than conditional text generation. Steering methods are generally lightweight, whereas the proposed method requires pretraining the model from scratch. Furthermore, the proposed method should be compared against a broader range of conditional text generation methods. Therefore, I believe the paper requires substantial rewriting.

  2. The proposed framework incurs substantial computational overhead, including pretraining a language model from scratch and a SAE-based feature extraction pipeline. However, the proposed models exhibit significantly degraded general capabilities compared to decoder-only baselines. It remains unclear whether these improvements justify the substantial computational costs and performance degradation.

  3. The Introduction makes overly dismissive claims about existing post-training methods, describing them as "putting lipstick on a pig" and providing only "surface-level tendencies" or "relatively shallow safety training." These statements are not sufficiently justified. Especially considering the degradation of general capabilities caused by the method, it may sometimes perform worse than post-training methods.

Questions for authors

see weaknesses

1 comment

We thank the reviewer for their valuable feedback, recognition of our work’s contributions, and the important questions they raise. In particular, we appreciate the recognition that learning a dedicated control interface during pretraining “offers an alternative perspective to conventional post-training and inference-time steering methods,” and of our evaluation across multiple baselines with a controlled 400M comparison.

RE: Focus on steering instead of conditional text generation and choice of baselines for comparison

As we state in the response to all reviewers, our aim with this work is: “Given a hypothetical model training run, how do alternative, early interventions in pretraining stack up against post-training interventions in control efficacy and robustness?” Our choice to use steering to evaluate pretraining interventions comes from a desire to avoid value lock-in and the adversarial nature of the setting. The goal is flexible alignment that withstands attacks from user-controlled context, focusing on steering robustness rather than just conditional generation without consideration for previous context.

Existing conditional generation baselines are unsuitable for direct comparison. CTRL (Keskar et al.) is only 1.63 billion parameters, was trained in 2019, and uses 55 control codes over restricted domains; pretraining a separate 3B CTRL model on our training stack just as a baseline would surpass our computational capacity. Instead, as an analogue to how SteerLM (Dong et al.) adds formatted language tokens to SFT data as conditioning signals, we ran a dedicated experiment representing conditional post-training on top of an existing chat model: we further finetune post-trained Pythia on our synthetic steering subset with SteerLM-style conditioning signals. We find weaker steering (Figure 2) and robustness (Figure 3, Figure 4) compared to our pretraining intervention.

We have also edited the related works section to highlight the distinction between our method and conditional generation more clearly.

RE: Computational overhead for our method

We view pretraining as a fixed cost. We are exploring the efficacy of an alternative intervention early in the training pipeline, and we do not propose re-training models from scratch to add conditional generation. We view the overhead as the extra annotation cost on the pretraining data. The amortized data labelling cost for a given document using the 65k-feature SAE on Gemma3-4B is about half of the compute required for a training pass (forward + backward) on our 3B model. With our multi-epoch pretraining, data annotation amounted to 5-10% of pretraining cost. In future work we intend to explore methods to distill the annotation into an efficient model to further reduce this overhead. We will update the paper to more clearly discuss this cost and frame it appropriately.

RE: General model capability in comparison to baselines

We agree that the capability of Mancelle is important. As described in the response to all reviewers, we investigated the performance drop after submission, and our updated results show improvements. While our results do still show a decrease in capabilities, we attribute this to the novelty of our recipe and architecture, and we expect that future iteration will close the remaining gap.

RE: Introduction tone

We agree that the tone is too strong for an academic setting — thank you for the feedback. We have edited it accordingly and re-organized it to more clearly justify our criticism or post-training methods for alignment. Our justification is that today’s language models are pretrained with next-token prediction, which relegates alignment and control to the final 1% of training tokens or inference-time interventions. These interventions construct a chat assistant on top of a base that was optimized for a different objective, and it should not be surprising when they prove brittle. This is validated by well-cited works observing failure modes stemming from adversarial attacks (Andriushchenko et al., 2025; Cunningham et al., 2026), benign post-training (Qi et al., 2024), and normal use (Lu et al., 2026). We apologize that our initial submission did not clearly connect these citations to our claims and have moved them to the first two paragraphs—where the claims are made. Furthermore, our updated robustness results provide additional direct evidence for our conclusions: the post-training baselines lose most of their steering under attack, while Mancelle-3B does not. Please see our overall response to all reviewers for the details.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to all reviewers

Our goal in this paper is to study the effects of alignment interventions during pre-training. In particular, we want to develop pre-training alignment interventions that preserve flexibility for a range of downstream applications. This is in contrast to post-training techniques (e.g., RLHF or steering), whose safeguards remain vulnerable to adversarial prompts, benign finetuning, and in-context drift (Sec. 1), and to early pre-training alignment methods that commit to a single alignment profile (e.g., Korbak et al. and Minder et al.). Our paper rests on the key insight that pretraining models with an explicit steering channel meets these criteria. Our contributions are as follows:

  • We provide a novel pre-training technique that uses data annotation and a new architecture to train a model that is steerable-by-design
  • We use this method to train Mancelle-400M and Mancelle-3B
  • We evaluate the steerability and robustness of these models.

We find that our models can be effectively steered with minimal decrease in instruction following or fluency and that they are more robust than other models of similar scale.

What reviewers found valuable. All three reviewers saw building control during pretraining as a meaningful alternative to post-training and inference-time steering. R57C found the method “novel and well motivated” with “strong” empirical results. 2ssD called the central design “insightful and creative,” singled out the training incentive to use the control channel as its most compelling part, and credited our two matched 400M controls and our reporting of absolute scores, fluency, and capability loss. MtpG noted the comprehensive analysis across multiple baselines. The main requests were a precise threat model and a clear account of the capability cost. We address both below and in the revised paper.

Robustness. Our threat model is that the attacker controls input tokens but not internal activations, weights, or the steering signal. Accordingly, we have added RL red-teaming (black-box, section 5.2) and soft-prompt optimization with gradient access (white-box as an upper-bound worst-case scenario, section 5.3). Against any of the four attacks, Mancelle-3B keeps at least 81% of its concept score, versus 32–41% for LoRA on all three open models and 19–23% for Prompt Steering on Qwen and SmolLM3. For example, the RL attacker breaks Prompt Steering on 37/39 steered concepts on Qwen but only 2/37 on Mancelle-3B. Direct attacks on internals fall outside this threat model, and we show they can remove most of the steering (App. F.4, G).

Capabilities. We traced most of our initial capability drop to three issues: (1) a bug in our SFT pipeline removed multiple-choice training data from Mancelle’s SFT but not Pythia’s; (2) free-text analysis showed that Mancelle often knew the correct answer but failed to map it to the answer letter; and (3) performance dropped when Mancelle generated from its own predicted conditioning features, as R57C suspected. We fixed the bug, added multiple-choice SFT data to map answers to their labels, and trained on self-predicted features. We also re-ran SFT on Pythia-2.8B with the updated dataset. ChatCORE improves from 0.1998 to 0.3170 (against Pythia’s updated 0.3383), while steerability goes from 1.38 to 1.31. As 2ssD suggests, we view the remaining gap as a control–capability tradeoff in a first demonstration of a new recipe. Finally, we would like to highlight that Mancelle’s decoder is 1.8B due to substantial parameters being allocated to the encoder—compared to Pythia’s 2.8B decoder.

Other changes. Revised introduction framing and tone (MtpG); splicing ablation (App. K, 2ssD); SteerLM-style conditional post-training baseline (MtpG).

References

Korbak et al. (2023). Pretraining Language Models with Human Preferences. https://arxiv.org/abs/2302.08582

Minder et al. (2026). Synthetic Persona Pretraining: Alignment from Token Zero. https://arxiv.org/abs/2608.13482

Dong et al. (2023). SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF. https://arxiv.org/abs/2310.05344

Keskar et al. (2019). CTRL: A Conditional Transformer Language Model for Controllable Generation. https://arxiv.org/abs/1909.05858

Andriushchenko et al. (2025). Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks. https://proceedings.iclr.cc/paper_files/paper/2025/file/63fa7efdd3bcf944a4bd6e0ff6a50041-Paper-Conference.pdf

Cunningham et al. (2026). Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks. https://proceedings.iclr.cc/paper_files/paper/2026/hash/4f697305ef1f868ad77c3c0027989a6f-Abstract-Conference.html

Qi et al. (2024). Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! https://arxiv.org/abs/2310.03693

Lu et al. (2026). The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models. https://proceedings.mlr.press/v306/lu26g.html

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.