- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces constitutional adapters (CAs) trained on synthesized corpora with a differential idea to substract from a controlled text group. During inference time, CAs can be applied with a tunable strength to achieve zero-shot transfer for improving the jailbreak defense, especifially for long padding and multi-turn attacks. The experimental results show the proposal effectively defend the attack and is more effective an system prompts and simple refusal steering ,demonstrating a better trade-off between over-refusal and general capability. The authors position CAs as a portable inference-time lever against both misuse and misalignment.
Critical strengths
-
The investigated alignment problem is important and the inference-time lightweight approache might be more practically relevant for API deployments. Framing constitution-derived behavior as a small, transferable adapter is a sensible systems goal.
-
The idea of of training CAs using matched control corpus + task-arithmetic / activation difference, is interesting. The paper shows control-only objects can hurt safety while difference objects help. Combined with long-context padding and multi-turn evaluations, this yields the paper’s most convincing empirical story.
Critical weaknesses
-
Claude-centric pipeline bias is under-examined relative to the generality claimed. Corpus generation, rewriting, filtering, and much of the judging stack sit inside the Claude family (Opus/Sonnet/Haiku). Gains may partly reflect transferring Claude-like stylistic and preference patterns into open models, as Claude was the most aligned models as stated by the author, rather than a portable “constitution” signal. Without stronger cross-judge, cross-generator, or non-Claude constitution controls, it is hard to justify that the pipeline is generalizable beyond this ecosystem.
-
The title claim that CAs jointly mitigate misuse and misalignment is overstated. Jailbreak / attack results are interesting and comparatively well supported. Misalignment evidence (e.g. eval awareness) is narrower, judge-dependent, and does not warrant equating “lower scores on these probes” with robust misalignment mitigation.
Questions for authors
- It is nice to have cross models results such as Gemma and Qwen; Will the corpus generated by a less aligned models also shows the similar effect to sufficiently defend the attack?
- Will the datasets be released for future studies?
Response to Reviewer ku5q
Thank you for the useful comments, and for noting the importance of our problem setting and the interest of the matched-control idea.
W1 – Claude-centric pipeline. Reviewer wnJa shared this concern. As noted in the general response, we have replaced all evaluation judges with non-Claude models (MiniMax-M3 and GLM-5.3), and the paper's relative conclusions largely hold. Every judged figure and table in the manuscript has been updated accordingly. The control corpus was written by the same Claude pipeline in a matched register, yet objects trained on it weaken static and multi-query defense (Figures 3–4), so Claude's style alone does not produce the effect. As a cross-generator control, a constitution corpus written entirely by a non-Claude model still raises mean defense in every attack family, though less (Q1 below). The new Limitations section also acknowledges that Claude wrote our corpora, based on Claude's constitution.
Revision: Sections 2.4, 3 and 5.1; Appendices D.7 and D.8
W2 – Misalignment results are less convincing. Our misalignment evidence rests on Petri (we removed the agentic evaluation; see k73x W4), where constitution and especially difference objects raise the alignment score substantially (difference ReLU steer +3.4 points on the 10-point scale), whereas the safety and constitution system prompts add only 0.9 and 0.3 points (the latter not significant). These effects change little if at all under the original Claude judge (Appendix D.7), showing judge-independence and robustness.
We agree that automated audits like Petri are not perfect, particularly at frontier scale, given evaluation awareness as well as what are likely to be more severe, out-of-distribution incidents (such as OpenAI-HuggingFace). However, at the 4B scale that is our focus here, we believe that Petri still provides useful signal, so we have kept the title. We are not aware of better methods for probing alignment at this scale, and we believe that in practice they have been useful for bootstrapping more aligned models (reported in the Claude 4 System Card). We have therefore elected to keep the title claim, and we hope that the additional judge results convince this reviewer that such a claim is warranted.
Revision: Sections 3.3, 5 and 5.1; Figure 5; Appendix D.6 (Figure 14)
Q1 – Corpora generated by less aligned models. This is a very interesting question. We suspected that the answer would be yes, given the multi-stage review pipeline (Figure 1) and the fact that during generation, constitutional information resides in the prompt and not simply the weights. However, generating these 40M+ token corpora is an expensive proposition (more than USD 10k), and restarting the whole pipeline from scratch was infeasible in the allotted review period. We therefore attempted a smaller-scale experiment, generating a constitution corpus with Qwen3.8-Max and the identical filtering pipeline, and training on 1M tokens of it.
Constitution LoRAs trained on the Qwen-written corpus still raise Qwen3.5-4B's mean defense in every attack family, though less than LoRAs trained on the same amount of Opus-written text: they raise defense success by 7.0, 9.5 and 8.6 pp on static, multi-query and multi-turn attacks, against 14.7, 38.0 and 23.3 pp for the Opus-written LoRAs, with less over-refusal and no capability difference. Part of the defense therefore does not depend on a Claude model having written the corpus, but the generator affects its size, at least through a pipeline tuned on Opus output.
Revision: Appendix D.8 (Figure 17), with pointers in Sections 2.1 and 5.1 and in Appendix A
Q2 – Will the datasets be released for future studies? Yes. The Reproducibility Statement says that all code and synthetic data on which the results depend have been uploaded to GitHub. The link is not provided yet to maintain blinding but will be included in the final paper upload.
Please let us know whether these responses address your concerns. If they do, we’d be grateful if you would reflect that in your assessment. If not, we’d welcome the chance to respond to anything outstanding before the discussion period closes.