acceptodds
Under review as a conference paper at ICLR 2027

From Behavior to Rules: Discovering Executable Constitutions for Language Models

Abstract

When model providers do not disclose the operational specifications governing refusals, auditors must infer their scope from black-box responses. We study behavioral constitution discovery: recovering a compact natural-language program that a separate surrogate model can execute to reproduce an unchanged source's refusal behavior. We measure fidelity through reconstruction error, the fraction of held-out prompts on which source and surrogate answer-or-refuse decisions differ. Our method actively searches for informative prompts and uses all four joint behavioral outcomes to induce rules. Source refusals supply positive evidence, including boundaries already shared by the surrogate, while source answers constrain each rule's allowed scope. Each candidate is tested with and without inclusion in the complete constitution on boundary and stress probes. Acceptance requires local improvement or non-worsening behavior supported by joint refusals, together with preservation of protected examples and non-increasing validation error. This permits explicit recovery of constraints that produce no immediate disagreement, while testing whether separately induced rules remain valid when composed. Experiments across within- and cross-family model pairs indicate that compact constitutions can achieve low reconstruction error and encode refusal boundaries omitted by disagreement-only discovery. When applied to a fixed surrogate, constitutions discovered for different sources induce source-specific refusal boundaries, demonstrating that the recovered rules capture distinct source behaviors. These findings support executable constitutions as concise, inspectable approximations of provider behavior whose fidelity can be tested without access to the original policy text.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, behavioral auditing, constitution discovery, natural-language rules

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_BmwiOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper describes work called behavioral constitution discovery where the authors infer / extract / discover the constitution (or at least a compacted version) of a given model to understand how they govern a model’s refusal to specific (harmful) prompts. The discovery process involves use of public topics to guide prompt generation and testing this to models being investigated through a series of comparing decisions, proposition of rules, testing, filtering of passing rules. Results show that the proposed behavioral constitution discovery seems to outperform existing rule discovery methods (ORPO, ProTegi, Natural Refine) in terms of surrogate refusals and reconstruction errors after a number of iterations on 240 prompts.

Critical strengths

The paper tackles an interesting problem of extracting or discovering constitutions or learned rules that guide a model’s refusal to harmful prompts to capture the developer’s intent. The authors test their proposed method of extracting / approximating the constitution of a model through a surrogate model setup with open-weight models. There might be some benefits in this line of work but the paper does not clearly motivate this.

Critical weaknesses

The first thing that prevents me from fully understanding the paper’s contributions is that the writing very much needs a significant overhaul from being AI-generated which is very obvious to me and to anyone who will read the paper. There are so many unusual jargons and terminologies that occur at every instance without proper context such as “batch-start constitution” “protected memory preservation” “overbreadth” “constitution-conditioned surrogate” “source–surrogate agreement” “discovery-time source queries” and many more. While I reviewed the authors’ acknowledgement that AI has indeed been used majorly in writing, I am inclined to believe that there had been minimal effort to write the core content of the paper. It’s very hard to grasp what the authors wanted to say and hard to differentiate which were the authors’ main idea vs. what has been AI generated from this work.

I am not sure what goal is from the proposed reconstruction process of an unseen constitution and the paper seems unsuccessful in clarifying this to readers and emphasizing its utility. After reconstructing a (potentially) small part of an off-the-shelf model’s constitution what happens after this? What is the extracted constitution supposed to look like? Is there even a guarantee that the approximated refusal patterns meant to reflect the constitution is semantically or structurally correct? If the tested models refuse basic harmful prompts and an arbitrary constitution-like text is drawn from this, is that even surprising? Most models must refuse harmful prompts, exercise care in giving financial advice, avoid giving suggestions to deceptive practices, etc so I also do not understand the uniqueness of the example topics the authors are investigating because they are not properly motivated in the paper.

Given the two issues above, I’m convinced with the current results and how it can be useful to readers. In terms of technical rigor from the results, it also seems lacking. What was the basis of using these select models and why only three pairs? Why did the authors not do the proposed constitution method with frontier models like Claude or OpenAI knowing that Anthropic/OpenAI have openly publicized versions of their constitutions online? Did the authors use the actual constitution text for each of the models they evaluated? These are basic questions expected by readers to be made very clear in the paper.

Questions for authors

See questions raised in the weaknesses section and address them.

Reviewer_A9pyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

what I understood is that they learn a rule list to put in the second model's prompt. A third model writes candidate rules, and they test those rules by seeing whether the second model now answers or refuses the same reques ts as the first. Since I am not familiar with this, I am not sure to what degree this method is novel or useful in practice. as far as I understand, a "constitution" is simply that list of written rules. "Executable" means another model receives the rules as instructions and follows them. however, the second model brings its own behaviour and interpretation of the instructions. For me, the strongest motivation is the auditing benefit. The paper still needs to show that its rule lists make behaviour easier to understand and predict, and that its method discovers better lists than simpler alternatives, which I am not familiar with. My main concern is whether the proposed method learns better rules than the alternatives. Only a few test cases separate the methods. The rule counts also disagree across sections, and the comparisons do not account for all the queries used to learn and check the rules. I currently recommend weak rejection.

Critical strengths
  1. When a model's refusal policy is unavailable, auditors could read the learned rules and give them to another model to see whether it answers and refuses the same requests. The authors also check each new rule together with the existing rules, including on requests the original model answers. This helps catch rules that make the second model refuse requests the original would answer.

  2. The authors keep one model fixed and give it rules learned from Qwen or Llama. It matches Qwen's refusals better with Qwen's rules, and Llama's refusals better with Llama's rules. However, Qwen and Llama disagree on only 12 test requests, so the evidence is limited.

Critical weaknesses
  1. Only a few test cases separate the methods. The same helper model writes the rules, generates the test requests and decides whether responses count as refusals. If it gets a few of those decisions wrong, the ranking could change. Please have independent reviewers check the cases where the methods differ, then test on a larger set of requests written by people or another model.

  2. Table 3 reports 15 rules, nine of which have a match in the version that learns only from disagreements between models. The text and Table 5 instead report 11 rules with nine matches. That leaves either six or two rules missing from that version. Please correct the counts and show which rules came from examples where the models agreed.

  3. Some methods may receive more information from the original model than others. The budget leaves out extra questions asked while checking and revising rules, even though those answers can help improve the rules. The authors acknowledge this. Please count these questions and check whether the proposed method still does better when every method can ask the same total number of questions.

Questions for authors
  1. Does your method still make fewer mistakes if independent reviewers check which responses count as refusals? Please focus on the cases where the methods differ, compare all methods on the same requests, and give an uncertainty interval for the accuracy difference.
  2. Which rule count is correct, and which rules are missing from the version that uses only disagreements? Please show the examples that led to those rules. Can you also test separately what is added by examples where both models answer and examples where both refuse?
  3. Does your method still make fewer mistakes and produce shorter rule lists when every method can ask the original model the same total number of questions? Please include questions asked while checking and revising rules.
Reviewer_1RajOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the problem of reconstructing the refusal behavior of black-box LLMs by discovering a compact set of executable NL rules. The authors propose a framework that uses a utility model to actively generate test prompts, compare the refusal decisions of a source model and a surrogate model, and iteratively discover and refine behavioral rules based on four possible outcomes: source-only refusal, surrogate-only refusal, joint refusal, and joint answer. Candidate rules are evaluated through boundary testing, stress testing, and validation checks before being added to the constitution. Experiments on three source-surrogate model pairs show that the proposed approach achieves lower reconstruction errors than the evaluated baselines while producing relatively compact rule sets.

Critical strengths
  • The paper studies a meaningful problem in LLM safety auditing: understanding the refusal boundaries of black-box models without access to their underlying safety specifications.

  • The proposed approach combines active prompt exploration, rule induction, and iterative validation into a systematic pipeline.

  • The results show that the framework achieves lower average reconstruction errors and produces smaller rule sets than the evaluated baselines

Critical weaknesses
  • The proposed framework is largely built upon existing techniques (active probing, LLM-based rule induction, counterexample-guided refinement, etc.). While combining these techniques into a unified pipeline is useful, I am not fully convinced that the paper introduces a sufficiently distinct technical contribution.

  • The discovery process mainly relies on a fixed number of iterations, without a clear understanding of whether or when the recovered rules converge toward a stable approximation of the source model's refusal boundaries. More importantly, the NL input space is infinite, whereas the framework organizes discovery around a limited set of generated topics. It is unclear how well this "topic"-driven approach can scale or generalize to unseen topics and more complex requests involving multiple behavioral boundaries.

  • Although the experiments include both within-family and cross-family model pairs, the rationale behind these particular combinations is not sufficiently explained.

Questions for authors
  1. What would the authors consider the key technical innovation beyond integrating existing active probing, rule induction, and prompt optimization techniques? How does the proposed approach fundamentally differ from prior work on inverse constitutional inference?

  2. How do the authors determine that the discovered constitution is sufficiently complete or stable? Given the enormous input space and the limited number of explored topics, what evidence supports generalization to unseen topics or cross-topic requests?

  3. What motivates the specific source-surrogate pairings in your exp. design? (e.g., Qwen only to Qwen, and a less powerful model as surrogate by default?) Would the same source model lead to similar discovered constitutions when paired with different surrogates? And how sensitive are the results to the choice of utility model?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.