acceptodds
Under review as a conference paper at ICLR 2027

Does My LLM Obey Me? Falsification-Based Attribution Tests for Prompt-Based Behavioral Conditioning

Abstract

Validating the effectiveness of a behavioral specification just by tracking the change in the behavior of a large language model (LLM) relative to a baseline can be deceptive, as the changes may arise from components of the specification other than the intended ones. To address this issue we propose four falsification-based attribution tests - misassignment, reversal, content-irrelevant control and attribute name substitution. We apply these to a behavioral policy system for emotional support dialogue and show that it passes its baseline test decisively. The fraction of response sentences that are considered advice goes down from when the policy is added with a low value of the "advise" parameter . On Qwen2.5-3B, with partial replications on Phi-3.5-mini and Mistral-7B, the central comparison survives independent human annotation of 40 conversations with a bias-corrected reanalysis. Renaming eight attributes without changing their values raises advice proportion by relative to the original names. Inverting all eight values (turning low values to high and vice versa) leaves it indeterminate under human annotation as under a classifier . Their within-conversation difference is . The conclusion remains unchanged under blind adjudication. The pattern is not specific to advice, and the same ordering appears on reflection. Relative sensitivity to the tested interventions is configuration-specific. On a non-counseling counted-outcome task, changing the value of "verbosity" from increases response length . Verbal encoding of the values as qualifier words instead of values (advise: becomes advise: rarely) partially rescues value sensitivity on advice . We investigate the relative name/value sensitivity changes across Qwen2.5 model configurations and prompt placements. We also find evidence against assuming that a specification validated in one configuration remains valid in another, and release the tests as "speccheck".

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
behavioral specification, prompt-based conditioning, attribution tests, falsification, instruction following, LLM evaluation methodology, emotional support dialogue, validation portability, prediction-powered inference, human annotation audit

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_e6tLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether structured prompts genuinely control LLM behavior through their specified parameters. The authors argue that conventional baseline comparisons can demonstrate changes in model behavior but cannot establish whether these changes are driven by the intended control variables. To address this limitation, they propose four falsification-based attribution tests: policy misassignment, parameter reversal, content-irrelevant controls, and attribute-name substitution. They also introduce speccheck, a tool for evaluating the effectiveness of prompt-based behavioral conditioning.

The authors primarily evaluate their approach on an emotional support dialogue task using Qwen2.5-3B and 119 conversations, measuring the proportion of advice-related sentences in generated responses. Adding a behavioral specification reduces the advice proportion from 38.6% to 24.5%. However, reversing parameter values produces an inconclusive effect, whereas replacing attribute names leads to a substantial behavioral change. This relative difference is further supported by independent human annotations of 40 conversations. Additional experiments across model sizes, prompt placements, parameter representations, and a response-length task suggest that behavioral control is configuration-dependent and cannot be reliably validated through baseline comparisons alone.

Critical strengths

Practical relevance. The paper highlights that conventional baseline comparisons cannot establish whether changes in LLM behavior are driven by the intended control parameters. The proposed falsification-based tests, although relatively straightforward, provide a practical framework for identifying unreliable behavioral control in structured prompts. This has potential value for evaluating the reliability of prompt-based LLM systems.

Empirical support for the main findings. The authors conduct paired experiments on 119 conversations and further validate their central findings through independent human annotations of 40 conversations and blind adjudication. Additional experiments across model sizes, prompt placements, and parameter representations provide evidence that behavioral control is configuration-dependent. These evaluations strengthen the credibility of the main empirical observations.

Critical weaknesses

Construct validity concerns in the core experiments. Although the paper specifies numerical control parameters such as advise: 0.10/0.90, the behavioral interpretation of these numerical levels remains insufficiently defined. Consequently, the weak sensitivity to numerical values may reflect ambiguity in the chosen prompt representation rather than a general failure of numerical behavioral control. This limits the interpretability of the central experimental findings.

Limited methodological novelty and explanatory power. The four proposed falsification tests largely combine conventional control experiments and prompt perturbation techniques, offering limited methodological innovation. Although the authors examine synonyms, nonsense labels, and translations, these interventions still modify multiple linguistic properties simultaneously, making it difficult to identify the specific factors responsible for the observed behavioral changes. Consequently, the framework's explanatory value beyond existing prompt robustness evaluations remains insufficiently demonstrated.

Questions for authors

The paper does not explicitly define how numerical parameters such as advise: 0.10/0.90 should correspond to actual advice-giving behavior. How do the authors distinguish between the model's failure to follow numerical instructions and insufficiently specified prompt semantics? What evidence from the existing experiments supports this distinction?

Reviewer_q6BrOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes four attribution tests: misassignment, reversal, content-irrelevant control and attribute name substitution. The authors find that renaming attributes in the specification has a larger effect than inverting all of the values at once

Critical strengths
  • The four attribution tests are interesting and each cover a distinct area
Critical weaknesses
  • The author only uses one dataset (i.e. the emotional support dialogue). It’s unclear how the conclusions generalize to other domains.
  • The descriptions of the four falsification-based attribution tests are not very intuitive. It would be helpful to include a table in the main text that shows an example of each.
  • The paper is hard to read throughout. The authors did not provide enough context for the reader to understand what the tasks look like and go straight to reporting the results without defining key terms. The paper will need significant revision for it to be more readable.
Questions for authors
  • Did you experiment with bigger commercial models outside of the Qwen and Phi model families?
Reviewer_ohrhOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the author identified an issue that the change in the model may come from different sources, rather than only the intended specificiation, the author tested it in terms of misassignment, reversal, content-irrelevant content and attribute name substitution.

Critical strengths
  1. This paper identifies the issue of order following ability of LLM, that the changes do not necessarily come from the intended perspective.

  2. The four validation perspectives reveal some insights into the problem. For example, the structured policy overall helps but may not be enough to control the individual attribute.

Critical weaknesses

Overall I am not very suprised that the model is not following the policy parameters, because it is in nature a language model and prompt specification cannot be guaranteed. The misassignment, reversal, content-irrelevant content and attribute name substitution are good angles to analyze it, while I am not seeing a solid method to improve this problem.

Also, the findings seem to be domain dependent, and lack of anaysis on the generalizatiom.

Questions for authors
  1. For your attribute-name substitution, do you permute or give new attributes, and how do you think it would affect your results.

  2. Is there any way to resolve the issue for the problem that you identified in this paper. It seems that the structured policy still helps in a global way, and what are your thoughts on individual control.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.