acceptodds
Under review as a conference paper at ICLR 2027

Model Hypnosis: Strong control of AI via additive subliminal effects

Abstract

We demonstrate that AI models are broadly susceptible to a phenomenon we call _model hypnosis_, in which individually weak and seemingly irrelevant cues in the prompt can be systematically combined to strongly control model behavior. Model hypnosis occurs across model families and scales, including in frontier reasoning models, and hypnotic prompts can transfer between models. Because the model is controlled by inconspicuous textual choices, such as paraphrases and typos, model hypnosis presents new challenges and avenues for AI safety, and is a major hurdle for AI interpretability.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
subliminal, model hypnosis, semantic meaning, adversarial examples, robustness, AI control

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QkUdOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper claims to introduce a new phenomena in which careful changes in the input that seem inconspicuous lead to intriguing changes in the outputs.

Critical strengths

The paper is relatively easy to read and follow.

Critical weaknesses

The novelty and significance of the results in the paper are negligible and its abstract, intro and discussion vastly overclaim them. The paper offers a new framing and terminology for adversarial examples in LLMs which is based on anthropomorphization that is both misleading and creates a false impression of novelty. Adversarial example were heavily studied in LLMs, both instant and reasoning ones. A simple search in google scholar shows surveys that cite over 100 papers each (for example [1]).

The only place in the paper where the authors compare their findings to adversarial example is in section 1.2, however its justification on why a new term is needed is not convincing. In addition, in most of the paper the new term is used to describe what would count as a general adversarial example, and not one that was generated by the specific procedure suggested in the paper. For example in the abstract and in the first paragraph of the intro.

[1] Advancing adversarial and LLM robustness in trustworthy AI: a comprehensive survey (Tang et al. 2026)

Questions for authors

No questions for the authors.

Reviewer_a8XTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

the author describes models as susceptible to model hypnosis – where individually weak and irrelevant cues in the prompt can be combined to control model behavior.

Critical strengths
  • The motivation of the work and connection to existing cognitive phenomena (e.g. the Ericksonian approach to hypnosis) is very interesting.

  • The framework for identifying paraphrases is systematic and well-explained.

  • The author experimented with both reasoning and non-reasoning models and tested a wide range of models

  • The paper is generally well-written and the plots, tables, and examples are clear and well-presented.

Critical weaknesses
  • While the framework for generating families of cues with prompt templates is general, the author focuses on three narrow domains (a number preference, moral judgment, and philosophical question). It is unclear how the method would generalize to other QA domains in human-AI interaction.

  • The paper could consider applying mechanistic interpretability methods to examine which parts of the prompt (e.g. at a sentence or token level) led to the changed final response. For example, the authors could consider running additional ablation studies (e.g. by ablating sentences or tokens) to better understand the causal relationship between each component of the adversarial prompt and the final output probability. This will provide more insight into what exactly led to the different response in the model.

  • The paper can benefit from analyzing open-ended explanations for cot justifications for the answer (although models might not be self-consistent).

Questions for authors
  • Please double check the citation style. You should use \citep{} in overleaf.

  • Did you look at reasoning traces or model explanations for the changed answers besides looking at binary yes/no responses?

  • Did you test whether model hypnosis transfers between reasoning models?

Reviewer_9sfzOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces "model hypnosis", a phenomenon in which individually weak and seemingly irrelevant cues in the prompt can be systematically combined to strongly control model behavior. This means that text which does not look suspicious could result in adversarial outputs from the model. Thus, model hypnosis poses a real threat towards AI safety and puts emphasis on better alignment techniques and AI control methods.

Critical strengths
  • The problem is important: Coming up with methods which result in changes in downstream effects by altering inputs in a semantically equivalent manner, helps gain insights into how we can prevent adversarial states for LLMs.
  • The cue generation is simple and easy to implement: The method proposed to generate semantically preserving cues by treating the prompt as a set of slots which can be varied, is simple and easy to implement.
  • Linear fit to constraint search space: Fitting a linear model on random prompts and then extrapolating it to stack cues with aligned scores and using them to construct hypnosis prompts is a nice idea.
  • Breadth of evaluation: The evaluation spans 16 non-reasoning models across sizes from 3B to 72B, open and closed reasoning models at several thinking budgets.
Critical weaknesses
  • Limited Novelty: The core idea behind model hypnosis is semantically preserving modifications of inputs leading to variation in outputs of a model. This is present as a well established direction in literature: The 2023 works - AN LLM CAN FOOL ITSELF [1] and PromptBench [2] proposed character level (similar to this paper's "typo" type), and word level (similar to this paper's "paraphrasing" type) modifications to the input prompts which change the outputs significantly. Similarly, SECA [3] searches for semantically coherent rephrasing that cause change in the outputs. PAA [4] extends this idea to inflation of scores when asking LLMs to review research papers. Even though the authors state that their work exposes a different structure, it is very weakly justified.
  • No baseline search methods: The literature for the research direction is rich but the authors don't compare their cue generation method with any relevant baseline.
  • Isolation of cue level effects: The authors calculate the effective number of cues that induce an effect but provide no ablation on the cue/slot axis to isolate the cue level effects.
  • Lack of study of mechanisms: The abstract frames hypnosis as a major hurdle to interpretability but the paper provides no mechanistic analysis of hypnosis and doesn't provide any defense mechanisms against the issue.
  • Anthropomorphizing: The human hypnosis analogies are welcome but risk anthropomorphizing and overstatement of the effect observed.

References -

[1] AN LLM CAN FOOL ITSELF: A PROMPT-BASED ADVERSARIAL ATTACK

[2] PromptBench: A Unified Library for Evaluation of Large Language Models

[3] SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations

[4] Paraphrasing Adversarial Attack on LLM-as-a-Reviewer

Questions for authors
  1. Can you justify the contribution of the work compared to the literature as stated in the weaknesses?
  2. How do you establish that cues are irrelevant to the target question? Can you report a human or LLM-judge relevance rating for each cue, and the correlation between cue scores and semantic association with the answer.
  3. Can you add baseline methods to compare the cue search method with?
  4. Can you provide ablation on the cue/slot axis?
  5. A mechanistic analysis would help frame the challenges to interpretability more formally. Can you provide that?
  6. How persistent is the effect? Does random paraphrasing or perturbations to a small set of cues remove the effect?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.