acceptodds
Withdrawn from ICLR 2027

How to Remove Subliminal Signals?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Subliminal signals are subtle statistical patterns in model-generated data that can transmit information about the teacher even when individual examples appear benign. We use the *teacher state* to denote any source-specific factor that affects the teacher's generation distribution, such as a behavioral trait, system prompt, or training or inference-time intervention. We study how to remove information about this state when the state itself is unknown, no clean reference model is available, and the downstream training procedure is not under our control. Our approach uses verified task-preserving transformations that change the representation of an output without changing its task-relevant content. We introduce *Self-Sanitized Generation (SSG)*, a data-side sanitization procedure that uses only samples and response probabilities from the original teacher. SSG removes state information at the distribution level, including signals that survive ordinary rewriting. Under a standard exponential-tilting model in which the teacher state reweights the probabilities of generated outputs, the sanitized distribution becomes exactly independent of the state. We further characterize when exact cancellation is possible beyond this model and quantify the remaining state dependence when exact removal cannot be achieved. Finally, we validate SSG on a storytelling task in which the subliminal signal is a preference for a specific animal, injected via system instructions on Qwen3.5-9B, with paraphrasing as our task-preserving transformations.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Subliminal Learning, Distillation, AI Safety

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_eSwWOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The present theoretical claim and empirical results about how their SSG technique removes the subliminal learning effect. Description of SSG:

  • Generate a sample from the teacher
  • Generate N rephrasings
  • Get the logprobs of these N rephrasings using the teacher
  • If the N rephrasings are similar enough in their logprobs, keep the sample from the teacher; otherwise, loop back to the start
  • Then generate a new final rephrasing of the sample and train on it
Critical strengths

Strong positive points

  • Target a real problem: subliminal learning
  • Theoretical results may be sound or useful (this remains uncertain)

Weak positive points

  • The decomposition of the subliminal learning problem into within- and across-orbit effects
  • SSG seems like a simple technique
Critical weaknesses

Strong negative points

  • The CI in their empirical results is misleading. It appears to be computed with 50x50 data points, assumed independent, while actually there are only 50 independent prompts (and 50 orderings). Their real CI could be several times larger than reported, and they don't precisely describe how they compute their CIs.
  • The "validation" through empirical results is misleading. The empirical results show very weak effects, with no meaningful baselines and a single setup (1 teacher, 1 student, 1 eval, 1 random seed). The abstract and paper should not claim that SSG is "validated" through empirical results; instead, they should report a single preliminary experiment with null results. E.g., they observe a reduction in subliminal learning from 82% to 77%, but their CI ranges are only separated by 1%, while they underestimate their CIs by several times.
Questions for authors

Additional things to clarify or fix:

  • The paper seems to introduce SSG as a solution while the empirical results are close to the null effect (a single preliminary experiment with a small effect), and it is impossible to evaluate relative to other techniques since they are not included.
  • The "AI DISCLOSURE" section does not seem to contain all the required items listed in https://iclr.cc/Conferences/2027/AIPolicyForAuthors
  • Clarify the cost of the SSG in the contribution or abstract. If it multiplies the inference cost by x16 or more (e.g., when using 15 transformations), then this is important to know.
  • Describe SSG in simpler terms early in the paper
Reviewer_QR73OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to remove subliminal signals from the teacher's outputs through Self-Sanitized Generation (SSG). The proposed method rewrites the teacher's outputs while preserving task-relevant information, and samples with normalized geometric mean probabilities of the teacher. It requires no prior knowledge about what the subliminal signals are. The authors theoretically show that a simple rewrite with uniform sampling removes subliminal signals carried by choices among equivalent forms of the same context, but not signals from which content is chosen. They thus propose sampling from the geometric mean with a rejection-sampling implementation. Finally, the paper shows that the proposed method slightly reduces subliminal learning in a storytelling setting.

Critical strengths
  • The setting is interesting and underexplored: can one remove subliminal signals from the teacher without knowing anything about the traits?
  • The theory is interesting and seems correct.
Critical weaknesses
  • The empirical result is very weak. The proposed method reduces the preference for owls by only 5% (from 82% to 77%), which basically fails to remove the subliminal signal. In a paper called "How to Remove Subliminal Signals?", which seems to promise a strong defense against subliminal learning, the authors fail to remove the subliminal signals empirically.
  • The empirical result also misses the uniform sampling baseline. The authors do not show SSG is better than rewriting with uniform sampling, which is one of the paper's main theoretical results.
  • The proposed method is very expensive in practice, requiring several rewrites, recomputing probabilities, and additional sampling.
  • The model used for rewrites might introduce its own bias and subliminal signals.

Minor:

  • The term “representation” is a bit confusing to people in the LM interpretability literature, where it generally refers to the internal activations. Here, the authors use it to denote different formats of the same content of a response for a task.
Questions for authors
  • How well does rewriting with uniform sampling perform at removing subliminal learning?
  • What is the acceptance rate for the rewrites empirically?
Reviewer_8BWVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the problem of information removal on a hidden teacher state from generated data without a clean reference model. Given a set of task-preserving transformation, the paper proposed self-sanitized generation (SSG) which samples from normalized mean of teacher's probability over each orbit, using rejection sampling with uniform rewriting. Theoretical results on state independence and KL distance are provided under an exponential-tilt model with a balance condition. In experiment, SSG lower the teacher preference by 5 per cent.

Critical strengths
  1. The problem is relevant with a clear motivation in privacy, intellectual property, etc.
  2. The theory is elegant and the paper is clearly written.
  3. The idea of split into orbit-level and with-orbit information gives a clear explanation on why rewriting alone cannot remove the signal and the proposed method is simple and straightforward to understand.
Critical weaknesses
  1. The methodological novelty is incremental. The method applies existing tools to a new problem. The construction of P^\hat{P} is logarithmic pooling (Genest & Zidek, 1986), the KL-projection characterization follows classical results on information projections (Csiszár, 1975), and Algorithm 1 is standard rejection sampling with no algorithmic novelty. The novelty therefore lies mainly in the application.

  2. The key assumption is strong, especially the balance condition. The balance condition (3) in Assumption 1 requires the orbit-averaged effect of the teacher state to be the same for every orbit in Yx\mathcal{Y}_x. In effect, the state may change how a response is presented but not its task content. This is a strong assumption, and it would help if the authors gave examples that satisfy it. In particular, the paper gives two motivating examples: Figure 1 and the prime pair in Section 2.2. Do these examples satisfy the balance condition? The balance condition should also be stated more prominently in the abstract and the introduction.

  3. The experiments are limited and do not match the setting of the theory. The paper has a single experiment, in which SSG removes only 5 of the 67 percentage points of transfer. Several questions remain open:

    • Since there is no paraphrase-only baseline, even this modest improvement may come from uniform rewriting (the final paraphrasing step) rather than from the rejection step.
    • Are the LLM-generated paraphrases uniform over an orbit, as the analysis assumes?
    • Does the paraphraser use the owl system prompt? If so, the transformations are not independent of the state, as Section 2.1 requires.
Questions for authors

Please see weaknesses for questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.