acceptodds
Withdrawn from ICLR 2027

I-State: A Self-Regulating Controller for Transformer Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A language model's resistance to manipulation can be measured, but it cannot be certified. We built a controller whose off-state we could prove exact, then found it can never reach that state — and closed the gap. I-State attaches to a frozen model a typed state on three timescales: a protected core the conversation cannot edit, a mood that decays at a fixed rate, and a memory of sustained pressure. It reads the model's own activations and sets its own strength — no dial, no human in the loop. Because one gate multiplies every trainable term — a FiLM modulation, a low-rank adapter and a refusal direction — a calm model is the base model on identical tokens: every measurement is a per-example counterfactual, not an estimate. On a benchmark we built for multi-turn persuasion against non-harmful deployment policies, the model holds its policy .25 to .52 more often than the frozen base across two model families from 1.5B to 14B, over bases holding .11 to .36. Removing the dynamics alone costs a third to two-thirds of that gain across three retrainings. The affect map has trivial kernel, so no readable input drives the gate to zero: uncorrected it idles between attacks, at 2.4 MMLU points. Re-centring the origin returns benign behaviour to the base model's own rate at unchanged violation, without retraining, and brings the deployed cost to 0.1 MMLU points — against 2.8 for the best always-on constant. An off-state a human sets is a switch; only one the dynamics reach is a policy.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI safety, controllable generation, activation steering, adapters, causal attribution, inference-time intervention, self-regulating systems

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_bJewOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a self-regulating controller. There is a probe that reads the activations, updates states, and gates interventions. It also studies the controller dynamics and claims that algebraically exact off-state cannot be reached, before trying to close the gap. There are experimental results demonstrating some benefits of the proposed controller on a generated synthetic benchmark.

Critical strengths
  • The paper studies an important area of research and is trying to propose a controller strategy that helps the frozen LM stick to the policy.
  • There are interesting analysis of the controller dynamics and the discoveries about the unreachable null.
Critical weaknesses
  • The paper is very poorly written. It is dense with weird breaks and terms right from the abstract till the end of the paper. The quality of the writing is severely impacting the readability of the paper and I believe it needs a complete re-write to ensure the community can parse the paper properly.
  • The paper over claims a bunch of things, right from the beginning of the paper: "Every inference-time intervention on a frozen language model—a steering vector, an adapter, a rerouting objective—is deployed at a strength a human fixes in advance. " This is completely a false claim.
  • The authors try to close the unreachability gap but seems limited to 1.5B model.
  • The experimental results are quite mixed and unconvincing. In line 459, the paper explicitly says that a constant dose attains lower violation at lower benign cost, suggesting the dynamic controller is not specifically useful.
  • There is no results with relevant baselines like CAST, llm-steer-instruct, etc
  • There is no sufficient statistical reporting and the paper also acknowledged this...
Questions for authors

NA/ see above

Reviewer_v7tROpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I tried reading this paper, and I cannot understand what it is about.

Critical strengths

None

Critical weaknesses
  • The paper is incomprehensible.
    • Honestly, this paper reads as entirely AI written to me (while I sensed this subjectively, I also checked the abstract in Pangram, which returned a rating of 100% AI-generated); the abstract and introduction fail to clearly or plainly explain what the paper is about in, while using fancy terminology and flowery phrases.
Questions for authors
  • The declared usage of LLMs includes only minor writing help: "prose was polished throughout—tightened, de-duplicated, and the abstract revised across drafts—with all scientific content and framing the authors'.". Upon reading this manuscript, I suspect that it was more than simply "polished" by an LLM -- I ask the authors to please honestly state and clarify the usage of LLMs in the construction of this manuscript.
Reviewer_tzEeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

I-STATE adds a probe, state dynamics and gated adapters to a frozen LM. Its exact off-state is unreachable under the original dynamics. HOLD experiments show increased hold rates but a negative comparison against constant doses. The paper diagnoses this behavior and introduces conjunctive injection and re-centered dynamics as distinct corrections.

Critical strengths

S1. The pre-registered comparison candidly reports a negative result: six constant doses beat the autonomous arm on violation and benign cost (Sec. 5.6). This is useful evidence beyond an on/off gain.

S2. Proposition 1 and benign-input measurements explain why an algebraically exact off-state need not be reached in deployment. The distinction between correcting dynamics and correcting intervention gating is a concrete design lesson.

Critical weaknesses

W1. Hold improvement is not yet clearly tied to substantive policy protection. On HOLD v1.1, the shipped arm raises hold from 0.357 to 0.607 while violation remains 0.307 (Table 12), a limitation the authors acknowledge. The outcome definitions, NLI scorer and LOOSE-mode aggregation need clarification. Keyword refusal ATEs also do not automatically measure content-level protection. Conjunctive injection reports violation 0.291, but without uncertainty; post-attack benign pass remains 0.721 versus 0.966 for the base. Its benefits should therefore be assessed separately from the shipped variant.

W2. Stateful control is confounded with intervention strength. FiLM and LoRA use σg\sigma^g, whereas the reported product aperture σgσμ\sigma^g\sigma^\mu gates only the original refusal push. Product selectivity thus cannot characterize all working paths. The paper corrects this wiring, but the corrected system needs its own comparison. Removing recurrence also reduces accumulated drive, so an unmatched memoryless ablation cannot isolate memory's contribution. The corrected-controller MMLU benefit is potentially relevant, but its reported 300-question contrast lacks uncertainty.

Questions for authors
  1. Outcome meaning: Please define hold, violation, the remaining category and their scorers. A small blinded audit of cases changing to hold would clarify whether substantive failures are repaired.
  2. Dynamics: Please distinguish working-gate and product-aperture distributions, and compare with a memoryless arm matched in intervention strength.
  3. Corrected system: On one HOLD version and common arcs, compare base, original, corrected and simple dosing variants using violation, hold, both benign probes and scenario-clustered uncertainty. Please also quantify MMLU uncertainty.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.