acceptodds
Under review as a conference paper at ICLR 2027

Reasoning Under Intervention: Learned Soft Prefixes as Behavioral Probes

Abstract

Correct answers alone do not reveal how a language model organizes reasoning. We study this question with learned soft prefixes, short continuous vectors optimized while the model and the reasoning problem remain fixed. Rather than decoding the prefix itself, we interpret it through its behavioral effects by asking which inputs it changes, which it leaves unchanged, and whether those effects depend on meaningful properties of the problem. We apply this framework to syllogistic reasoning in Qwen3.6-35B-A3B MoE, Qwen3-8B, and Gemma 4 31B. Broad-target prefixes show that all three models can be steered, while also revealing different sensitivity to the answer interface. The Qwen models remain strongly steerable under randomized answer mappings, whereas Gemma depends more strongly on how the answers are represented. Broad steering alone, however, does not establish that an intervention responds selectively to the logical problem. We therefore pair valid syllogisms that share the same correct answer but differ in whether they rely on the nonempty-term assumption. In both Qwen models, learned prefixes separate these groups strongly, while Gemma shows little comparable effect. Their answer margins also shift differently, which a uniform answer-score bias cannot explain. A series of controls shows that the effect is stronger than for random groupings, is not tied to one presentation, and changes with the stated logical convention, while failing to transfer to an equivalent task formulation. These results give the three models distinct behavioral profiles in syllogistic reasoning. Together, this shows that behavioral intervention probes can reveal differences in how models organize reasoning that are not visible from traditional benchmarks alone.

Primary area:
interpretability and explainable AI
Keywords:
language model reasoning, behavioral probing, soft prompts, model interventions, logical reasoning, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PvNGOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper is about the use of soft prefixes and how they can influence the reasoning of an LLM. The authors use this framework for syllogistic reasoning in Qwen and Gemma models. They interpret these prefixes by looking at how they change the inputs.

Critical strengths
  • The study of the authors related to steering is interesting
  • The experiments are done on multiple model families (Qwen and Gemma)
Critical weaknesses
  • It is difficult to read and follow the paper, connections between ideas and experiments is not straight forward
  • no comparison has been made against other similar methods. The authors only show how their soft prefixes influence the reasoning but do not present similar steering methods and how they differ
Questions for authors
  • could you further explain what exactly you want to show with your work in simple terms?
  • do you have any comparison of your soft prefixes with other steering methods?
Reviewer_rMjgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, they want to show that language models can use the soft prefixes that they have learned as behavioural probes to find out that how they process logical reasoning problems, and their approach is that they use the learned prefixes directly so they do it by seeing that how interventions can have an effect on prediction across different inputs and conditions. They do it on two Qwen models and one Gemma and with two different experiments, one of them is that they do it with the broad steering and the second one is with a stronger same-answer test that is in a way that uses two groups of valid syllogisms. They show that there is separation between the two groups in different Qwen models, and also Gemma has a weaker separation. Also show these effects under different conditions, such as answer mappings, random groupings, logical conventions, and task formulations.

Critical strengths

Their clear distinction between broad steering and conditional intervention is solid also their same answer experiment in 5.2 is solid, as they mentioned that if a prefix changes the answer of a model, it does not mean that it is related to the logical structure of the problem, and this experiment does it in a way that both groups have the same correct answer, but the prefix is trained to have a different effect. Their consideration of answer margin, besides flip rate, is also solid, becuase the initial confidence of the model has an effect. Their broader experiments in 5.3 that they test to see under what conditions their methods work are also solid.

Critical weaknesses

In 5.2, they show that the learned prefix separates the assumption-dependent and assumption-independent syllogisms, but as they also mention in 6, these two have different formal structures. So, it is not clear that if prefixes are responding to the non-empty term assumption or they are learning these patterns. I know that they try to address it and they use random grouping in 5.3, but this still is not convincing because it only shows that their grouping is easier to separate than arbitrary groups.

In 5.4, the result is not strong enough in both directions. For example, assumption-independent forms show the convention effect, but the biggest flip gaps in table 2 are for assumption-dependent forms.

In 5.5 and 5.7 they show that the separation is strong when they change the prompt wording or order, but not when they have the same logical relationships as an equivalent inconsistency task. I know they mentioned it in 6, but it matters because when the underlying logical relationship is the same the prefix does not separate the two groups. So it is not clear what the learned prefix is. This also matters because they use these interventions to understand how models organize reasoning. It might be sensitive to the logical distinction, or it might be learning something specific to the task formulation, but the results sort of support a more task-specific interpretation.

Questions for authors

Can you give more direct control (something more than comparing only with random groupings) for the different formal properties besides assumption dependence of the two groups that is mentioned in 6? Like, an experiment that the quantifier patterns are fixed but the relevant logical assumption changes.

Can you explain the results in 5.2 and 5.4 because they show different patterns. In 5.2, the largest flip gaps is for the assumption-dependent forms, and in 5.4 the effects are stronger for the assumption-independent forms.

In 5.7, when the task is changed to equivalent inconsistency formulation the prefixes does not have the same separation so the logical relationship is the same. Can you explain that this is because of a different task formulation or that the conditional intervention is more difficult to learn?

2 comments

Thank you for recognizing our experiments and for your thoughtful questions. We ran follow-up tests to address them.

Q1. Is the prefix responding to the nonempty-term assumption, or just simple patterns?

We agree random groupings cannot settle this. In our original pairs, dependent syllogisms have only "All"/"No" premises, while independent partners contain "Some". The prefix could potentially exploit this.

First, we tested syllogisms with exactly the same premises but different conclusions. For example, consider the premises "All X are Y" and "All Y are Z". Under our convention, both conclusions are valid: "Some X are Z", which requires X to exist, and "All X are Z", which remains valid even if X is empty. Across five held-out comparisons, dependent-targeting prefixes still changed that group's answers much more often, with flip gaps of 91.6 percentage points in Qwen3.6 and 87.2 in Qwen3-8B. This cannot rely only on "Some" in the premises.

Second, we changed the training data to make that shortcut insufficient. We added independent examples that also have only "All"/"No" premises. Checking for "Some" in the premises could no longer distinguish the groups. We trained fresh prefixes and found that they could still distinguish unseen problems with identical premises.

ModelTarget dependentTarget independent
Qwen3.688.3 [53.3, 100.0]56.7 [20.0, 86.7]
Qwen3-8B90.0 [60.0, 100.0]69.6 [36.7, 95.0]

This does not eliminate all structural explanations. The original pairs shared conclusions while the new ones share premises. A combined premise/conclusion pattern could still work. Our aim is to rule out simpler explanations and characterize behavior. Inevitably there will always be patterns to logic.

Q2. Why do Sections 5.2 and 5.4 show different results?

The two experiments deliberately test different things. In Section 5.2, we change the syllogisms while keeping the logical convention fixed. In Section 5.4, we keep the syllogisms and prefixes fixed but change the convention.

As you noted, these changes affect the prefixes in different ways. We find this the interesting part. How easily a prefix distinguishes logical problems and how it responds to changes in logical rules are separate aspects of the model's behavior. Understanding these differences is precisely our aim.

The groups remain distinguishable when nonemptiness is stated explicitly. Changing the convention affects the prefixes most clearly for independent targeting (average interaction 0.72 [0.10, 1.37] in score), though the effect varies by wording.

Q3. Why doesn't the prefix work on the equivalent task? Is it simply harder to learn?

We tested two questions: Does a prefix trained on one task work on the other? And can we train a new prefix directly on the other task?

First, we checked whether the result was simply caused by comparing different problems. In our original experiments, we only tested problems that the model answered correctly before adding a prefix. However, the model sometimes answered a problem correctly in one version of the task but not the other. This meant we were potentially comparing different sets of problems.

To eliminate this possibility, we repeated the experiment using exactly the same problems that the model answered correctly in both versions. We also used the same learned prefixes. This way, the only thing that changed was how we asked the logical question. The flip gaps are:

ModelValidity taskEquivalent task
Qwen3.684.15.4
Qwen3-8B80.62.6

Strong separation still does not carry over to the equivalent task, even after longer training on validity and evaluation on identical examples. Some changes in the answer scores do carry over.

But this does not mean the equivalent task cannot be learned. We separately trained prefixes directly on that task, extending all runs to 1,000 steps. They now distinguish unseen problems in both directions: 53.4 and 68.0 points in Qwen3.6, and 21.9 and 29.7 points in Qwen3-8B.

So the conclusion is simple: both tasks can support selective interventions, but a prefix trained on one need not work on the other. We see this task dependence as part of the behavioral profile.

We acknowledge the small number of logical forms and limited generality. We see this as a promising proof of concept for studying more complex logical problems and developing richer behavioral profiles across models. We will include the new results and limitations in the revision, and welcome further discussion.

Thank you for the detailed response and the additional experiments. The new controls help rule out reliance on “Some” in the premises alone. However, as you acknowledge, combined premise–conclusion patterns remain a possible explanation, so it is still unclear how specifically the interventions reflect assumption dependence. The additional training on the inconsistency task also clarifies that selective interventions can be learned in both formulations. However, the limited transfer of the same prefix between equivalent tasks leaves my concern about the task-specific nature of the learned effect unresolved. These additions strengthen the behavioral findings, but I remain uncertain about the specificity and generality of their interpretation. These were central concerns in my original review, so I appreciate the authors’ efforts but will maintain my original scores.

Reviewer_Q3uhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes using learned soft prefixes as behavioral intervention probes to study how language models organize reasoning. Instead of interpreting the learned prefix vectors directly, the authors characterize their effects through controlled behavioral comparisons. The experiments focus on syllogistic reasoning across three models: Qwen3.6-35B-A3B MoE, Qwen3-8B, and Gemma 4 31B.

The paper first examines broad answer steering and then introduces a more challenging same-answer setting, where the prefix is trained to selectively alter predictions for two groups of valid syllogisms that differ in their dependence on the nonempty-term assumption. The results show substantial selective effects in both Qwen models but much weaker effects in Gemma. Additional experiments investigate random groupings, answer mappings, logical conventions, presentation changes, and transfer to an equivalent task formulation.

Critical strengths
  1. Well-designed controlled experiments for distinguishing general answer steering from selective intervention effects.

A key strength is the progression from broad steering to the more challenging same-answer setting, where both groups share the same correct label. This design helps rule out explanations based solely on a global preference for a particular answer. The paper further includes several useful controls, such as randomized answer mappings, norm-matched random prefixes, and 50 random groupings. These experiments provide meaningful evidence that the observed separation is not simply a consequence of arbitrary group assignments or uniform answer-score shifts.

  1. Interesting empirical findings about model-specific behavioral differences.

The paper identifies substantially different intervention responses across the three models. In particular, the Qwen models exhibit strong separation between assumption-dependent and assumption-independent syllogisms, while Gemma shows little comparable separation. The authors also investigate the effects of output vocabulary and evaluate the models under shared answer mappings and examples. These findings suggest that overall steerability and selective intervention behavior capture different aspects of model behavior, providing a useful perspective beyond standard reasoning accuracy.

Critical weaknesses
  1. The connection between the learned intervention effects and the intended logical property remains insufficiently established.

The central claim relies on interpreting the selective intervention effects as evidence of sensitivity to the nonempty-term assumption. However, as acknowledged in the paper, the assumption-dependent and assumption-independent groups also differ in correlated formal features, particularly their quantifier patterns. Although the random-grouping controls demonstrate that the semantic grouping is more easily separated than arbitrary groupings, they do not fully establish which features the learned prefixes actually exploit.

Furthermore, the primary same-answer prompts do not explicitly specify the nonempty-term convention (Appendix A.3). While the convention-switching experiments provide additional evidence, the convention interactions are statistically reliable primarily when assumption-independent forms are targeted, rather than in the assumption-dependent targeting direction that produces the strongest flip gaps (Table 3). Consequently, the evidence does not yet clearly distinguish sensitivity to the intended logical convention from sensitivity to correlated structural patterns or prompt-dependent heuristics. This ambiguity weakens the interpretation of the main result.

  1. The generality of the proposed behavioral probing framework is limited by the narrow evaluation setting and lack of transfer across equivalent task formulations.

The main semantic intervention experiments rely on a small number of independent logical pairs (eight eligible pairs for each Qwen model), all drawn from the same syllogistic reasoning benchmark. Although the authors provide statistical sensitivity analyses, the limited diversity of logical structures makes it difficult to assess how broadly the findings generalize.

More importantly, the strong separation observed in the validity task largely disappears when the same logical relationship is expressed through an equivalent inconsistency task (Section 5.7 and Appendix D.4). Even prefixes trained directly on the dual formulation do not exhibit statistically reliable separation. While the paper acknowledges this limitation, it raises a fundamental question about whether the learned prefixes reveal generalizable properties of reasoning or primarily task-specific response patterns. This significantly limits the broader implications of the proposed framework.

Questions for authors
  1. Regarding the interpretation of the semantic intervention effects (Weakness 1):

Could the authors provide stronger evidence distinguishing sensitivity to the nonempty-term assumption from sensitivity to correlated formal features, such as quantifier patterns? In particular, how should we reconcile the large assumption-dependent targeting gaps in the main experiments with the statistically inconclusive convention interactions for that same targeting direction? Clarifying whether the main effect persists under explicitly specified logical conventions, and whether it can be explained by simpler structural cues, would substantially strengthen the central interpretation.

  1. Regarding generalizability and task-formulation dependence (Weakness 2):

How should the failure to transfer to the logically equivalent inconsistency task be interpreted in the context of the proposed behavioral probing framework? In particular, could the authors clarify whether the loss of separation persists when evaluating exactly the same clean-correct logical pairs under both task formulations, rather than using view-specific eligibility? This would help determine whether the observed behavior reflects a more general logical distinction or is primarily tied to the specific validity-task interface.

1 comment

Thank you for recognizing the value of our experiments and for your thoughtful questions. We ran several follow-up tests.

Q1. Is the prefix responding to the nonempty-term assumption, or just simple patterns?

We agree random groupings can't settle this. In our original pairs, dependent syllogisms have only "All"/"No" premises, while independent partners contain "Some".

First, we tested syllogisms with exactly the same premises but different conclusions. For example, consider the premises "All X are Y" and "All Y are Z". Under our convention, both conclusions are valid: "Some X are Z", which requires X to exist, and "All X are Z", which remains valid even if X is empty. Across five held-out comparisons, dependent-targeting prefixes still changed that group's answers much more often, with flip gaps of 91.6 percentage points in Qwen3.6 and 87.2 in Qwen3-8B. This cannot rely only on "Some" in the premises.

Second, we changed the training data to make that shortcut insufficient. We added independent examples that also have only "All"/"No" premises. That made checking for "Some" in the premises insufficient. We trained fresh prefixes and found that they could still distinguish unseen problems with identical premises.

ModelTarget dependentTarget independent
Qwen3.688.3 [53.3, 100.0]56.7 [20.0, 86.7]
Qwen3-8B90.0 [60.0, 100.0]69.6 [36.7, 95.0]

This does not eliminate every structural explanation. The original pairs shared conclusions while the new ones share premises. A combined premise+conclusion pattern could still work. Our aim is to rule out simpler explanations and characterize the behavior. Inevitably there will always be patterns to logic.

What about the stated logical convention? We changed only the instruction about whether categories may be empty, keeping the syllogisms and prefixes fixed. The groups remain distinguishable with explicit nonemptiness. Changing the convention can change the effect, most clearly for independent targeting (average interaction 0.72 [0.10, 1.37] in score units at 1,000 steps), although results vary by wording and the instruction text necessarily changes.

Why do Sections 5.2 and 5.4 show different results? The two experiments deliberately test different things. In Section 5.2, we change the syllogisms while keeping the logical convention fixed. In Section 5.4, we keep the syllogisms and prefixes fixed but change the convention.

As the reviewer notes, the two changes affect the prefixes in different ways. We consider this difference interesting in itself. It shows that how easily a prefix distinguishes logical problems and how it responds to changes in logical rules are separate aspects of the model's behavior. Understanding these differences is precisely the purpose of our approach.

Q2. Does the failure to transfer persist on exactly the same examples?

You are correct. The original evaluations could include different examples. We repeated the comparison using only problems answered correctly without a prefix in both tasks, and applied the same trained prefixes to exactly the same examples. One task asks "Does the conclusion follow?", the other asks the logically equivalent question "Is it impossible for the premises to be true while the conclusion is false?" The results are:

ModelValidity taskEquivalent task
Qwen3.684.15.4
Qwen3-8B80.62.6

These are flip gaps in percentage points. Even after training, strong separation does not carry over to the equivalent task. Different test examples were not the cause.

But this does not mean the equivalent task cannot be learned. We separately trained prefixes directly on that task and extended them to 1,000 steps. They now distinguish unseen problems in both directions: 53.4 and 68.0 points in Qwen3.6, and 21.9 and 29.7 points in Qwen3-8B. These are separate evaluations.

So the result is simpler than it first appeared. Both tasks can support selective interventions, but a prefix trained on one task doesn't always work on the other. We see this task dependence as part of the behavioral profile, not as evidence of a task-independent internal mechanism.

We acknowledge the small number of logical forms and limited generality. We see this as a promising proof of concept for studying more complex logical problems and developing richer behavioral profiles across models. We will include the new results and limitations in the revision. Thank you, and we hope we have appropriately addressed your comment.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Comment to all reviewers

We want to thank all three reviewers for their time and feedback. We wanted to bring together some common themes.

First of all, it seems that all three reviewers found our paper difficult to read. We apologize for that. We put a lot of effort into the writing, but the concepts are sometimes counterintuitive and we did not explain them clearly enough. We would like to share our overall thought process.

What are we trying to show?

In essence, we want to study the logical reasoning behavior of each model. Not necessarily how well it thinks or how easily it fails, but what its failures can tell us about how it approaches logical problems.

Our thought process is basically as follows:

  1. Just asking a model to do well on logic benchmarks doesn't show us too much about how it reasons, because the model has been pretrained on many similar patterns.

  2. What's more interesting is what can make it fail, not simply whether it fails. Simply knowing that a chess player loses tells us little about how they play. However, if we can trick them just by playing an illegal move, that tells us they don't really understand chess. On the other hand, if they fall for an elaborate trap like a mating net, they may actually be a great chess player, and we used their chess knowledge to trick them. We want to understand the patterns behind a model's failures, rather than just demonstrate that it can fail.

  3. Just using logical traps written in natural language is difficult to control, though. These models have seen so much pretraining data that they may have encountered similar patterns before.

  4. Instead, we use a very standard soft-prefix tuning method as a simple way to break the model without modifying the underlying logical problem. We do not need to decide in advance what kind of logical trap might work. We simply specify the objective and let gradient descent find a prefix that produces it.

  5. The problem is that these learned prefix vectors are black boxes that are hard to interpret directly.

  6. One way we can understand these prefixes is by seeing how they affect other problems and conditions that we do understand. For example, suppose a prefix makes the model incorrectly answer "invalid" to a bunch of valid problems. Is it really interfering with logical reasoning, or simply making the model answer "invalid" to everything? We can test this by applying the same prefix to a different set of problems where all the correct answers are "invalid". If it suddenly gets everything right, that suggests it might just be pushing the model toward one answer.

  7. This lets us build a picture of how these prefixes work through their behavioral effects and better understand which distinctions influence a model's reasoning behavior. We are not claiming to fully identify the model's internal reasoning mechanism, but rather to characterize its behavior under controlled interventions.

The entire paper is centered around this concept. We hope this explanation makes more sense! As we explained to Reviewer PvNG, we use standard prefix tuning to study the behavior it produces, not to develop a better steering method.

Shared questions raised by Reviewers Q3uh and rMjg

Both reviewers raised similar concerns, which we addressed in our individual replies.

Could the prefixes just be detecting simple patterns? This is a fair concern. We tested new syllogisms with identical premises, and trained new prefixes on a broader set of examples where simply looking for "Some" in the premises would not be enough. The prefixes still distinguished the groups on unseen problems. This rules out that simple shortcut, although more complicated patterns could still explain the behavior. Inevitably all logic is based on patterns so this is unavoidable.

Why do the results change when we change the logical convention? In one experiment, we compare different syllogisms under the same rule. In another, we keep the syllogisms and prefixes fixed but change the rule about whether terms can be empty. We find that the prefixes respond differently to these two changes, and even the differences between targeting directions are informative. That is precisely the kind of behavior we're trying to understand.

Why doesn't the effect transfer to an equivalent task? We tested the same prefixes on exactly the same problems, asking the logically equivalent question in another way. The strong effect did not carry over. But when we trained new prefixes directly on the second task, they could distinguish the groups. Both tasks can support selective interventions, but a prefix trained on one need not work on the other.

These results do not reveal an internal reasoning mechanism. They help us understand which differences matter to the model and where the effects stop. We recognize the small sample and will include the new evidence and limitations in the revision.

We thank all three reviewers again and welcome further discussion.

Area_Chair_x2ZUOpenReview ↗

Reminder: Please Read and Discuss the Author Rebuttal

Dear Reviewers,

As the discussion period ends on November 18 (AoE), please take this final opportunity to review the authors’ rebuttal and discuss any remaining concerns with them. Even if the rebuttal does not change your assessment, I would appreciate it if you could briefly acknowledge that you have read and considered it.

Thank you for your time and contribution.

AC, ICLR 2027

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.