acceptodds
Under review as a conference paper at ICLR 2027

Prompt-Adapted Transport Control: Contextual Tokenwise LLM Behaviour Steering via Flow Matching

Abstract

Many activation-steering methods compress a behavioural contrast into a fixed global direction, ignoring how the desired transformation varies across contextual token states. We introduce Prompt-Adapted Transport Control (PATC), which learns flows at two complementary resolutions. A token-level flow transports contextual prompt states toward the target-behaviour distribution; PATC selects, orients, and aggregates behaviour displacements aligned with the training behavior distribution, retains the components that best characterize how the behavior changes, and removes overlap with the source-activation subspace to obtain a prompt-adapted unit direction. Because local token transports vary in norm with context, a separate flow trained on mean-pooled response activations supplies a common magnitude from the terminal prompt state. Their product forms a fixed cache applied to all tokens throughout generation without further flow evaluation. We evaluate anger and anxiety at approximately matched achieved strength, isolating intervention quality from steering intensity; these dimensions also permit comparison with held-out COVID-Worry human reference profiles. Across evaluation, PATC consistently balances affect, lexical quality, task compliance, and human-profile fidelity.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
activation steering, representation engineering, flow matching, large language models, prompt-conditioned control, inference-time intervention, behavioral steering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_G4HXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed PATC, an activation steering method that leverages flow matching to construct prompt specific steering vectors. The method trains flow matching models at different resolutions, a token level to estimate behavioral steering directions and a response level to estimate steering strength, The resulting steering vector is applied throughout generation. Authors evaluate PATC on anger and anxiety using Qwen and Gemma.

Critical strengths
  1. The paper proposes an interesting approach to prompt adaptive activation steering by leveraging token level information to construct steering directions and using a separate response level flow to estimate steering magnitudes. The idea of aggregating token level transport information into a single cached steering vector is appealing, as it allows the method to incorporate fine grained contextual information without requiring repeated flow evaluations during generation.
  2. The experimental results demonstrate that PATC generally achieves a favorable trade off between steering effectiveness and generation quality. In particular, the method shows improvements in lexical diversity, repetition reduction, and task preservation compared with several existing activation steering methods. The evaluation at approximately matched achieved affect also makes the comparisons more convincing by reducing the influence of differences in steering strength.
Critical weaknesses
  1. Insufficient justification for the complex steering framework. The overall steering framework is relatively complex, involving several design choices whose necessity is not sufficiently justified. Although the authors provide intuitive motivations and conduct some ablation studies, several components remain insufficiently explored. For example, it is unclear why the local transport directions predicted by the flow matching model need to be corrected based on the empirical mean direction. Such corrections may potentially discard meaningful local variations. Similarly, the necessity of displacement projection and source nulling is not thoroughly investigated. More targeted ablation studies would help clarify the individual contributions of these components.
  2. Limited scope of experimental evaluation. The experimental evaluation is relatively narrow. Although the paper presents PATC as a general framework for behavioral steering, the experiments focus primarily on anger and anxiety. It remains unclear whether the proposed method generalizes to other behavioral characteristics, such as confidence, caution, or skepticism. Furthermore, the evaluation relies on a relatively small set of prompts, which may limit the generalizability of the findings.
Questions for authors

Could the authors provide further justification for the design choices in the steering direction construction and provide more targeted ablation studies to demonstrate the necessity of individual components? In particular, why is it necessary to correct the local transport directions predicted by flow matching based on the empirical mean direction, given that flow matching is intended to learn transformations. Plus, could the authors clarify the necessity of introducing an additional calibration gain g alongside the response level magnitude estimator?

4 comments
AuthorsOpenReview ↗

Rebuttal 1

Thank you for your thorough review and the interesting questions and concerns raised! We think they have made the paper clearer and we hope we've addressed your concerns below.


Clarifying role of empirical mean direction

“For example, it is unclear why the local transport directions predicted by the flow matching model need to be corrected based on the empirical mean direction. Such corrections may potentially discard meaningful local variations.”

Thank you for raising this point. We clarify two aspects of the construction:

  • The mean does not determine the local displacement. Each displacement is independently predicted by the token flow: δi=Φτtok(hi)−hi.\delta_i=\Phi_{\tau}^{\mathrm{tok}}(h_i)-h_i.

  • The mean provides a behavioural reference for aggregation. Alignment with the empirical target-minus-source direction is used only to select and weight the local transports most relevant to the target behaviour. The selected vectors are not projected onto the mean, so their prompt-specific components are retained.

We have revised the method description to distinguish mean-referenced token selection from the independently predicted local transports of individual components.


Question about importance of gain parameter

“Plus, could the authors clarify the necessity of introducing an additional calibration gain gg alongside the response-level magnitude estimator?”

  • No additional gain is fitted. We use gpool=1g_{\mathrm{pool}}=1 throughout all reported experiments; the intervention scale is therefore determined directly by mpool(P)m_{\mathrm{pool}}(P). The symbol was included only to express the general formulation, but it misleadingly suggests another calibrated component. In practice, gg could be used to increase steering intensity beyond the flow endpoint through the magnitude, but this is not part of the evaluation and only added for completeness. We will remove it and write the reported intervention directly as vP=mpool(P)d^PATC(P),v_P=m_{\mathrm{pool}}(P)\widehat d_{\mathrm{PATC}}(P), exactly representing the implementation.

Experimental scope: Added Confidence and Skepticism

“Limited scope of experimental evaluation. The experimental evaluation is relatively narrow. Although the paper presents PATC as a general framework for behavioral steering, the experiments focus primarily on anger and anxiety. It remains unclear whether the proposed method generalizes to other behavioral characteristics, such as confidence, caution, or skepticism. Furthermore, the evaluation relies on a relatively small set of prompts, which may limit the generalizability of the findings.”

We added a two-model audit on confidence and skepticism

  • Additional targets and models: We trained target-specific PATC flows for both Gemma-3-12B and Qwen3-14B and compared them with the corresponding unsteered model on all 25 behavioural prompts.

  • Evaluation breadth: The prompts span opinion, planning, advice, narrative, and reflection. Three fresh blinded judging passes independently randomized A/B placement; we report per-prompt majority decisions.

ModelTargetPATCTieUnsteered
Gemma-3-12BConfidence1375
Gemma-3-12BSkepticism1933
Qwen3-14BConfidence2500
Qwen3-14BSkepticism1438
  • Interpretation: Confidence transfers particularly clearly to Qwen, where PATC is preferred on all 25 prompts. Skepticism is strong on Gemma and directionally positive but less certain on Qwen. The reported outputs remain fluent and complete.

  • Qualitative corroboration: Appendix G includes a Qwen confidence example and two Gemma skepticism examples unanimously preferred by all three judging passes.

  • Scope of this audit: We present these results as preliminary evidence of transfer, not as a full benchmark against other steering methods. Unlike anger and anxiety, confidence and skepticism lack the human-validated evaluators required to match methods at comparable achieved strength. A broader comparison based only on model judgments could therefore confound steering quality with differences in steering intensity.

See Appendix G, Table 19 and the accompanying examples.

AuthorsOpenReview ↗

Rebuttal 2

Projection and source nulling ablations

“Similarly, the necessity of displacement projection and source nulling is not thoroughly investigated.”

We added a controlled ablation and a direct activation-space analysis.

GeometryAngerMTLD ↑Rep.-3 ↓Full pass ↑
None3.96642.2818.8384.67
Projection only4.13645.3916.3978.00
Nulling only4.08448.794.9675.33
Projection -> nulling4.22151.383.8384.67
  • Both operations contribute: Removing projection reduces MTLD by 2.59 and full pass by 9.34 points. Removing source nulling reduces MTLD by 5.99, increases repeated trigrams by 12.56 points, and reduces full pass by 6.67 points.

  • The operations impose different constraints: Projection restricts the token-wise consensus to the fitted response-displacement subspace, while source nulling removes components aligned with the fitted source-activation subspace.

To examine how each operation changes the constructed direction, we measure the fraction of its squared norm captured by the fitted displacement and source subspaces (“subspace energy”). High displacement energy indicates greater support from observed source-to-target transformations; low source energy indicates less residual alignment with source activations.

GeometryDisplacement energy ↑Source energy ↓
None51.35%31.77%
Projection only100.00%29.12%
Nulling only42.11%0.00%
Projection -> nulling82.74%0.00%
  • The full sequence removes source alignment while retaining displacement support: Projection alone retains 29.12% source energy because the subspaces overlap, while nulling alone leaves only 42.11% displacement energy. Projection followed by nulling achieves zero source energy while retaining 82.74% displacement energy, exceeding nulling alone on all 75 prompts.

  • Either operation alone lowers full pass: Projection only retains source-aligned components, whereas nulling only removes overlapping components without first restricting the consensus to the behaviour-supported space. Normalization then applies the full response-level magnitude to these imbalanced directions, reducing full pass by 6.67–9.34 points.

  • “None” retains tasks through weaker, lower-quality steering: The unprocessed direction matches the complete method’s full-pass rate but achieves weaker anger, lower lexical diversity, and substantially more repetition. Its full-pass result therefore does not reproduce the complete method’s steering–quality balance.

  • The interaction is specific to token-wise construction: With a pooled-flow direction, adding projection before source nulling leaves MTLD and full pass essentially unchanged.

Pooled-flow geometryAngerMTLD ↑Rep.-3 ↓Full pass ↑
Nulling only4.26339.458.2683.33
Projection + nulling4.37739.438.4483.33

This indicates that the favourable interaction arises within PATC’s token-wise direction construction rather than from applying the same operations to any flow displacement.

See Table 4 and Appendix C.5, Figure 3.


Why the main evaluation focuses on anger and anxiety

“Limited scope of experimental evaluation. The experimental evaluation is relatively narrow...”

We agree that behavioural breadth matters. We deliberately designed the main study around targets for which a well-posed comparison is possible: frozen evaluators validated on participant-held-out human text, approximate matching of achieved strength, and comparison against a complete human affect profile. These controls make differences between methods interpretable; expanding to many targets evaluated only by uncalibrated model judgments would increase label coverage without providing equally reliable evidence. Without validated measures to match achieved behavioural strength, comparisons on additional targets could simply reward stronger interventions rather than genuinely better trade-offs between steering quality and task preservation.

AuthorsOpenReview ↗

Rebuttal 3 (Evaluation now spans three models and 75 behavioural prompts)

Evaluation now spans three models and 75 behavioural prompts

“Furthermore, the evaluation relies on a relatively small set of prompts, which may limit the generalizability of the findings.”

  • Main evaluation expanded without changing its design: We added Llama-3-8B-Instruct and expanded the behavioural suite from 25 to 75 prompts while retaining the same frozen evaluation protocol. The submitted 25-prompt suite was already balanced across opinion, planning, advice, narrative, and reflection, although this structure and its rationale were not sufficiently clear in the original description. The expanded suite preserves that design, increasing coverage from five to 15 prompts per response type. These types test distinct generation demands: defending a position, organizing actions, giving recommendations, maintaining narrative coherence, and developing an introspective explanation.

  • Results persist across response types: Across the resulting 24 model–target–strength blocks, PATC ranks first or second in 19/24 MTLD and 20/24 repetition comparisons. When stratified by response type, PATC ranks first or second in MTLD for all five types and has the lowest repetition in four of five; TruthFlow leads on narrative.

  • Broader behavioural and task coverage: Together with the 50-prompt strict suite, the evaluation now covers three model families and 125 prompts spanning both behavioural expression and task preservation. The complete prompts are provided in the Experimental Prompt Suites appendix.

We hope the additional ablations, expanded three-model and 75-prompt evaluation, and methodological clarifications address your concerns. If so, we would be grateful if you could update your rating to reflect the revised evidence. If any concern remains unresolved, we would greatly appreciate your guidance on which point requires further clarification.

AuthorsOpenReview ↗

We hope we addressed your concerns

Dear Reviewer,

We hope the expanded three-model and 75-prompt evaluation, additional-behaviour audit, and new component ablations resolve the concerns raised in your review. If these additions address your reservations, we would be grateful if you would reconsider your rating. If anything remains unresolved, we would appreciate knowing which concern requires further evidence or clarification.

Thank you!

Reviewer_7ngDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PATC, an activation-steering method that constructs a fixed, prompt-dependent intervention by estimating its direction and magnitude separately. PATC learns from token-level differences between neutral and target-behavior responses, then combines information from multiple prompt tokens to determine the steering direction. Meanwhile, a separate flow trained on mean-pooled response activations determines the steering magnitude. The resulting vector is computed once and reused throughout generation. Experiments cover anger and anxiety steering on Qwen3-14B and Gemma-3-12B, comparing PATC with CAA, TruthFlow, and ODESteer at approximately matched affective strength. PATC generally improves lexical diversity over CAA and TruthFlow while maintaining competitive task-and-format compliance. The paper also evaluates alignment with held-out human affect profiles from COVID-Worry, with results varying across models and baselines. The main contribution is a steering framework that combines token-level direction construction with response-level magnitude estimation while avoiding further flow evaluations during decoding.

Critical strengths
  1. The method offers an effective improvement to steering-vector estimation. PATC derives the steering direction from multiple contextual token states and its magnitude from a separate response-level flow, caching the resulting vector for efficient decoding. The magnitude ablation shows that, despite similar intervention norms and anger scores on behavioral prompts, selected-token norms lead to larger interventions and lower full-pass rates on strict tasks. This supports estimating direction and magnitude separately.
  2. The experiments provide support for the method through calibrated and controlled comparisons. Experiments cover two models, two affective targets, and multiple strength levels, supplemented by a clean-final schedule control and direction-construction ablations. These choices lend credibility to PATC’s reported lexical-quality advantages over CAA and TruthFlow.
Critical weaknesses
  1. The need for a separate response-level flow is not fully established. Appendix C.3 shows that pooled-response magnitude preserves strict-task performance better than selected-token norms. However, the token-derived magnitude becomes substantially larger on strict prompts, leaving open whether a calibrated constant magnitude, clipping, or normalization could provide similar benefits. The reliability of querying a flow trained on pooled response states with a terminal prompt state also remains insufficiently examined.
  2. The results do not consistently establish an advantage over existing methods. PATC generally improves lexical quality over CAA and TruthFlow, but its task-preservation gains are less consistent. ODESteer achieves higher strict-task full-pass rates across the reported Gemma settings and lower human-profile distances on Qwen, although the latter comparison is not fully strength-matched. Table 3 also shows that a fixed global-mean direction achieves comparable task preservation, albeit with worse lexical quality. These mixed results leave uncertainty about whether PATC’s benefits justify its additional complexity over existing or simpler alternatives.
Questions for authors
  1. Appendix D validates anger and anxiety predictions on human texts. Can the authors provide independent intensity judgments on generated responses at the frozen settings and validation for the remaining seven affect dimensions?
  2. What evidence supports this transfer, and does pooled-flow magnitude outperform a calibrated constant magnitude or simple norm control? These comparisons would clarify the value of the second flow.
  3. Can the authors report the training cost of both flows and compare practical inference costs with the baselines, including cache-construction overhead, time to first token, and total generation time?
5 comments
AuthorsOpenReview ↗

Rebuttal 1

Thank you for your time taken to review the paper and the very helpful feedback! We hope we are able to address your concerns in the following sections.


Question 1: Independent Intensity Judgments

“Appendix D validates anger and anxiety predictions on human texts. Can the authors provide independent intensity judgments on generated responses at the frozen settings and validation for the remaining seven affect dimensions?”

We have extended/added those now and both checks leave our conclusions unchanged.

  • The refined audit finds no systematic intensity bias: The submitted paper already included blinded intensity judgments on generated responses; we have now completed the audit at every exact frozen operating point. A blinded model judge rated ten randomized, same-prompt pairs using target intensity only. Results vary by setting rather than consistently favouring any method, supporting approximate (not exact) matching. See Appendix D, Table 15.

  • Held-out validation is now reported for all nine dimensions: For the seven additional heads, participant-held-out long-response correlations range from ρ=.226\rho=.226 to .631.631, with six above .40.40. Full metrics and calibration curves appear in Appendix D, Table 16 and Figure 5.

  • Weaker heads do not determine the profile result: Excluding desire preserves the three-model ordering: PATC .525, TruthFlow .657, CAA .669, and ODESteer .803. See Appendix Table 17.


Question 2: Magnitude Controls and Terminal-State Transfer

“The need for a separate response-level flow is not fully established… leaving open whether a calibrated constant magnitude, clipping, or normalization could provide similar benefits.”

“What evidence supports this transfer, and does pooled-flow magnitude outperform a calibrated constant magnitude or simple norm control?”

We agree that this is a valid concern. We added magnitude-source and query-representation controls.

Magnitude controls

See Appendix C.3, Table 11.

Magnitude sourceAngerMTLD ↑Rep.-3 ↓Full pass ↑
Pooled-response flow4.22151.383.8384.67
Calibrated constant4.20649.343.9183.33
Selected-token median4.22252.094.7972.67
  • A calibrated constant performs comparably: It matches pooled-flow anger, 4.206 vs. 4.221, and full pass, 83.3% vs. 84.7%, with slightly lower MTLD. An appropriately calibrated constant can therefore recover a similar operating point.

  • The flow avoids separate norm calibration: The constant was obtained by averaging pooled-flow predictions on a disjoint prompt set. Without the flow, its value would require a separate norm-selection procedure. We present the response flow as a practical calibration mechanism, not as mathematically necessary or uniformly superior.

  • Token norms overscale strict prompts: Selected-token magnitude matches anger at 4.222 but increases the mean strict-prompt norm from 70.23 to 91.03 and reduces full pass by 12 points. Token-level transport norms therefore do not reliably determine response-level intervention scale.

Terminal-state transfer

See Appendix C.4, Table 12. We compared terminal-state and mean-pooled-prompt querying while holding the response flow, direction construction, prompts, and decoding fixed.

Response-flow queryGainNormAngerMTLD ↑Rep.-3 ↓Full pass ↑
Terminal prompt state1.00072.254.22151.383.8384.67
Mean prompt, norm-matched1.11272.263.94049.744.1684.00
Mean prompt, affect-matched1.19577.624.26044.437.0083.33
  • Matched affect: At comparable anger, 4.221 vs. 4.260, terminal querying has higher MTLD, 51.38 vs. 44.43, and lower repetition, 3.83% vs. 7.00%, with comparable full pass.

  • Better training support with terminal state: Terminal states are substantially closer to pooled-response training states than mean-pooled prompt states, with normalized nearest-neighbour distances of 5.72 vs. 42.79.

  • Better expression at equal norm: At virtually identical mean norms, terminal querying produces higher anger on 18 of 25 prompts, with a median paired difference of +0.231 and p=.0018p=.0018.

  • Why the terminal state: It has attended to the complete prompt, whereas mean pooling combines states representing incomplete prefixes. We therefore use it as an inference-time proxy for the response state, while the response flow supplies the learned response-scale reference. These results support this choice in the evaluated setting without claiming universal optimality.

AuthorsOpenReview ↗

Rebuttal 2

Training and inference costs

“Can the authors report the training cost of both flows and compare practical inference costs with the baselines, including cache-construction overhead, time to first token, and total generation time?”

Certainly. We added this analysis in Appendix B.1, Table 5. We benchmarked Qwen3-14B on two RTX 3090 GPUs with batch size one, one warm-up prompt, ten measured prompts, and exactly 128 generated tokens. TTFT includes prompt-specific cache construction; model and checkpoint loading are excluded.

MethodCache (s)TTFT (s)Decode (s)Total (s)Overhead
Unsteered0.0000.071±.0046.593±.0356.593±.035—
CAA0.0000.075±.0026.613±.0836.613±.083+0.3%
TruthFlow0.126±.0020.199±.0056.602±.0726.728±.072+2.1%
PATC0.117±.0050.190±.0086.561±.0316.678±.030+1.3%
ODESteer0.0000.073±.0047.492±.0717.492±.071+13.6%

PATC incurs 0.1170.117 seconds of one-time cache construction but requires no flow evaluation during decoding. Its total measured overhead is 1.3%1.3\%, compared with 2.1%2.1\% for TruthFlow and 13.6%13.6\% for ODESteer.

Each PATC flow contains 6.83M parameters. Starting from recorded activations, complete retraining took 33.4 seconds for the token flow and 20.9 seconds for the response flow, each on one RTX 3090. These measurements include preprocessing, diagnostics, and checkpoint writing but exclude the shared one-time extraction of base-model activations.


Question 3: Expanded evaluation confirms PATC’s cross-model balance

“The results do not consistently establish an advantage over existing methods.”

We agree that PATC does not dominate every method on every metric. Since the review, however, we have substantially expanded both experiments:

  • Experiment 1 now evaluates three models and 75 behavioural prompts: We added Llama-3-8B-Instruct, and tripled the behavioural suite from 25 to 75 prompts.
  • Experiment 2 now includes the same third model: We extended human-profile matching to Llama using all four methods and ten seeds.

The expanded evidence supports PATC’s advantage as a consistent quality–preservation balance rather than universal dominance.

  • PATC ranks first or second in 58/72 matched comparisons: Across the 24 model–target–strength blocks, PATC ranks first or second in 19/24 MTLD, 20/24 repetition, and 19/24 full-pass comparisons. Its three-model macro-average improves MTLD by 11.2%, reduces repetition by 53.4%, and improves full pass by 1.9 points relative to the next-best method on each metric.

  • The result persists across response forms: The submitted suite already covered opinion, planning, advice, narrative, and reflection; we expanded it from five to 15 prompts per type without changing that design. Stratified by response type, PATC ranks first or second in MTLD for all five types and has the lowest repetition in four of five. TruthFlow leads both lexical measures for narrative.

  • ODESteer leads on Gemma but is architecture-sensitive: Its model-average full pass is 94.8% on Gemma but falls to 72.2% on Qwen and 63.8% on Llama, the lowest result on both latter models. PATC records 89.5%, 90.8%, and 72.2%, respectively: second on Gemma and first on Qwen and Llama. We therefore describe PATC as more consistent across architectures, not universally superior.

  • Prompt adaptation improves quality over the simpler global direction: At comparable anger and essentially unchanged full pass, PATC increases MTLD from 45.49 to 51.38 and reduces repetition from 4.75% to 3.83%. Across 75 same-prompt, same-seed outputs, PATC has lower per-response trigram repetition in 45 cases, the global direction in 28, with two ties. See Table 3 and the accompanying appendix examples.

  • The expanded human-profile study favours PATC across models: At the selected stronger points, PATC’s three-model nine-axis RMSE is .496, compared with .633 for TruthFlow, .643 for CAA, and .773 for ODESteer. ODESteer remains best on Qwen, while PATC is best on Gemma and Llama. ODESteer does not reach PATC’s stronger achieved-anger point on Qwen or Llama, so we report the residual mismatches and interpret this as cross-model consistency rather than an exactly strength-matched universal advantage. See Table 2 and the complete ODESteer sweep in the appendix.

  • The additional structure has low measured cost: PATC adds 1.3% total inference overhead in the representative benchmark, below the measured overheads of TruthFlow and ODESteer.

Together, the expanded evaluation supports a narrower but stronger claim: PATC consistently improves expressive quality while retaining competitive strict-task performance across three architectures, two targets, four strengths, and substantially broader prompt coverage.

AuthorsOpenReview ↗

We hope we addressed your concerns

Dear Reviewer,

We hope the independent evaluator checks, magnitude and terminal-state controls, computational analysis, and expanded three-model results address the questions in your review. If the revised evidence resolves your concerns, we would appreciate an updated assessment. Otherwise, we would be grateful for any indication of which issue remains outstanding so that we can respond directly. Thank you for taking the time.

Reviewer_gRpyOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The authors investigate the problem of LLM behavior steering through a novel method PATC. The method is evaluated on intervention quality and matched affect on two models targetting at anger and anxiety. Results are reported to be good compared with baselines.

Critical strengths
  1. The paper is generally well-written, I have found it easy to follow.

  2. The problem of LLM behavior control is important and timely. The method is well-motivated and novel.

  3. In the evaluation, the baselines include a comprehensive suite comprised of CAA, TruthFlow and ODESteer.

  4. The quantitative results demonstrate clear advantage of the method over the baselines.

Critical weaknesses
  1. The experimental results on two models and two targets feels thin. The paper would benefit from more experimental validation if possible.

  2. The mathematical formulation of the complete intervention v_P is key to fully understand the method, but not motivated. Expanding the motivation for why this particular formulation is adopted in 3.3 would improve the paper's delivery.

  3. The writings in the Method section can be improved and follow a more intuitive introduction of the method. I believe that would make the paper easier to understand.

Questions for authors
  1. What is the major difference between the token-level flow and the response-level flow?

  2. What does Figure 2 tell us about the method? The gaps between the baselines and the PATC method does not seem to have a clear trend as shown in this Figure.

2 comments

Thank you for the positive assessment and constructive suggestions! We revised the method presentation and added experimental evidence addressing each point to address your concerns. Thank you!


“The experimental results on two models and two targets feels thin. The paper would benefit from more experimental validation if possible.”

We added a two-model audit of two additional behaviours, confidence and skepticism.

  • Setup: We trained target-specific PATC flows for Gemma-3-12B and Qwen3-14B and compared PATC with the corresponding unsteered model on all 25 behavioural prompts. Three blinded judging passes independently randomized A/B placement, and we report per-prompt majority decisions.
ModelTargetPATCTieUnsteered
Gemma-3-12BConfidence1375
Gemma-3-12BSkepticism1933
Qwen3-14BConfidence2500
Qwen3-14BSkepticism1438

Confidence transfers particularly clearly to Qwen, while skepticism is strong on Gemma and directionally positive on Qwen. These results provide initial evidence that the same training and inference procedure transfers beyond affective targets. See Appendix G, Table 19 and the accompanying examples.

We also clarify that the main evaluation is broader than its two target labels suggest: it covers 25 behavioural prompts across opinion, planning, advice, narrative, and reflection, together with 50 strict prompts spanning classification, extraction, rewriting, structured generation, multilingual comprehension, reasoning, coding, and exact-format tasks.


“The mathematical formulation of the complete intervention vPv_P is key to fully understand the method, but not motivated.” “What is the major difference between the token-level flow and the response-level flow?”

The two flows estimate different properties of the intervention:

  • Token-level flow, where to move: It is trained on individual contextual token activations and evaluated across the prompt tokens. Their local transports are combined to construct a prompt-adapted unit direction d^PATC(P)\widehat d_{\mathrm{PATC}}(P).

  • Response-level flow, how far to move: It is trained on mean-pooled response activations. At inference, only the norm of its predicted displacement is retained as the response-level scale mpool(P)m_{\mathrm{pool}}(P).

The reported intervention is therefore

vP=mpool(P)d^PATC(P).v_P=m_{\mathrm{pool}}(P)\widehat d_{\mathrm{PATC}}(P).

This separation prevents variation in local token-transport norms from automatically determining generation-level steering strength. The resulting vector is constructed once and cached throughout decoding.

The magnitude ablation supports this motivation: selected-token magnitude matches pooled-flow anger (4.2224.222 versus 4.2214.221) but reduces strict-task full pass from 84.67%84.67\% to 72.67%72.67\%. An appropriately calibrated constant performs comparably at this operating point, so we present the response flow as a practical calibration mechanism rather than a mathematically necessary factorization. See Appendix C.3, Table 11.


“The writings in the Method section can be improved...”

We revised Section 3 to make the method’s progression more explicit:

  1. learn transport at token and response resolutions;
  2. construct a prompt-adapted direction from contextual token transports;
  3. estimate its response-level magnitude; and
  4. combine both quantities into one intervention cached throughout generation.

The revision defines the prompt and contextual states before their first use, presents token-consensus construction and DSN geometry as distinct steps, and explicitly motivates the complete intervention as a prompt-adapted direction multiplied by a response-level scale.


“What does Figure 2 tell us about the method? The gaps between the baselines and the PATC method does not seem to have a clear trend as shown in this Figure.”

Figure 2 is a strength-matching diagnostic, not evidence that PATC’s performance advantage follows a monotonic trend.

  • Each point compares achieved affect at an independently frozen operating point.
  • Positive values mean that the corresponding baseline produced stronger measured affect than PATC.
  • Because each method is calibrated separately at each model, target, and strength, no monotonic pattern across points is expected.
  • The intended result is that the gaps remain small: median absolute gaps are 0.0260.026, 0.0480.048, and 0.0390.039 for TruthFlow, CAA, and ODESteer, respectively. They are at most 0.100.10 in 15/16 settings for both TruthFlow and CAA and all 16 settings for ODESteer.

Thus, Figure 2 establishes that the subsequent lexical-quality and task-preservation comparisons are made at approximately matched achieved affect rather than being explained by systematically weaker PATC steering. We revised the caption and surrounding text to make this purpose explicit.

Thank you again for your positive assessment. Together with the concerns of the other two reviewers and yours about the original evaluation being slightly thin, we expanded the empirical evidence:

  • Broader main evaluation: We added Llama-3-8B-Instruct and tripled the behavioural suite from 25 to 75 prompts, yielding 24 model–target–strength blocks across three architectures.

  • Consistent expanded results: PATC ranks first or second in 58/72 affect-matched MTLD, repetition, and full-pass comparisons. Its macro-average improves MTLD by 11.2%, reduces repetition by 53.4%, and improves full pass by 1.9 points relative to the next-best method on each metric.

  • Expanded human-profile evaluation: Adding Llama strengthens the cross-model result: PATC achieves a nine-axis RMSE of .496 versus .633 for the next-best method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.