- Rating
- 3(+1)
- Confidence
- 1(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Structure-Aligned Steering, which encodes hidden states of a frozen language model into a low-dimensional space with explicit task structure, performs task-specific operations in that space, and decodes the results back into the model. Experiments on weekday cycles, number cycles, RGB colors, and modular arithmetic demonstrate the feasibility of incorporating explicit task rules into structured activation control.
Critical strengths
- The experimental evaluation is extensive and provides reasonably convincing evidence. The paper evaluates four tasks across eight models and includes supplementary analyses of intervention layers, random seeds, structural perturbations, and coordinate-region holdouts. The authors also report failure cases and transfer limitations, helping readers assess the scope of the findings. (§4, Appendices D–J)
- The framework is clear and offers a useful direction for future research. The encode–operate–decode design is intuitive and accommodates different task structures, including cycles and color spaces, within a common interface. It provides a useful approach to incorporating explicit task structure into language-model activation control. (§3)
- The evaluation of structured control goes beyond final-answer accuracy. The authors examine continuous interpolation, repeated operations, cycle closure, and distance preservation. The weekday experiments demonstrate the feasibility of continuous stepping and cyclic composition, while the broader analyses distinguish output correctness from structural consistency, providing a valuable evaluation perspective for future work. (§§4.1–4.2, Appendices A.4 and E.6)
Critical weaknesses
- The fairness of the baseline comparisons needs further examination. The proposed modular arithmetic system receives operand-level supervision and uses a predefined exact modular addition rule, whereas LoRA and Prefix Tuning are trained only on final answers. Matching parameter counts and training examples therefore does not isolate the contribution of the geometric interface: the performance gap may also reflect additional supervision and built-in computation. Comparisons with baselines using the same supervision and computational module would strengthen the evidence. (§4.3, Appendices B.6 and C.7)
- Generalization remains limited. Appendix G.6 reports that average modular arithmetic accuracy across models drops from 95.5% on the original prompt template to 30.4% on new templates, indicating substantial sensitivity to input phrasing. Moreover, experiments with different moduli and operand ranges retrain the entire system, providing limited evidence of transfer without retraining. (Appendices G.5–G.6)
Questions for authors
- Learning geometric structure: Both the input construction and output supervision contain task geometry. Could the authors include a fixed coordinate extractor with a learned decoder—for example, recovering coordinates from RGB mixtures using a linear pseudoinverse—to clarify the need for a learned encoder? Would similar geometric organization and performance emerge if semantic anchors were replaced with random linearly independent vectors?
- Structured baselines: Could the authors compare against a codebook-based method using the same inputs and supervision, which maps inputs to discrete entries, applies predefined cyclic or compositional rules, and directly outputs the corresponding labels? Appendix G.4 already shows that direct anchor injection matches the learned decoder on discrete modular arithmetic. This additional comparison would help clarify the unique benefits of the continuous geometric interface for task accuracy, continuous control, and compositional generalization. A good answer will improve the score.
- Training fairness and transfer: Could the authors report the full training budgets for the proposed method, LoRA, and Prefix Tuning, including interface pretraining, optimization steps, wall-clock time, and hyperparameter search ranges, and evaluate whether the baselines improve with more extensive training? For new prompt templates, could the authors also report baseline transfer performance and use oracle operands or oracle results to determine whether the proposed method’s degradation primarily arises from readout or write-back?
7 comments
W2 and Q3 (generalization and transfer)
Thank you for the suggestion to separate read-out errors from write-back errors on new templates, and for pointing out that our experiments on other operand ranges and moduli retrained the whole system. We again use Qwen3-4B at layer 27.
(a) Source of the degradation on new templates. Following your suggestion, we separate read-out errors from write-back errors: we replace the aligner's prediction by the true residue, and also the decoder by the constructed anchor of that residue. We evaluate on the original template and the four new templates of App. G.6 (500 test prompts each), and run the LoRA and prefix adapters selected in our reply to W1 on the same prompts.
| template | aligner residue accuracy | full system | true residue + decoder | true residue + anchor | LoRA | prefix tuning | unmodified model |
|---|---|---|---|---|---|---|---|
| original | .872 | .872 | 1.000 | 1.000 | .576 | .358 | .246 |
| words | .442 | .434 | .922 | 1.000 | .418 | .302 | .236 |
| remainder | .322 | .266 | .802 | 1.000 | .294 | .350 | .354 |
| function | .140 | .140 | 1.000 | 1.000 | .274 | .236 | .214 |
| instruction | .268 | .268 | 1.000 | 1.000 | .338 | .234 | .194 |
When the true residue is given, the write-back is correct on all or most prompts of every template, and full-system accuracy follows the aligner's residue accuracy. The degradation therefore comes from the operand read-out, trained on a single template, and not from the interface. LoRA and prefix tuning degrade as well, and have no component that can be retrained separately.
(b) New templates: retraining the aligner only. We keep the steerer frozen and retrain the aligner for one epoch on 18,000 prompts drawn from several templates, once with the evaluated template left out and once with all five templates included (full-system accuracy).
| template | aligner of the paper | retrained, template left out | retrained on all five templates |
|---|---|---|---|
| original | .872 | – | .810 |
| words | .434 | .602 | .738 |
| remainder | .266 | .354 | .618 |
| function | .140 | .556 | .802 |
| instruction | .268 | .494 | .814 |
Retraining the aligner alone recovers a large part of the accuracy on the new templates, also when the evaluated template was left out, and the steerer is not touched. The original template loses some accuracy because it now receives only a fifth of the retraining prompts.
(c) New operand ranges: retraining the aligner only. The paper retrained the whole system for the operand ranges 0–39 and 0–49. Here we keep the mod-7 steerer trained with operands 0–29, and retrain only the aligner for one epoch.
| operand range | aligner of the paper, not retrained | aligner retrained only | true residue + decoder | whole system retrained (paper) |
|---|---|---|---|---|
| 0–39 | .598 | .924 | 1.000 | .910 |
| 0–49 | .460 | .930 | 1.000 | .942 |
The write-back of the frozen steerer is exact on the new ranges, and retraining the aligner alone matches retraining the whole system.
(d) New moduli: retraining the aligner only. A steerer has one anchor per residue, so each modulus k needs its own steerer, and we use the mod-k steerers of the paper. The aligner predicts the two operands and does not contain the modulus, so we ask whether it must be retrained with the steerer: we reuse the mod-7 aligner without any training, and retrain only the aligner on mod-k prompts for one epoch.
| modulus k | mod-7 aligner, not retrained | aligner retrained only | true residue + decoder | whole system retrained (paper) | unmodified model |
|---|---|---|---|---|---|
| 5 | .516 | .946 | 1.000 | .954 | .648 |
| 6 | .702 | .908 | 1.000 | .934 | .372 |
| 8 | .756 | .958 | 1.000 | .932 | .308 |
| 9 | .548 | .912 | 1.000 | .952 | .338 |
Without training, the mod-7 aligner works only in part, because "mod k" in the prompt shifts the states it reads, as a new template does. Retraining the aligner alone matches retraining the whole system, and the write-back with the true residue is exact for every k.
Cost of retraining (one H200).
| what is retrained | training time |
|---|---|
| aligner only (one epoch, with hidden-state extraction) | 13 to 16 s |
| whole system (steerer and one-epoch aligner) | about 120 s |
In all three cases, a change of the prompt distribution is handled by retraining the lightweight aligner at a small fraction of the cost. This is the modularity the paper claims: the steerer is reused unchanged, and only the read-out depends on the prompt.
Q1 (learning geometric structure)
Thank you for the suggestion to demonstrate the necessity of the encoder by comparing it with the pseudoinverse and random-anchor baselines. On the paper's tasks they show that a learned encoder is not required, so we also added a task in which it is.
(a) The two controls on RGB and Number. We use Qwen3-4B at layer 27 with five seeds and the paper's settings otherwise. The fixed extractor replaces the encoder by the frozen linear pseudoinverse of the anchor basis, and only the decoder is trained. Random anchors replace each token's unembedding direction by a random Gaussian vector of the same norm, with the encoder and decoder trained as usual.
| encoder / anchors | RGB accuracy | RGB distance correlation | Number accuracy |
|---|---|---|---|
| learned / semantic (paper) | 70.3±9.8% | .81 | 97.9±1.2% |
| fixed pseudoinverse / semantic | 73.8±1.7% | 1 (by construction) | 97.9±0.4% |
| learned / random | 63.4±9.7% | .81±.06 | 95.0±1.2% |
The fixed pseudoinverse matches the learned encoder, and with random anchors the encoder still arranges its inputs by the task geometry, with somewhat lower accuracy. The reason is that the inputs of these tasks are anchor mixtures: the coordinates are linear in the input, so a linear extractor is exact, and any linearly independent vectors parameterize the task domain equally well. This agrees with the paper's theoretical framework, which neither assumes semantic anchors (Thm. 1) nor singles out one internal chart (App. A.1).
(b) A task in which the encoder must be learned. A learned encoder is needed when the input is a natural hidden state in which the task variable has not been computed. We therefore steer on prompts such as "(417, 286) -> ", with operands a and b between 0 and 999 and target residue r = (a + b) mod 100. The prompt does not name the operation, so the unmodified model is at chance. We use Llama-3.1-8B, for which every number from 0 to 999 is a single token, and read the last-token state at layer 24 (three quarters of the depth, as in the paper). The encoder maps it to the coordinates , the position of r on seven circles. The model represents numbers on circles of these periods, and a single circle of period 100, in the manner of the paper's mod-7 task, fails here (our reply to Reviewer dazJ, W2 and Q2, explains how the periods are chosen). The decoder writes a state back from them, and we train with where is passed to the decoder without gradient, is the soft target of Eq. 7 over the training residues, and is the model's mean state when it is about to answer r, with its typical norm; the three terms follow the paper's losses (Eqs. 6 to 8). Details are in Sec. 4.3 and App. C.9, G.12 of the revised paper.
To test generalization, 20 of the 100 operand classes (an operand's last two digits) never appear in a training prompt, and independently 20 of the 100 residues never appear as a training answer. We compare three ways of reading the state, each with a decoder of the same architecture and loss. Ours is the learned encoder. The fixed linear extractor is the pseudoinverse map fitted on the same states and coordinates. Random input vectors is the counterpart of random anchors: every operand value gets a fixed random vector, and the encoder reads the sum of the two vectors instead of the model's state. Accuracy is the fraction of test prompts on which the model's top-1 token after the write is the correct residue (mean ± sd over five seeds).
| read side | all seen | an operand unseen | residue unseen | both unseen |
|---|---|---|---|---|
| ours | 1.000±.001 | .991±.003 | .949±.042 | .925±.037 |
| fixed linear extractor | .433±.028 | .223±.011 | .041±.011 | .020±.006 |
| random input vectors | 1.000±.000 | .006±.002 | .949±.053 | .004±.003 |
The fixed linear extractor is lower in every column: the position of the residue on a circle is obtained by adding the angles of the two operands, which is a nonlinear function of how the operands are represented that a linear map cannot compute. Random input vectors are perfect when both operands were seen and at chance when one was not: a network can memorize a code for every seen value, but the vector of an unseen value carries no information. The learned encoder stays accurate on operands it never saw, so it relies on the model's own representation of numbers. Together with (a): anchor mixtures need only a linear extractor, whereas natural states, in particular with operands unseen in training, need the learned nonlinear encoder.
Dear Reviewer KZZK,
Thank you for your detailed review. Our responses follow; the changes to the paper are in Revision Notes 3, 4, 5 and 7.
W1 and Q3 (training fairness)
Thank you for the suggestion to better demonstrate the contribution of our steerer by matching the training budgets of the steerer and the PEFT baselines. All experiments below use Qwen3-4B at layer 27.
(a) Budgets in the submission. We first report the budgets and hyperparameters.
The aligner and the baselines were matched in examples, epochs, batch size and steps. Two things were not: the steerer has 2,000 additional steps, and the baselines' learning rate was fixed, not tuned. Experiment (b) removes both.
(b) Longer training with a tuned learning rate. We now train the baselines much longer, for 20 epochs, and sweep the same five learning rates (1e-4, 2e-4, 5e-4, 1e-3 and 2e-3) for every method. The learning rate is selected on a validation split, and the test set is evaluated once. LoRA is run with two scalings, α = r/2 as in the paper and α = 2r. For ours, the aligner is trained for 10 epochs, and the 2,000 steerer steps with batch size 16 correspond to about 1.8 epochs of the 18k prompts, so our method uses about 11.8 epochs in total, far fewer than the baselines. Training time is for one learning rate on one H200 and, for our method, includes the extraction of hidden states and the training of the steerer.
More training does not close the gap. LoRA fits the training prompts perfectly, but its test accuracy stays far below ours. The test operand pairs do not occur in training, so LoRA lacks generalization to new pairs, not optimization, whereas our steerer achieves it. Our method also needs far less training time.
(c) The same operand supervision for LoRA. We add to LoRA an auxiliary head with the architecture of our aligner, which predicts the two operands from the output of the adapted layer. The training loss is the cross-entropy of the answer plus that of the two operands. Data, epochs, learning-rate grid and selection rule are those of (b). The head is discarded at inference, so the reported accuracy is that of LoRA's own answer.
Operand supervision brings LoRA a further gain, but it stays below ours. The remaining gap is not a lack of information: LoRA lacks a way to turn the result of the rule into the model's answer (the operands predicted by the auxiliary head, passed through the exact modular rule, give the correct result on 99.6% of the test prompts), and this write-back is what the interface provides. This is the advantage of the steerer over PEFT: a computation carried out exactly outside the model can be written back into it. In line with your point, the distinctive advantage of our system is therefore the combination of a read-out with operand supervision, an exact rule, and the write-back through the interface. The revised paper presents the gain in this way (see Sec. 4.3 and App. G.1).