- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies design choices for diffusion-based meta-models of LLM activations. It makes three main contributions: a velocity-to-x (v2x) parameterization that improves Fréchet distance, evidence that more task-relevant activation priors improve SDEdit-corrected steering, and a systematic characterization of the interaction between steering strength and diffusion noise level. The paper evaluates these choices primarily on Llama-3.2-1B activations and studies both generative quality and downstream activation steering.
Overall, I found the paper interesting and appreciated the breadth of the empirical study, particularly the systematic exploration of diffusion design choices and steering behavior. However, I have several concerns about how the main results should be interpreted.
Critical strengths
The paper provides a broad and systematic empirical investigation of diffusion-based activation meta-models. I particularly appreciate that the authors explore several design axes, including parameterization, loss weighting, training distribution, model scale, and SDEdit noise strength.
Another strength is the negative result that substantially better Fréchet distance does not translate into better downstream steering. If I am interpreting this result correctly, it is useful evidence that generative-quality metrics may not be reliable proxies for the downstream utility of activation diffusion models.
Critical weaknesses
-
It is not clear to me that the trait-aligned prior result isolates "projection" from direct trait injection.
My reading of the paper is that the trait-specific GLPs are trained on large corpora whose responses already strongly express the target trait, including activations extracted in the presence of the trait-eliciting system prompts. In contrast, the steering vectors are estimated from only 20 QA pairs. Unless I have misunderstood the setup, it therefore seems possible that the trait-specific GLP itself contains substantial information about the desired behavior and contributes additional steering, rather than merely projecting an off-manifold activation back toward a plausible activation distribution. I may have overlooked a control, but I could not identify an experiment that cleanly separates these possibilities. GLP-only (), mismatched/cross-trait priors, or steering vectors constructed from the same large trait corpus would substantially help clarify this point. -
I am not fully convinced that the downstream comparison supports the conclusions drawn about v2x.
Figure 15 appears to show no steering advantage for v2x, but, as far as I understand, both standard FM and v2x are trained at the same learning rate in this experiment, whereas Table 1 identifies substantially different best learning rates for the two methods. I would be hesitant to conclude that v2x provides no downstream benefit without a comparison in which each method is independently tuned, ideally across multiple training seeds. More broadly, the proposed v2x parameterization seems closely derived from EDM. it retains the EDM-style pre-conditioning and interpolation structure while replacing the low-noise endpoint with flow-matching velocity prediction and normalizing the target variance. I think the methodological novelty should therefore be positioned carefully, particularly given that its main demonstrated improvement is currently limited to the FD metric.
Questions for authors
-
It would be helpful if the authors could clarify whether the trait-specific GLP changes trait expression when applied with . Have you also tried mismatching the steering vector and the trait prior (e.g., a pessimistic steering vector with a deceptive GLP)? These controls seem important for distinguishing generic manifold correction from direct behavioral steering by the prior.
-
I may be missing a technical reason for using a shared learning rate in Figure 15, but given the different optimal learning rates reported in Table 1, could the authors provide a downstream comparison where standard FM and v2x are independently tuned? Would the conclusion that their steering performance is essentially identical still hold?
-
From my reading, some choices such as fine-tuning epoch count and the system-prompt-included extraction variant were selected based on downstream steering performance. Were these decisions made using a separate validation set from the 20 questions used for the reported evaluation? If not, how sensitive are the reported Pareto improvements to these choices?
-
The current evaluation is primarily on one 1B model, one layer, 20 evaluation questions per trait, and one LLM judge. I understand that exhaustive replication is expensive, but it would help clarify which conclusions the authors expect to generalize beyond this particular setting, especially given the differences from the original GLP results on Llama-8B.
3 comments
We would like to remind the reviewer that the discussion period is approaching an end, and we would be happy to address any further questions! We also hope that the responses and additional experiments provided address the main concerns.
To recap, regarding the weaknesses: We provided additional experiments isolating trait injection from projection, provided additional experiments w.r.t. v2x and clarified the contribution: v2x mainly improves the FD score, which also shows that the FD score does not directly translate to improved steering performance. We also provided additional experiments aimed at answering the additional questions the reviewer raised.
Thank you for your additional experiments and efforts. You addressed most of my concerns. I found this paper interesting and learned something from it, though I still have concerns about it's novelty level. Overall, I am positively inclined toward the work.
We thank the reviewer for the constructive feedback. We have worked to address the issues noted, and list them below in order:
W1 / Q1. Does the trait-aligned prior project, or inject the trait?
We ran the three controls the reviewer proposes, on all five traits (App. J.6).
Our reading: the prior does not significantly steer by itself, but the reviewer is right that a matching prior carries information about the steered behaviour, which shows up only together with the steering vector. A prior closer to the steered distribution preserves more of the steering.
W2 / Q2. v2x against standard FM, independently tuned
We trained both methods at their own best learning rates from Table 1 (1.25e-5 for v2x, 2e-4 for standard flow matching) with three seeds each and evaluated all five traits (App. J.7, Figs. 32–33, replacing the single-seed figure of the submission). The conclusion holds: v2x has the lower FD (0.24 against 0.33) and no steering advantage. Averaged over traits and α at t = 0.5, standard flow matching is 2.9 points more coherent with a 1.5 points lower trait score, and it is ahead on both axes at high α for evil and pessimistic (α = 4: 46 / 47 against 38 / 41 for evil).
Novelty. We agree that v2x is not, by itself, a significant deviation from standard diffusion model practice. However, we are not aware of a paper that uses velocity prediction at low noise levels and x-prediction at high noise levels, and we were surprised that this is significantly more effective than EDM on activation data, when it comes to the FD metric. Similar relatively small changes in parameterisation are sometimes found to result in large improvements, which is why we are interested in it, not necessarily due to significant algorithmic innovation. We have now clarified and narrowed the positioning of v2x in the contributions section of the paper.
Q3. Were epoch count and trait-aligned activation extraction method chosen on a validation set?
No, in the submission they were chosen on the 20 evaluation questions. We redid this with separate validation and test questions (App. J.2). On validation, 10 against 20 epochs and extraction with against without the system prompt differ by about one point of our summary statistic (Table 6), and priors trained on unfiltered or differently filtered data give the same test curves (Fig. 26). The improvement of the trait-aligned prior over the Alpaca prior for evil, pessimistic and deceptive is also present on the test questions (Fig. 23).
Q4. Scope