- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes cognitive steering - using cognitive models as data generating processes to steer language models. For 9 models with 16 parameters, the authors sample the models' choice distributions, which are then used to train seven interventions on Llama3.1-8B and Qwen3-8B, including hard and soft prompts, contrastive steering baselines, optimized steering vectors an affine map and LoRA distillation. Of these, optimized vectors, affine maps and distillation recover target parameters better than an unsteered model within distribution. Out of distribution, steered models write free text in realistic scenarios, and then an LLM judge compares responses pairwise on parameter's quality. The results show that distillation gives the steepest slops for most parameters.
Critical strengths
Goal of the paper is generally appealing - using interpretable and empirically validated model as steering targets instead of ad hoc descriptions and contrastive pairs seems very compelling. The paper also provides a broad set of cognitive models and seven interventions.
Critical weaknesses
It seems that the evaluation contradicts the motivation of the paper in a way, as OOD parameters are scored by an LLM judge against natural language descriptions, which the paper argues is imprecise and biased. Also cognitive model value is in computations, not just parameters themselves - the present test only assesses wither a quality increases with the target, but doesn't really test if the parameter effects depend on the context, or whether the trade-offs between parameters hold.
Questions for authors
-
Could you evaluate a prompting baseline out of distribution? That can be a system prompt containing same parameter descriptions the judge uses at matched low and high targets?
-
What does the prepended header contain for distillation, and does it include semantically meaningful parameter names? If so, could this be influencing the results?
-
Since the LLM judge scores outputs of natural language descriptions, how does the OOD evaluation avoid the problems the introduction attributes to linguistic specifications? Can you validate judge decisions against human ratings on a subset?
-
Since each scenario has a factorial variant of the situation, do the steered models show context dependence the cognitive models predict?
Not a question but there seems to be a citation error on line 291.