- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
- The authors aim to understand the internal structure of conflicts between different values that a language model might face during deployment, as measured by model activations. This could enable evaluating model adherence to new value pairs without having to collect additional data.
- To do so, they fit value-specific loadings on a basis learned from model activations on different source pairs, and find that the difference in loadings between two values is reflective of the conflict. Critically, this is not directly computed using the conflicting pair, which the authors use to claim generalization from composing individual value vectors.
- The authors also show that these directions can be used to steer models toward each individual value within a conflicting pair (for Qwen2.5-7B across eight pairs), and that they form a low-rank subspace in the model activations. However, they find limited generalization across different types of value conflicts, such as those involving protective values.
Critical strengths
While past work has extracted behavior-specific activations, the observation that behavioral activations can be composed and used to steer model behavior in new contexts is understudied in past work. A method that does this would be useful for constitution design if sufficiently validated across values.
Critical weaknesses
- Evaluation relies on unvalidated manual prompts despite the existence of a number of existing established value conflict benchmarks, including “DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life” (Chiu et al, ICLR 2025), “Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas” (Chiu et al, ICLR 2026), “Generative Value Conflicts Reveal LLM Priorities” (Liu et al, ICLR 2026), “CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives” (Lee et al, ICLR 2026), “Stress-testing model specs reveals character differences among language models” (Zhang et al, 2025), etc., which are not referenced. It’s unclear how the final list of values was decided, how the prompts were created, and how the authors validated that the prompts appropriately tested adherence to the values, which are all central to the evaluation. Using existing benchmarks could allow the authors to scale the method beyond the relatively small number of pairs evaluated.
- It’s not clear that the proposed method is identifying any fine-grained differences between values beyond a high-level helpfulness-harmlessness direction. The results seem very low-rank (the authors show that rank-2 does not improve upon rank-1), and helpful and protective values seem to generally map onto the sign on a single axis. While the authors show that the effect is not just refusal, there are many other ways to softly comply or not comply with a request. Results showing generalization to other latent bases of LLM values would be more interesting and novel.
- The paper is very technically dense and difficult to parse.
Questions for authors
- Why was a bag-of-words baseline used in Section 5.2, rather than stronger/more modern baselines such as sentence embedding models or existing value-specific extraction methods such as persona vectors?
- How does the proposed method compare to just using low-rank representations of raw activations (e.g. via dimensionality reduction on the set of pooled activations)?
- How was the LLM judge (Kimi K2.6) validated?
3 comments
Author Response to Reviewer RRS1 (Part 2/2)
Why was a bag-of-words baseline used in Section 5.2, rather than stronger/more modern baselines such as sentence embedding models or existing value-specific extraction methods such as persona vectors?
How does the proposed method compare to just using low-rank representations of raw activations (e.g. via dimensionality reduction on the set of pooled activations)?
Bag of words is an interpretable lexical control that receives the same request, clause orientation, and response prefix. It tests whether a simple surface-text predictor explains the monitoring result. It is not evidence of superiority over modern sentence encoders. Likewise, the main composition claim does not depend on beating this baseline: it is evaluated through held-out-pair prediction, target-pair-fitted references, and activation interventions.
Persona vectors are relevant precedents for extracting behavioral directions. CVP additionally imposes the relation across pairs and tests the resulting held-out prediction. This is a distinct structural question, not a demonstrated head-to-head advantage over persona vectors.
Raw-activation PCA is also a different objective: it preserves activation variance, whereas our SVD operates on normalized, outcome-fitted source directions. The latter emphasizes variation associated with conflict outcomes. This methodological distinction motivates the construction; it does not substitute for an empirical claim that CVP outperforms PCA.
How was the LLM judge (Kimi K2.6) validated?
The generator remains frozen. Kimi applies explicit definitions and a behavioral rubric without receiving the system priority. The submission reports that alternative-rubric re-judging preserves 98.6% of decisive binary labels, which establishes within-judge stability. To validate the judge beyond this check, we added an independent-judge and human validation on Qwen2.5-7B in the instruction context of the sixteen-pair graph. Claude Haiku 5.5 re-judged all 12,480 responses with the same prompt and rubric. During the rebuttal period, two annotators from our group independently labeled a pool of 1,000 responses, blind to the system priority and to the judge scores. From this pool we sampled 300 responses from three helpfulness/autonomy pairs and three protective-value pairs (25 per pair and clause orientation). Each label marks the response as preserving the first value more, the second value more, or neither clearly. We report label agreement, and the AUROC of readouts fitted and calibrated on Kimi labels when evaluated against each judge's labels.
| Comparison | Responses | Decisive-label agreement | Cohen's κ (three-way) |
|---|---|---|---|
| Kimi vs. Claude Haiku 5.5 | 12,480 | 0.87 | 0.74 |
| Kimi vs. human | 300 | 0.78 | 0.61 |
| Claude Haiku 5.5 vs. human | 300 | 0.81 | 0.65 |
| Human vs. human | 300 | 0.84 | 0.70 |
Comparisons with human labels average over the two annotators.
| Readout (fitted on Kimi labels) | Pairs | Kimi labels | Claude Haiku 5.5 labels | Kimi labels (annotated subset) | Human labels (annotated subset) |
|---|---|---|---|---|---|
| Source-only composition, full response | Helpfulness / autonomy | 0.821 | 0.793 | 0.78 | 0.762 [0.691, 0.827] |
| Source-only composition, full response | Protective-value | 0.708 | 0.681 | 0.68 | 0.637 [0.558, 0.713] |
| Target-supervised monitoring, prompt state | Three monitoring pairs | 0.847 | 0.812 | 0.79 | 0.771 [0.703, 0.832] |
The first two label columns cover all eligible Qwen2.5-7B pairs. The annotated-subset columns cover the six annotated pairs and use responses on which both annotators give the same decisive label; human-label AUROC reports request-bootstrap 95% intervals.
Haiku agrees with Kimi on 87% of decisive labels (κ = 0.74), and Kimi's agreement with the annotators (κ = 0.61) is close to inter-annotator agreement (κ = 0.70). Readouts fitted on Kimi labels lose 0.027–0.035 AUROC under Haiku labels. On the annotated subset, human-label AUROC is 0.018–0.043 below Kimi-label AUROC and remains above chance in all three settings. The largest gap and the lowest value are on protective-value pairs (0.637 [0.558, 0.713]), consistent with the weaker composition reported for these pairs.
The supported contribution is a lightweight method that composes useful readouts without target-pair supervision, together with evidence identifying where its shared representation succeeds and breaks down.
Thanks to the authors for their response. Unfortunately, the rebuttal does not meaningfully address my major reservations with the submission (which I elaborate below), so I am maintaining my recommendation of clear reject.
R1 (Clarity): To be clear, I understood the method but feel that the paper is incredibly difficult to parse. This risks seriously limiting the paper's usefulness to the broader ICLR community. For example, the paper frequently invents new technical terms with insufficient definition that it only uses sparingly (e.g. "source-selected full-response layers"). Additionally, quantitative results are frequently presented in quick succession without substantial analysis, making it difficult for the reader to draw clear conclusions from the work (e.g. "Extrapolation from the ten-pair graph predicted non-manipulation above wellbeing, confirmed in Qwen2.5-7B and 14B, but placed honesty and fairness above privacy in the opposite order to the observed directions. Privacy and fairness each had only one source edge. For honesty–privacy, the ten-pair graph gives the wrong sign in 88% of request-bootstrap replicates at Qwen2.5-14B and 85% at Mistral. The sixteen-pair graph improves recovery (Figure 6b). These extrapolations use the ten-pair graph to predict new relations. Section 5.1 instead holds out one pair at a time from the sixteen-pair graph.") The rebuttal re-summarizes the paper method, which does not address this concern.
R2 (Relation to Existing Benchmarks): It's still not clear to me why the authors created their own benchmark rather than using existing established value conflict benchmarks, as the authors explicitly concede that "Existing scenarios could support this question with an appropriate value mapping, source graph, and experimental split" in the rebuttal. Additionally, critical details about the dataset construction process remain unknown. The authors acknowledge in their response that this is a "presentation and reproducibility shortcoming", but have not revised the paper or provided methodological details that would clarify this in the rebuttal. While they do promise to include more information about this in a future revision, we need to be able to understand how the benchmark was actually constructed in order to assess whether a paper built on the benchmark should be accepted, so promising to clarify this later does not meaningfully address my concerns.
R3 (Low Rank): I originally wrote that "It's not clear that the proposed method is identifying any fine-grained differences between values beyond a high-level helpfulness-harmlessness direction." This was based on the paper's Figure 4(a), which suggested that a rank-1 version of the method performed similarly to a rank-2 version of the method, and even outperformed it in some cases. Showing a single example of the method predicting a different conflict does not meaningfully refute this, and the authors concede that "a dominant fulfillment-versus-protection tendency is a plausible interpretation", so I still have this concern.
R4 (Baselines): To assess whether Constitutional Value Potentials is a useful method for studying the internal structure of different value conflicts, the authors should compare their method to strong existing baselines that could be applied to the same task. In my review, I suggested that the existing baseline (bag-of-words) was insufficient, and that directly comparing to persona vectors, low-rank representations of raw activations, and sentence embedding models without privileged access to activations would strengthen this result. The authors have replied by describing the baselines I suggested, but have not actually evaluated them on the paper's tasks, so this doesn't address my concern at all.
R5 (Judge Validation): The comparison to human-annotated responses is genuinely useful, and I appreciate the authors clarifying that their LLM judge was validated. I'd recommend that the authors revise the paper to include this information.
Based on these major unresolved concerns, I continue to believe that this paper should not be accepted to ICLR.
Author Response to Reviewer RRS1 (Part 1/2)
Thank you for the thoughtful review. We appreciate your recognition of compositional transfer as the potentially useful contribution, and the concrete suggestions about evaluation and baselines. We agree that the presentation should make the scientific question and supervision boundaries easier to follow. Below we respond point by point to each concern.
R1: The Method in Three Steps
Response: The method can be summarized in three steps: fit directions for observed conflicts, extract their shared basis, and fit value loadings whose differences reconstruct those directions. A held-out pair is then read through the difference of its two loadings. The central experiment asks whether this construction predicts outcomes when that pair's labels were unavailable for fitting and calibration.
R2: Relation to Existing Value-Conflict Benchmarks
Response: The cited datasets are relevant, and their connection to our work deserves explicit treatment. In particular, ConflictScope$^{[1]}$ evaluates priorities through generated value-conflict scenarios, and CLASH$^{[2]}$ studies value reasoning and steerability. Our contribution concerns the activation-space composition of relations across held-out value pairs. Existing scenarios could support this question with an appropriate value mapping, source graph, and experimental split; their relevance is not in dispute.
Our dataset provides a controlled implementation of that test. It uses ten operationally defined assistant-relevant values, with the sixteen-pair graph including six additional protective-value pairs that challenge the dominant shared component. Requests are concrete tasks, and the user request remains unchanged when the priority instruction is reversed. The expanded construction pipeline includes model-assisted drafting, screening for task form and intended conflict, and manual additions. These are construction controls, not independent human validation. The complete prompt and rubric materials are already assembled; the missing access link and insufficient explanation of provenance are presentation and reproducibility shortcomings.
References: [1] Liu et al., "Generative Value Conflicts Reveal LLM Priorities," arXiv:2509.25369. [2] Lee et al., "CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives," ICLR 2026.
R3: What the Low Rank Establishes
Response: We agree that a dominant fulfillment-versus-protection tendency is a plausible interpretation. Low rank does not establish a separate semantic direction for every value. However, it also does not imply that all protective-value distinctions are unrepresented. Source-only honesty–privacy transfers, while fairness–privacy exposes substantial failures:
The rank-two ablation covers five eligible pairs from the ten-pair graph; its small average gain does not establish that every value conflict is intrinsically one-dimensional. Our evidence supports partial composition and identifies its limitations.