- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper presents an evaluation of the propensity of language models to assist users who seek to gain or shift power. It distinguishes three types of request: self-empowerment (SE), disempowerment (DE), and power grabbing (PG), along with control requests that shift no power. It then ablates on these request types depending on whether a third party would be affected by the request, and on the scale of that party. The authors suggest that model bias varies with the request type, with the identity of the affected party, and with the language in which the request is made.
Critical strengths
S1. The question is timely and important. The concern that small, systematic differences in assistance could compound at deployment scale is plausible and well motivated.
S2. The design is careful. Paired prompts that differ only in nationality, requester identity, or language, together with a control set, allow the authors to separate biases specific to power-shifting requests from general refusal tendencies. The panel is broad (24 models, balanced across developer countries), the judge is validated against human annotators.
Critical weaknesses
W1. What is the baseline? The paper never discusses how an "ideal" model should behave. My intuition is that PG and DE requests should always be refused and SE requests should always be accepted. Even if that is roughly right, the paper should include a broader discussion of this point, possibly supported by human validation.
W2. The paper mostly reports aggregated results. This means the reported findings may depend largely on the specific roster of models the authors chose, and may not generalize to other models. It is unclear how representative the 24 models are, and the pooled estimates may depend heavily on which models were included. It would be far more informative to report and focus primarily on the behavior of individual models, and to discuss the general behavior of the panel only occasionally.
W3. The claim that models are biased against the US is the most striking one in the paper and is stated in the abstract. Yet it is the one analysis without per-model signed estimates, in the main text or the appendix. Figure 18 shows unlabeled, unsigned per-model values, and Figure 10 splits the effect only by developer country. Again, I'd be more interested about the behavior of single models, than the average behavior of all models, especially because I want to understand which are the models driving this bias. Furthermore (W3a) The raw refusal difference between sides is at most 1.7 percentage points (Figure 2A), and the odds ratios are modest (0.79 to 0.84). (W3b) Table 17 shows that users and targets from China are refused more in every request type, the control included. The "anti-US" pattern arises from the difference between the target and user effects, not from a straightforward preference.
W4. Taken together, W1 to W3 mean the paper does not yet let a researcher, developer, or policymaker make an informed decision. Doing so requires both a baseline for expected behavior and a per-model view that identifies which models are most problematic and in which direction.
W5. Appendix A.2 states that language-model writer agents generated the requests from a specification. Validation was done mostly by LLM graders and auditors (Appendix A.3). Human involvement appears limited to reviewing the rewritten requests, and it is unclear how many requests a human actually read. How do we know that these queries reflect realistic scenarios? On a side note, (W5a) the main text also contains no example request, and Appendix D gives only one per dataset. Several examples per request type early in the paper would help both comprehension and validation.
W6. The authors write that all experiments have been run with reasoning turned off. They justify this by citing Greenwald et al. (1998), the implicit association test in humans, and Bai et al. (2025), implicit associations in LLMs. Neither supports measuring refusal with reasoning off. Both concern implicit association measures, a different construct from whether a deployed assistant helps a user. Modern models are run with reasoning turned on, so propensity evaluations should reflect this to obtain a realistic estimate of model behavior. The authors' own Appendix C.7 shows that enabling reasoning changes refusal substantially and unevenly across models. Unfortunately, I believe this choice invalidates the main findings. (W6b) Related, the authors claim to have turned off reasoning for all models, but for some of the listed models this is not possible according to their providers' documentation. For example, Grok-4.3 and Kimi-K3 reason by default and do not expose any option to disable reasoning, and Gemini 3.1 Flash-Lite offers "minimal" thinking as its lowest level rather than an off setting.
W7. The paper does not engage sufficiently with previous literature. Propensity evaluations, which measure what models would do rather than what they can do, are a broad and growing field, and the authors should situate their work within it. For instance: evaluations of honesty under pressure (MASK; Ren et al., 2025); PropensityBench (Sehwag et al., 2026), and Self Preservation (TBSP, Migliarini et al., 2026).
W8. The paper is very difficult to read. (W8a) The introduction and abstract are full of terms that are only defined later in the paper. They could be made leaner and focused on introducing the issue. In general, the abstract should be understandable to someone who has not yet read the paper. (W8b) The methods section is a barrage of statistics that does not help the reader and belongs in the appendix. (W8c) all results sections contain a wall of numbers and acronyms which are unhelpful to understand the claim, and the plots barely make this better. The writing needs to be greatly improved before being considered for acceptance
W9. The capability index the authors construct is unusual. Why combine GPQA with a subset of MMLU-Pro? It would be cleaner to use GPQA alone, or an established index such as the Epoch Capabilities Index or the Artificial Analysis Intelligence Index. Moreover, the results the authors report on these benchmarks are surprising, especially for MMLU-Pro, which is close to saturated. The scores reported in this paper do not match publicly reported results. Notably, grok-4.3 is publicly reported at 90.1% on GPQA Diamond, whereas the paper reports 48.5%.
Questions for authors
All of the above, and also:
Q10. What happens if the model is explicitly instructed to refuse power-shifting requests, or explicitly instructed to be unbiased? I would of course expect refusal to increase, but does some residual bias remain, and how large is it?
2 comments
Thank you for carefully reading the revision and for these two remaining points, which we have addressed in a new revision, now uploaded.
RW1. We agree. We have removed "favoring power flow away from the US" from the abstract, and we now focus on the same-user US-target bias in the corresponding bullet point of the Introduction. Section 3.2's paragraph on this now opens with the result of Table 21, given directly in the main text, instead of being left only to the appendix. Additionally, Section 3.2 now explicitly states that the direct US-China comparison shows no significant bias.
RW2. All 24 models received the same parameter through OpenRouter: 'reasoning: {enabled: false}'. The endpoints we used throughout the study are listed in Appendix Table 5, so this configuration can be reproduced. Indeed, Moonshot's documentation says that they don't support the thinking parameter, but this refers to their own API. Instead, we accessed kimi-k3 through BaseTen, whose endpoint lets reasoning be switched off. We did not rely on this parameter, but verified this thoroughly to be sure all models answered without reasoning tokens, and excluded models where this was not the case. We base our confidence on three facts: 1) Every response returned zero reasoning tokens, for all models. 2) On the GPQA Diamond questions, kimi-k3 was billed a median of 7 output tokens per question, and since reasoning tokens are billed as output tokens, this leaves no room for hidden reasoning. The same endpoint with reasoning enabled returned reasoning tokens on all 198 questions, with a median of 164 output tokens. 3) GPQA Diamond accuracy rose from 59.1% (reasoning off) to 84.8% (reasoning on), which strongly suggests that reasoning produced the expected increase in performance. We added this explanation to Appendix A.9.
Thank you for the thorough revision. Most of my concerns have been addressed, but two points remain.
RW1. Framing of the US result. The pooled anti-US bias in DE and PG is now well supported, but the paper presents it more broadly than the evidence allows. The direct US--China comparison is null (Table 20: DE OR 0.81, q = 0.37; PG OR 0.82, q = 0.36). Only the comparison with rivals differs from the control (Table 14). Raw refusal differences are also at most 2.1 percentage points. The clearest evidence is Table 21: with the user held fixed, US targets are refused less than neutral or US-allied targets. I suggest leading with this result, stating explicitly that the US-China comparison is null, and softening "favoring power flow away from the US" in the abstract. The study measures refusal asymmetries, not power flow. Additionally these results should not be buried in the appendix, but presented more clearly in the main paper.
RW2. Reasoning off. The authors state that reasoning was disabled for all models, but providers report this is not possible for some of them. For Kimi-K3, Moonshot says it "always reasons and does not support the thinking parameter" (https://platform.kimi.ai/docs/guide/use-thinking-models). Please report the exact parameter used for each model, and how you were able to run these models with reasoning disabled, if at all.