- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a novel auditing pipeline for detecting conditionally activated biases that could be introduced during the fine-tuning process of an LLM. The authors call this "guided benchmarking". Unlike standard behavioural benchmarks that use model-independent contexts for eliciting biases of a model, the authors consider situations where the bias might only be appearing during a specific context and remain hidden otherwise.
The proposed guided benchmarking is conducted in three stages. First, a weights of the models are analysed to identify layers that are susceptible to a particular task direction. Activation differences are then computed. Finally, Patchscope decoding is applied to generated LLM-guided contexts of the targeted probes.
Critical strengths
The main strength of this paper is its novel proposal (guided benchmarking), which can in principle identify biases that might only appear under specific conditions. As far as I am aware this is a novel way to investigate biases in LLMs and can complement the existing static bias evaluation benchmarks.
On the methodological side, the dual approach for localising the largest weight changes (i.e. hot-layer selection) and removing effects from fine-tuning artefacts is an interesting and potentially useful innovation in similar problems.
Empirical results are overall strong and shows the effectiveness of the proposed method over baselines such as ADL over multiple datasets. I also found the ablations to be thorough and systematically considering the different settings of the proposal.
Critical weaknesses
Although I cannot point out "critical" weaknesses, I thought not investigating latent social biases (e.g. gender, race related biases) was a missed opportunity in the paper. Biases related to "Cats", "oak", "UK" etc. are not that all interesting or impactful compared to ones that have social implications (and legal ones as well if used to make decisions affecting humans).
Questions for authors
-
Do you have any thoughts about how to perform guided benchmark when you do not have access to the model weights (such as in the case of API only models)?
-
Why do not you consider latent social bias evaluations (I see that you do mention this in the related work section) as an evaluation task? Will the proposed method be effective for that too?
Author Response
We thank the reviewer for recognizing the value of guided benchmarking and its potential to complement static evaluations. We provide clarification for the weakness and questions as below:
W1 / Q2. Social biases.
We agree that demographic biases are an important evaluation target. In our main experiments, we selected the four preference biases because they are established targets in subliminal-learning and phantom-transfer work, enabling controlled evaluation against known implanted behaviors and comparison with prior work.
In addition to that, our risky-finance study (§6.2) also evaluates a conditional policy with potentially harmful real-world consequences. For example, when asked how to build good financial habits, the biased student recommends investing all savings in rapidly developing areas, whereas the matched control recommends budgeting and building an emergency fund (Appendix I). Guided benchmarking surfaces the risk-seeking policy, achieving GT-anchored PHR of 0.69 versus 0.16 for the matched control. This demonstrates the practical value of targeted auditing for a potentially harmful behavior that generic evaluation largely misses.
The pipeline does not require advance knowledge of the bias category. Conditional demographic biases, such as discrimination, are an important untested setting that we now identify explicitly in §7.
Q1. API-only access.
The current implementation requires weights and activations, but the broader principle could extend to API-only models by replacing internal decoding with adaptive behavioral exploration. An auditor could query diverse contexts, identify recurring response patterns, and generate increasingly targeted probes. If the base model is also accessible, response comparisons could help separate unexpected changes from ordinary task adaptation. Candidate behaviors should then be evaluated on fresh probes.
The central challenge is obtaining the initial signal: a narrow behavior may remain invisible until the search reaches a relevant context. Internal decoding supplies clues in our current method; an API-only approach would need to discover them through queries, potentially requiring a larger budget. We identify this extension in §7, while leaving its effectiveness and query efficiency for future evaluation.