- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper explores characterizing bias across the prompt space, rather than the norm of bias evaluation across a set of predefined evaluation datasets. They introduce COMPASS, which induces a latent geometry that preserves semantic consistency by placing prompts from similar contexts near one another, and keeps bias close by bringing prompts with similar measured bias together. Their method provides, for both observed and unobserved prompts, an estimated bias value and an associated uncertainty and selects which prompts should next be evaluated on the target LLM.
Critical strengths
The paper addresses an interesting research question: characterizing bias across the prompt space, rather than evaluating bias against a set of predefined evaluation datasets. However, it may be too ambitious a research question at the same time. General the quality of writing is good though some content is missing. I look forward to the rebuttal phase to resolve uncertainties.
Critical weaknesses
W1: The claim of bias estimation over an LLM's prompt space feels overstated. We never know the diverse prompt space of an LLM.
W2: The 3D PCA figures need better description. Many times, like in Fig 1 estimated bias and bias itself do not have the same pattern. How do you describe it (e.g. Fig 1. c). Some other 3D PCA results, like Fig 8 in the appendix, are not described well either. It is not well described what the reader should take from the figure
W3 Each prompt is evaluated 10 times, but the paper does not explain why. The results do not show CI evaluation. How do the values vary across 10 queries, both for synthetic or empirical evaluation
W4: The paper lacks a related work section. In the absence of a proper related work it is not possible to assess novelty and understand how the work position in existing literature
W5: Line 458 talk about bias estimation accuracy, but no accuracy is reported in all the paper
W6: The paper proposes no baseline comparisons to show the method meets the target.
W7: There is not a clear limitations section or list of assumptions under which the method is valid
W8: The paper criticizes other papers that ‘rely on predefined benchmarks or fixed sets of prompts to elicit and quantify biased model behavior.’ But eventually the paper itself relies on a set of generated prompts.
W9: The proposed dataset is and evaluation pipeline are fully machine-generated, with no human evaluation.
W10: The paper stated that prompts belonging to the same topic tend to cluster together, and their relative positions also reflect similar bias. It can be natural that clusters happen based on topic, but it does not represent bias. In one topic, there might be a variety of biases.
Questions for authors
Q1: Why do you generate a dataset rather than testing the approach on existing datasets? I understand the goal is to find bias beyond known predefined sets, but an existing set can be a good test set to see if the method is working properly.
Q2: How does each of these assumptions impact the model outcome:
Q2a:Prompts are artificially assigned a bias value. Why an artificial bias value can be valuable. They may be meaningless and do not reflect real model bias.
Q2b:Why do you use a Beta distribution? How it matches the real bias value. Particularly, there is a claim that the synthetic experiment has a ground truth label. But artificial assignment of the bias score without considering the semantics of the prompt is not counted as ground truth
Q2c: My understanding from these sentences is that bias is artificially assigned and may not reflect LLM bias on a given topic. ‘We define several configurations by varying both the number of topics and the allocation of bias categories across them’, or ‘assign a single bias distribution to each topic’, or ‘assigning low-, medium-, or high-bias distributions per topic’. Considering a fixed distribution of bias per topic or assigning them does not make sense, as it does not match the reality of the prompt itself. Even within a fixed topic, you may have different levels of bias per prompt. It is not something to assign. It should vary with the reality of the data.
4 comments
W6. In the revised version of the paper, we have extended the experiments provided in Section 4.2, providing a comparative analysis with a frozen pre-trained instantiation of BERT. In particular, we replaced our embedder with the pre-trained version of BERT, and compared the clustering tendency in terms of Hopkins statistic () and the optimal number of clusters, retrieved via Elbow method (). In the following table (Table 1 in the revised paper), we report the corresponding results. The value is computed over 30 independent runs, with the mean and standard deviation reported.
| Embedder | Balanced: | Balanced: | Majority: | Majority: | Heterogeneous: | Heterogeneous: | LLaMA3.1-8b: | LLaMA3.1-8b: | Qwen3-8b: | Qwen3-8b: |
|---|---|---|---|---|---|---|---|---|---|---|
| BERT | 0.753 ( 0.004) | 10 | 0.754 ( 0.004) | 10 | 0.755 ( 0.004) | 10 | 0.755 ( 0.002) | 10 | 0.750 ( 0.002) | 8 |
| 0.873 ( 0.005) | 3 | 0.938 ( 0.004) | 3 | 0.954 ( 0.002) | 7 | 0.843 ( 0.003) | 19 | 0.885 ( 0.003) | 19 |
The results show that our method successfully induces the desired geometry, across both synthetic and LLM-estimated bias configurations. Moreover, we provide quantitative results in terms of three different metrics: (i) the proportion of neighbor samples whose topic differs from that of the reference sample (% topic error); (ii) the mean absolute error between the bias value of the reference sample and those of its neighbors (bias error); and (iii) the proportion of neighbors whose absolute bias difference exceeds the 0.5 threshold (% bias error > 0.5). Figure 3 (in the revised version of the paper) depicts the corresponding error distributions, again comparing and BERT as representation learning component. consistently induces a more homogeneous and homophilic representation of the latent space, overcoming BERT across all metrics and configurations. Finally, we also provide a qualitative analysis displaying the 3D projections of the latent space (Figure 14 (in the revised manuscript): the results show that BERT clearly fails to meaningfully cluster the latent space.
W7. The Conclusions of the original submission already discussed the main assumption of the framework and its pool-based nature. In the revised version, we have collected the limitations in a dedicated “Limitations and future work” paragraph and added the one raised by the reviewer. In summary:
- Local regularity of bias in the learned space. COMPASS assumes that prompts close in the latent space induce similar bias. We consider this reasonable because bias is expected to reflect recurring associations rather than independent variation across prompts, and our empirical results are consistent with it.
- Choice of the bias score. COMPASS is not tied to the entropy-based score: any response-based score can serve as the exploration target.
- Validity of the entropy-based score (new). The score is meaningful only for prompts that give no reason to prefer one sensitive class. Our prompts were generated with this constraint, and come from a single generator LLM.
- Pool-based exploration (now under the new paragraph). The audit can only find bias among the prompts contained in the pool. An encoder-decoder extension, whose decoder maps promising latent points back to new prompts, would turn selection into generative exploration and broaden the coverage beyond the original pool.
W8. We agree that our experiments also rely on a finite set of prompts. The difference from prior work lies in how this set is used. Benchmark-based evaluations probe the model on every prompt of a dataset, and their conclusions apply to that dataset. COMPASS instead learns a latent representation whose geometry reflects both semantic consistency and bias closeness, and uses it to estimate bias, with an associated uncertainty, on prompts that were never evaluated. As shown in Figure 4, the estimation error on held-out prompts remains low when only 20% of the pool is evaluated, and improvements become negligible beyond 40%. Exhaustive probing of the pool is therefore not required. The pool still bounds what the audit can find, as we now state in the “Limitations and future work” paragraph of the Conclusions. To overcome this limit, an encoder-decoder architecture could generate new prompts from promising latent regions and extend the exploration beyond the original pool.
W9. The prompts used in our experiments follow two simple, fixed formats: a sentence with a single blank (fill-in-the-blank) or a short question paired with a predefined set of answers (multiple-choice). The system prompts used for generation explicitly require that the sensitive attribute is not disclosed or implied by the context (Table 4), and examples are reported in Table 3. To ensure structural validity of the final prompts, we performed the following pipeline: first, we run a code-base check to ensure valid responses (e.g., the presence of a single "_" in the “fill-in-the-blank” task prompts); secondly, all the authors performed a human validation on a random sample of 1000 prompts. On the evaluation side, no model is used as a judge: the interpretation function deterministically maps a single-word completion or a selected option to a class of the sensitive attribute, using the response sets listed in Table 6. Finally, we remark that the full dataset is released in our anonymized repository, so every prompt and its empirical bias score can be inspected directly.
W10. This is exactly what we show in our experiments: the induced latent representations produced with COMPASS cluster both in terms of topic and bias, even when multiple biases occur within the same topic. We prove this both on synthetic experiments (please, refer to Figure 1.c in the main paper, and Figure 6 in the Appendix), and on empirical estimate bias (e.g., Figure 3a and 3.b in the main paper).
Q1. Existing bias benchmarks are valuable for external validation. However, most existing benchmarks score predefined answers or model probabilities, whereas our entropy-based evaluation requires multiple stochastic responses to each prompt in order to estimate a distribution over sensitive-property categories. In particular, our entropy-based experimental instantiation requires a response interpretation function () that maps every potentially free-form model generation to a well-defined sensitive-property category.
There are also two practical considerations. First, many established bias benchmarks are relatively small and were designed for exhaustive evaluation, making them less suitable for studying query-efficient active exploration and learning curves across different query budgets. Second, evaluation over a curated benchmark would primarily measure the ability to recover bias cases already represented by its predefined prompts. Our objective instead requires a large and potentially redundant candidate pool in which the framework must identify informative or high-bias regions without exhaustively querying the target model.
We therefore generated controlled fill-in-the-blank and multiple-choice prompts with an explicit response space. This provides a deterministic and reproducible , supports repeated sampling from the target LLM, and supplies a sufficiently large and topic-structured candidate pool for evaluating query-efficient exploration. The synthetic configurations additionally allow controlled recovery experiments with a known simulation target.
We agree, however, that this controlled design does not replace validation on established benchmarks. COMPASS itself is agnostic to the particular bias functional: an existing benchmark can be used whenever it provides, or permits the construction of, a prompt-level response-based score.
Q2a. We agree that the synthetic scores do not reflect real model bias, and they are not meant to. Real bias is the object of the empirical setting, where scores are measured from the responses of the target LLM. The synthetic setting serves a different purpose: it provides a known target against which we can check whether COMPASS recovers a prescribed bias structure and extends it to held-out prompts, which cannot be verified when the true bias is unknown. We have replaced “ground truth” with “known synthetic target” throughout the paper.
Q2b. Since the bias score lies in [0, 1], we sample it from a Beta distribution, the standard flexible family on this interval. The choice is not meant to match the distribution of real bias values. The parameters (, ) let us generate regions of low, intermediate or high bias (means of about 0.05, 0.5 and 0.95; Table 6), with a controlled amount of variability within each region, so that the method can be tested across qualitatively different regimes.
Q2c. We agree that bias may vary across prompts within the same topic. Assigning one distribution per topic is the simplest test, and the heterogeneous configurations (Figure 1c, Figure 6) already relax it by letting several bias regimes coexist within each topic. Finally, in the empirical setting bias is not assigned at all: it is measured for each prompt, and the new quantitative results on real LLMs evaluate COMPASS in exactly this case.
W1. We agree that the prompt space of an LLM cannot be enumerated, and this is the core motivation behind our approach. Indeed , our formulation (Section 2) defines bias over the prompt space X, while the experiments (Section 4.1) instantiate a finite pool D⊂X, since exploring X directly is infeasible.
W2. We improved the description related to the 3D PCA figures. Specifically, we clarified that the figures display the 3D projections of the learned latent space, in which each dot represents the latent representation of a given textual content. Each point is colored according to its topic (first column) and its (empirical) bias value (second column). The figures further display the convex hull derived from the GΨ (third and fourth columns), colored in terms of estimated bias (third column), and corresponding uncertainty (fourth column).
W3. Our bias definition relies on estimating the conditional response distribution q(x) on a target LLM M. To this end, we need to repeatedly query M to obtain the corresponding estimation. In our default experimental setting, we set R (the parameter which controls the number of invocations) equal to 10. However, in the revised version of the submission, we performed a robustness analysis varying the R parameter, which controls the LLM responses, Specifically, we considered a subset of randomly sampled 19,000 prompts (1,000 per topic) and recomputed the bias scores by varying R∈[25,50,100], by probing LlaMa3.1-8b (gender, fill-in-the-blank) and Qwen3-8b (race, multiple-choice). We computed the discrepancy between the bias estimated by varying R with our default setting (R~=10). The following table (Table 12 in the revised manuscript) depicts the results in terms of average, standard deviation, median, min, and max.
As shown in the table, the empirical bias scores remain stable as the number of responses increases, indicating that increasing the number of responses does not substantially affect the overall distribution of the empirical bias scores.
Notice that in the synthetic configuration no empirical estimation is required, since the bias is fixed a-priori to validate the effectiveness of COMPASS through multiple in-vitro experiments.
W4. We agree with the Reviewer that avoiding discussing the related works prevents positioning the paper in the current literature. However, we respectfully notice that the lack of a dedication section merely reflects a design choice: we discuss the existing literature and the related approaches in the Introduction section, following the ICLR tradition of merging the two sections (see, e.g., [I, II]).
I. Matrix Manifold Neural Networks++. ICLR 2024
II. Manipulating dropout reveals an optimal balance of efficiency and robustness in biological and machine visual systems. ICLR 2024
W5. Thank you for the comment. We have replaced the word “accuracy” with “performance” as kindly suggested.