acceptodds
Under review as a conference paper at ICLR 2027

COMPASS: Navigating Prompt Latent Space for Bias Estimation in Large Language Models

Abstract

Addressing bias in Large Language Models (LLMs) is critical for their reliable and responsible deployment. Current evaluation frameworks primarily rely on predefined benchmarks or fixed sets of prompts to elicit and quantify biased model behavior. However, these approaches cover only a limited set of interactions, leaving largely unexplored how bias varies across the broader prompt space. Characterizing bias beyond such predefined sets is challenging, as the prompt space is effectively unbounded and exhaustive probing is computationally infeasible. To address this challenge, we introduce COMPASS, a framework that formulates bias assessment as an active exploration problem over a learned continuous latent representation of the prompt space, whose geometry is explicitly structured to preserve semantic consistency and bias closeness. Over this learned space, bias and uncertainty are estimated through a probabilistic surrogate model and used to guide the exploration toward regions of either high predicted bias or high uncertainty. The resulting iterative process enables bias estimation beyond the explicitly evaluated prompts, replacing exhaustive exploration with selective LLM probing. Experiments on synthetic and empirically estimated biases show that the learned latent geometry captures both semantic and bias structure across different configurations.

Primary area:
generative models
Keywords:
Bias Estimation, Large Language Models, Optimization Framework

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nPdzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper explores characterizing bias across the prompt space, rather than the norm of bias evaluation across a set of predefined evaluation datasets. They introduce COMPASS, which induces a latent geometry that preserves semantic consistency by placing prompts from similar contexts near one another, and keeps bias close by bringing prompts with similar measured bias together. Their method provides, for both observed and unobserved prompts, an estimated bias value and an associated uncertainty and selects which prompts should next be evaluated on the target LLM.

Critical strengths

The paper addresses an interesting research question: characterizing bias across the prompt space, rather than evaluating bias against a set of predefined evaluation datasets. However, it may be too ambitious a research question at the same time. General the quality of writing is good though some content is missing. I look forward to the rebuttal phase to resolve uncertainties.

Critical weaknesses

W1: The claim of bias estimation over an LLM's prompt space feels overstated. We never know the diverse prompt space of an LLM.

W2: The 3D PCA figures need better description. Many times, like in Fig 1 estimated bias and bias itself do not have the same pattern. How do you describe it (e.g. Fig 1. c). Some other 3D PCA results, like Fig 8 in the appendix, are not described well either. It is not well described what the reader should take from the figure

W3 Each prompt is evaluated 10 times, but the paper does not explain why. The results do not show CI evaluation. How do the values vary across 10 queries, both for synthetic or empirical evaluation

W4: The paper lacks a related work section. In the absence of a proper related work it is not possible to assess novelty and understand how the work position in existing literature

W5: Line 458 talk about bias estimation accuracy, but no accuracy is reported in all the paper

W6: The paper proposes no baseline comparisons to show the method meets the target.

W7: There is not a clear limitations section or list of assumptions under which the method is valid

W8: The paper criticizes other papers that ‘rely on predefined benchmarks or fixed sets of prompts to elicit and quantify biased model behavior.’ But eventually the paper itself relies on a set of generated prompts.

W9: The proposed dataset is and evaluation pipeline are fully machine-generated, with no human evaluation.

W10: The paper stated that prompts belonging to the same topic tend to cluster together, and their relative positions also reflect similar bias. It can be natural that clusters happen based on topic, but it does not represent bias. In one topic, there might be a variety of biases.

Questions for authors

Q1: Why do you generate a dataset rather than testing the approach on existing datasets? I understand the goal is to find bias beyond known predefined sets, but an existing set can be a good test set to see if the method is working properly.

Q2: How does each of these assumptions impact the model outcome:

Q2a:Prompts are artificially assigned a bias value. Why an artificial bias value can be valuable. They may be meaningless and do not reflect real model bias.

Q2b:Why do you use a Beta distribution? How it matches the real bias value. Particularly, there is a claim that the synthetic experiment has a ground truth label. But artificial assignment of the bias score without considering the semantics of the prompt is not counted as ground truth

Q2c: My understanding from these sentences is that bias is artificially assigned and may not reflect LLM bias on a given topic. ‘We define several configurations by varying both the number of topics and the allocation of bias categories across them’, or ‘assign a single bias distribution to each topic’, or ‘assigning low-, medium-, or high-bias distributions per topic’. Considering a fixed distribution of bias per topic or assigning them does not make sense, as it does not match the reality of the prompt itself. Even within a fixed topic, you may have different levels of bias per prompt. It is not something to assign. It should vary with the reality of the data.

4 comments

W1. We agree that the prompt space of an LLM cannot be enumerated, and this is the core motivation behind our approach. Indeed , our formulation (Section 2) defines bias over the prompt space X\mathcal{X}, while the experiments (Section 4.1) instantiate a finite pool D⊂X\mathcal{D} \subset \mathcal{X}, since exploring X\mathcal{X} directly is infeasible.

W2. We improved the description related to the 3D PCA figures. Specifically, we clarified that the figures display the 3D projections of the learned latent space, in which each dot represents the latent representation of a given textual content. Each point is colored according to its topic (first column) and its (empirical) bias value (second column). The figures further display the convex hull derived from the GΨ\mathcal{G}_\Psi (third and fourth columns), colored in terms of estimated bias (third column), and corresponding uncertainty (fourth column).

W3. Our bias definition relies on estimating the conditional response distribution q(xx) on a target LLM M\mathcal{M}. To this end, we need to repeatedly query M\mathcal{M} to obtain the corresponding estimation. In our default experimental setting, we set RR (the parameter which controls the number of invocations) equal to 10. However, in the revised version of the submission, we performed a robustness analysis varying the RR parameter, which controls the LLM responses, Specifically, we considered a subset of randomly sampled 19,000 prompts (1,000 per topic) and recomputed the bias scores by varying R∈[25,50,100]R \in [25, 50, 100], by probing LlaMa3.1-8b (gender, fill-in-the-blank) and Qwen3-8b (race, multiple-choice). We computed the discrepancy between the bias estimated by varying RR with our default setting (R~=10\tilde{R} = 10). The following table (Table 12 in the revised manuscript) depicts the results in terms of average, standard deviation, median, min, and max.

M\mathcal{M}RRMean (±\pm Std)MedianMinMax
LLaMa3.1-8B250.090 (±\pm 0.119)0.040.00.88
LLaMa3.1-8B500.102 (±\pm 0.135)0.040.00.96
LLaMa3.1-8B1000.109 (±\pm 0.141)0.040.00.91
Qwen3-8B250.041 (±\pm 0.069)0.00.00.48
Qwen3-8B500.048 (±\pm 0.079)0.00.00.56
Qwen3-8B1000.051 (±\pm 0.082)0.00.00.573

As shown in the table, the empirical bias scores remain stable as the number of responses increases, indicating that increasing the number of responses does not substantially affect the overall distribution of the empirical bias scores.

Notice that in the synthetic configuration no empirical estimation is required, since the bias is fixed a-priori to validate the effectiveness of COMPASS through multiple in-vitro experiments.

W4. We agree with the Reviewer that avoiding discussing the related works prevents positioning the paper in the current literature. However, we respectfully notice that the lack of a dedication section merely reflects a design choice: we discuss the existing literature and the related approaches in the Introduction section, following the ICLR tradition of merging the two sections (see, e.g., [I, II]).

I. Matrix Manifold Neural Networks++. ICLR 2024

II. Manipulating dropout reveals an optimal balance of efficiency and robustness in biological and machine visual systems. ICLR 2024

W5. Thank you for the comment. We have replaced the word “accuracy” with “performance” as kindly suggested.

W6. In the revised version of the paper, we have extended the experiments provided in Section 4.2, providing a comparative analysis with a frozen pre-trained instantiation of BERT. In particular, we replaced our embedder PΘP_\Theta with the pre-trained version of BERT, and compared the clustering tendency in terms of Hopkins statistic (HH) and the optimal number of clusters, retrieved via Elbow method (K∗K^*). In the following table (Table 1 in the revised paper), we report the corresponding results. The HH value is computed over 30 independent runs, with the mean and standard deviation reported.

EmbedderBalanced: HHBalanced: K∗K^*Majority: HHMajority: K∗K^*Heterogeneous: HHHeterogeneous: K∗K^*LLaMA3.1-8b: HHLLaMA3.1-8b: K∗K^*Qwen3-8b: HHQwen3-8b: K∗K^*
BERT0.753 (±\pm 0.004)100.754 (±\pm 0.004)100.755 (±\pm 0.004)100.755 (±\pm 0.002)100.750 (±\pm 0.002)8
PΘP_\Theta0.873 (±\pm 0.005)30.938 (±\pm 0.004)30.954 (±\pm 0.002)70.843 (±\pm 0.003)190.885 (±\pm 0.003)19

The results show that our method successfully induces the desired geometry, across both synthetic and LLM-estimated bias configurations. Moreover, we provide quantitative results in terms of three different metrics: (i) the proportion of neighbor samples whose topic differs from that of the reference sample (% topic error); (ii) the mean absolute error between the bias value of the reference sample and those of its neighbors (bias error); and (iii) the proportion of neighbors whose absolute bias difference exceeds the 0.5 threshold (% bias error > 0.5). Figure 3 (in the revised version of the paper) depicts the corresponding error distributions, again comparing PΘP_\Theta and BERT as representation learning component. PΘP_\Theta consistently induces a more homogeneous and homophilic representation of the latent space, overcoming BERT across all metrics and configurations. Finally, we also provide a qualitative analysis displaying the 3D projections of the latent space (Figure 14 (in the revised manuscript): the results show that BERT clearly fails to meaningfully cluster the latent space.

W7. The Conclusions of the original submission already discussed the main assumption of the framework and its pool-based nature. In the revised version, we have collected the limitations in a dedicated “Limitations and future work” paragraph and added the one raised by the reviewer. In summary:

  • Local regularity of bias in the learned space. COMPASS assumes that prompts close in the latent space induce similar bias. We consider this reasonable because bias is expected to reflect recurring associations rather than independent variation across prompts, and our empirical results are consistent with it.
  • Choice of the bias score. COMPASS is not tied to the entropy-based score: any response-based score can serve as the exploration target.
  • Validity of the entropy-based score (new). The score is meaningful only for prompts that give no reason to prefer one sensitive class. Our prompts were generated with this constraint, and come from a single generator LLM.
  • Pool-based exploration (now under the new paragraph). The audit can only find bias among the prompts contained in the pool. An encoder-decoder extension, whose decoder maps promising latent points back to new prompts, would turn selection into generative exploration and broaden the coverage beyond the original pool.

W8. We agree that our experiments also rely on a finite set of prompts. The difference from prior work lies in how this set is used. Benchmark-based evaluations probe the model on every prompt of a dataset, and their conclusions apply to that dataset. COMPASS instead learns a latent representation whose geometry reflects both semantic consistency and bias closeness, and uses it to estimate bias, with an associated uncertainty, on prompts that were never evaluated. As shown in Figure 4, the estimation error on held-out prompts remains low when only 20% of the pool is evaluated, and improvements become negligible beyond 40%. Exhaustive probing of the pool is therefore not required. The pool still bounds what the audit can find, as we now state in the “Limitations and future work” paragraph of the Conclusions. To overcome this limit, an encoder-decoder architecture could generate new prompts from promising latent regions and extend the exploration beyond the original pool.

W9. The prompts used in our experiments follow two simple, fixed formats: a sentence with a single blank (fill-in-the-blank) or a short question paired with a predefined set of answers (multiple-choice). The system prompts used for generation explicitly require that the sensitive attribute is not disclosed or implied by the context (Table 4), and examples are reported in Table 3. To ensure structural validity of the final prompts, we performed the following pipeline: first, we run a code-base check to ensure valid responses (e.g., the presence of a single "_" in the “fill-in-the-blank” task prompts); secondly, all the authors performed a human validation on a random sample of 1000 prompts. On the evaluation side, no model is used as a judge: the interpretation function fSf_\mathcal{S} deterministically maps a single-word completion or a selected option to a class of the sensitive attribute, using the response sets listed in Table 6. Finally, we remark that the full dataset is released in our anonymized repository, so every prompt and its empirical bias score can be inspected directly.

W10. This is exactly what we show in our experiments: the induced latent representations produced with COMPASS cluster both in terms of topic and bias, even when multiple biases occur within the same topic. We prove this both on synthetic experiments (please, refer to Figure 1.c in the main paper, and Figure 6 in the Appendix), and on empirical estimate bias (e.g., Figure 3a and 3.b in the main paper).

Q1. Existing bias benchmarks are valuable for external validation. However, most existing benchmarks score predefined answers or model probabilities, whereas our entropy-based evaluation requires multiple stochastic responses to each prompt in order to estimate a distribution over sensitive-property categories. In particular, our entropy-based experimental instantiation requires a response interpretation function (fS:Y→Sf_{\mathcal S}:\mathcal Y\rightarrow\mathcal S) that maps every potentially free-form model generation to a well-defined sensitive-property category.

There are also two practical considerations. First, many established bias benchmarks are relatively small and were designed for exhaustive evaluation, making them less suitable for studying query-efficient active exploration and learning curves across different query budgets. Second, evaluation over a curated benchmark would primarily measure the ability to recover bias cases already represented by its predefined prompts. Our objective instead requires a large and potentially redundant candidate pool in which the framework must identify informative or high-bias regions without exhaustively querying the target model.

We therefore generated controlled fill-in-the-blank and multiple-choice prompts with an explicit response space. This provides a deterministic and reproducible fSf_{\mathcal{S}}, supports repeated sampling from the target LLM, and supplies a sufficiently large and topic-structured candidate pool for evaluating query-efficient exploration. The synthetic configurations additionally allow controlled recovery experiments with a known simulation target.

We agree, however, that this controlled design does not replace validation on established benchmarks. COMPASS itself is agnostic to the particular bias functional: an existing benchmark can be used whenever it provides, or permits the construction of, a prompt-level response-based score.

Q2a. We agree that the synthetic scores do not reflect real model bias, and they are not meant to. Real bias is the object of the empirical setting, where scores are measured from the responses of the target LLM. The synthetic setting serves a different purpose: it provides a known target against which we can check whether COMPASS recovers a prescribed bias structure and extends it to held-out prompts, which cannot be verified when the true bias is unknown. We have replaced “ground truth” with “known synthetic target” throughout the paper.

Q2b. Since the bias score lies in [0, 1], we sample it from a Beta distribution, the standard flexible family on this interval. The choice is not meant to match the distribution of real bias values. The parameters (α\alpha, β\beta) let us generate regions of low, intermediate or high bias (means of about 0.05, 0.5 and 0.95; Table 6), with a controlled amount of variability within each region, so that the method can be tested across qualitatively different regimes.

Q2c. We agree that bias may vary across prompts within the same topic. Assigning one distribution per topic is the simplest test, and the heterogeneous configurations (Figure 1c, Figure 6) already relax it by letting several bias regimes coexist within each topic. Finally, in the empirical setting bias is not assigned at all: it is measured for each prompt, and the new quantitative results on real LLMs evaluate COMPASS in exactly this case.

Reviewer_VgkJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces COMPASS, a framework for estimating LLM bias across a broad prompt space by learning a continuous latent representation that preserves both semantic similarity and bias-related relationships. It combines Gaussian Process regression with uncertainty-guided active learning to predict bias for unevaluated prompts and selectively query informative prompts, with experiments on synthetic and empirically measured bias distributions.

Critical strengths
  • The paper addresses an important limitation of existing LLM bias evaluations, which typically rely on fixed benchmarks and may overlook biased behaviors in unexplored prompts. Formulating bias estimation as an active exploration problem is a meaningful direction toward more comprehensive and query-efficient evaluation.

  • COMPASS provides a clear and coherent framework that combines bias-aware representation learning, Gaussian Process regression, and uncertainty-guided active learning. The experiments across different synthetic configurations and LLMs provide encouraging evidence that the learned latent space can capture both semantic and bias-related structures.

Critical weaknesses
  • The experimental evaluation is insufficient to support the main claims of the paper. Most quantitative results are based on synthetic bias scores sampled from predefined topic-dependent distributions, while experiments on real LLMs mainly provide latent-space visualizations rather than quantitative bias estimation results. Moreover, the paper lacks comparisons with strong baselines, such as standard sentence embeddings combined with Gaussian Processes and active learning, under the same query budget. This makes it difficult to assess whether COMPASS genuinely improves bias estimation accuracy and query efficiency.

  • The definition and measurement of bias raise concerns about validity and reliability. The entropy-based metric measures how concentrated model responses are across demographic categories, but an uneven distribution does not necessarily indicate unfairness or harmful bias. Furthermore, empirical bias scores are estimated from only 10 generations per prompt, which can produce noisy and highly discrete estimates. Since these scores are also used to construct bias-aware triplets, unreliable measurements may affect both representation learning and the evaluation results.

Questions for authors
  • Can the authors provide quantitative bias estimation results on held-out prompts using empirically measured bias scores from real LLMs? In particular, how does COMPASS compare with simpler baselines, such as frozen sentence embeddings with GP regression and UCB acquisition, as well as random and uncertainty-based sampling, under the same total LLM query budget? These comparisons are important for establishing the benefits of the proposed bias-aware representation and iterative exploration.

  • How do the authors justify using response entropy as a measure of LLM bias, given that demographic response concentration does not necessarily imply unfairness? Additionally, how stable are the empirical bias scores when only 10 responses are sampled per prompt? Could the authors evaluate the sensitivity of these scores and the resulting bias estimation performance to the number of sampled responses?

4 comments

W1. In the revised version of the submission, we have extended the quantitative evaluation in several directions. First, we run additional experiments using real LLMs on: Llama 3.1-8B, (considering “gender” as the sensitive attribute and the “fill-in-the-blank” task) and Qwen3-8b (considering “race” and “multiple-choice”). Second, we integrate the latent space visualizations with quantitative estimation of the induced geometry. Specifically, we compare the clustering tendency induced by our embedder PΘP_{\Theta} and a pre-trained version of BERT, in terms of Hopkins statistic (HH) and the optimal number of clusters, retrieved via Elbow method (K∗K^*). In the following table (Table 1 in the revised paper), we report the corresponding results. The HH value is computed over 30 independent runs, with the mean and standard deviation reported.

EmbedderBalanced: HHBalanced: K∗K^*Majority: HHMajority: K∗K^*Heterogeneous: HHHeterogeneous: K∗K^*LLaMA3.1-8b: HHLLaMA3.1-8b: K∗K^*Qwen3-8b: HHQwen3-8b: K∗K^*
BERT0.753 (±\pm 0.004)100.754 (±\pm 0.004)100.755 (±\pm 0.004)100.755 (±\pm 0.002)100.750 (±\pm 0.002)8
PΘ{P}_\Theta0.873 (±\pm 0.005)30.938 (±\pm 0.004)30.954 (±\pm 0.002)70.843 (±\pm 0.003)190.885 (±\pm 0.003)19

The results show that our method successfully induces the desired geometry, across both synthetic and LLM-estimated bias configurations. Third, we provide quantitative results in terms of three different metrics: (i) the proportion of neighbor samples whose topic differs from that of the reference sample (% topic error); (ii) the mean absolute error between the bias value of the reference sample and those of its neighbors (bias error); and (iii) the proportion of neighbors whose absolute bias difference exceeds the 0.5 threshold (% bias error > 0.5).

Figure 3 (in the revised version of the paper) depicts the corresponding error distributions, again comparing PΘP_\Theta and BERT as representation learning component. PΘ{P}_\Theta consistently induces a more homogeneous and homophilic representation of the latent space, overcoming BERT across all metrics and configurations. Finally, we extended the results provided in Section 4.2 (Figure 4) to further include the aforesaid real LLMs.

Q1. Figure 4, which provides the mean absolute error on held-out (unevaluated) prompts, now includes LLaMa3.1-8B (gender, fill-in-the-blank) and Qwen3-8B (race, multiple-choice). The error remains low with only 20% of the prompts evaluated and improves negligibly beyond 40%, while disabling the exploration loop (K = 1) increases it in every configuration.

In another test (run to investigate targeted intervention strategies), using LLaMa3.1-8b as target model, “gender” as sensitive attribute and “fill-in-the-blank” as task (see our response to W3 below), the held-out prompts that COMPASS flags as “highly-biased” displays an average empirical bias of 0.893 (median 1.0), confirming that the flagged region does contain highly biased prompts.

In addition, we perform a sensitivity analysis by varying the implementation of the acquisition function used in the Active Learning step (Equation 11). Specifically, we replace the UCB function defined in Eq. 11 with (i) random sampling and (ii) sampling based only on the uncertainty (σ(z)\sigma(z)). The three strategies differ in where they spend queries: random and uncertainty sampling distribute them to reduce the estimation error, whereas the mean term of UCB also directs them toward regions of high predicted bias. The following table (Table 10 in the revised manuscript) shows the results in terms of clustering tendency. The results show that these strategies represent viable alternatives that can be suitably embedded within COMPASS, in place of the current implementation.

Sampling StrategyBalanced: HHBalanced: K∗K^*Majority: HHMajority: K∗K^*Heterogeneous: HHHeterogeneous: K∗K^*LLaMA3.1-8b: HHLLaMA3.1-8b: K∗K^*Qwen3-8b: HHQwen3-8b: K∗K^*
Random0.868 (±\pm 0.006)30.942 (±\pm 0.004)30.957 (±\pm 0.003)50.850 (±\pm 0.004)200.882 (±\pm 0.003)20
Uncertainty0.867 (±\pm 0.004)30.962 (±\pm 0.002)30.931 (±\pm 0.003)50.853 (±\pm 0.004)180.875 (±\pm 0.003)20
Uncertainty + Bias0.873 (±\pm 0.005)30.938 (±\pm 0.004)30.954 (±\pm 0.002)70.843 (±\pm 0.003)190.885 (±\pm 0.003)19

W2.1/Q2.1. We agree that response concentration is not, in isolation, a universal measure of discrimination or social harm. Our intended interpretation is narrower: for prompts that are deliberately underspecified with respect to a sensitive attribute, the score operationalizes demographic representation skew relative to an explicit parity reference. In fact,

1−H(q)log⁡n=DKL(q∣u)log⁡n,1-\frac{H(q)}{\log n}=\frac{D_{\mathrm{KL}}(q|u)}{\log n},

where uu is the uniform distribution over the sensitive-property classes. Thus, the proposed score is exactly a normalized divergence from a demographic-parity reference, rather than an unrelated measure of response confidence. This operationalization is closely related to the demographic-representation metric adopted in established literature, such as HELM, which compares observed demographic mentions with a uniform reference distribution (Liang et al., 2023), and to information-theoretic formulations of approximate demographic parity based on mutual information and KL divergence (Williamson and Menon, 2019; Song et al., 2019). Information-theoretic dependence measures have also been applied specifically to fairness in language generation (Wang et al., 2023).

In Section 3.1 of the revised manuscript, we clarify that this metric captures prompt-level representational disparity under an explicit uniform-reference assumption. When uniform representation is not the appropriate target, the reference distribution can be replaced by a task- or population-specific distribution. More generally, COMPASS is metric-agnostic and can operate with alternative response-based fairness or harm functionals.

I. Liang et al. (2023), “Holistic Evaluation of Language Models”, Transactions on Machine Learning Research

II. Williamson & Menon (2019), “Fairness Risk Measures”, Proceedings of the 36th International Conference on Machine Learning

III. Song et al. (2019), “Learning Controllable Fair Representations”, Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics

IV. Wang, Cheng & Henao (2023), “Toward Fairness in Text Generation via Mutual Information Minimization”, Proceedings of The 26th International Conference on Artificial Intelligence and Statistics

W2.2/Q2.2. We conducted a robustness analysis on the RR parameter, which controls the responses per prompt, on a subset of randomly sampled 19,000 prompts (1,000 per topic). Specifically, we recomputed the empirical bias scores using R∈[25,50,100]R \in [25, 50, 100], by probing Llama 3.1-8B and Qwen3-8B. For Llama 3.1-8B, we considered “gender” as the sensitive attribute and the “fill-in-the-blank” task, whereas for Qwen3-8B, we considered “race” as the sensitive attribute and the “multiple-choice” task. The following table (Table 12 in the revised manuscript) reports the mean, standard deviation, median, minimum, and maximum value of the absolute discrepancy between the bias estimated by varying RR and the one obtained with our default experimental setting (R~=10\tilde{R} = 10), across all prompts. As shown in the table, the empirical bias scores remain stable as the number of responses increases. This indicates that increasing the number of responses does not substantially affect the overall distribution of the empirical bias scores.

M\mathcal{M}RRMean (±\pm Std)MedianMinMax
LLaMa3.1-8B250.090 (±\pm 0.119)0.040.00.88
LLaMa3.1-8B500.102 (±\pm 0.135)0.040.00.96
LLaMa3.1-8B1000.109 (±\pm 0.141)0.040.00.91
Qwen3-8B250.041 (±\pm 0.069)0.00.00.48
Qwen3-8B500.048 (±\pm 0.079)0.00.00.56
Qwen3-8B1000.051 (±\pm 0.082)0.00.00.573
Reviewer_v55hOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

COMPASS proposes an active-learning framework for estimating bias across a large prompt space with fewer target-LLM queries. It combines semantic and bias-aware prompt representations, Gaussian Process prediction, and an UCB acquisition strategy.

Critical strengths
  1. Addresses the important problem of query-efficient LLM bias auditing.

  2. Evaluates multiple LLMs, sensitive attributes, and prompt formats.

  3. Provides synthetic experiments to investigate the proposed method’s behavior.

Critical weaknesses
  1. Real-model experiments mainly provide visualizations, with insufficient quantitative evaluation of held-out bias prediction.

  2. The main quantitative comparison is against COMPASS without iterative exploration ((K=1)). Comparisons with random sampling, uncertainty sampling, and other active-learning methods are missing.

  3. The paper does not demonstrate how COMPASS supports bias evaluation on specific downstream tasks or improves bias mitigation.

  4. The entropy-based score measures response concentration, which does not necessarily correspond to social bias or discrimination.

Questions for authors
  1. How can COMPASS be applied to evaluate bias in specific downstream tasks, beyond estimating bias scores over a predefined prompt pool?

  2. Can COMPASS guide bias mitigation in practice? For example, can its high-bias prompts inform data curation, model fine-tuning, or targeted interventions? This would require further validation beyond the current framework.

  3. Why are there no comparisons with stronger baselines, such as random sampling, uncertainty sampling, and existing query-efficient bias-auditing methods, under the same query budget?

  4. Can the authors report quantitative results on real LLMs, including held-out prediction error, high-bias prompt discovery performance, and actual query savings?

  5. How well does the proposed entropy-based metric reflect meaningful social bias, rather than simply concentrated response distributions?

6 comments

W1/Q4. We have extended the quantitative evaluation in Section 4.2 to real LLMs. We measure the mean absolute error (MAE) between the empirical bias scores and the COMPASS estimates on a held-out set while varying the share of initially evaluated prompts from 20% to 80%. In addition to the synthetic configurations, Figure 4 now includes LLaMa3.1-8B (gender, fill-in-the-blank) and Qwen3-8B (race, multiple-choice). The error remains low with only 20% of the prompts evaluated and improves negligibly beyond 40%, while disabling the exploration loop (K = 1) increases it in every configuration. Regarding high-bias prompt discovery, we run an additional test to investigate targeted intervention strategies, using LLaMa3.1-8b as target model, “gender” as sensitive attribute and “fill-in-the-blank” as task (see our response to W3 below). In such experimental configuration, the held-out prompts that COMPASS flags as high-bias in our fine-tuning experiment have an average empirical bias of 0.893 (median 1.0) on the original model, confirming that the flagged region does contain highly biased prompts. Regarding query savings, the following table (Table 11 in the revised manuscript) reports the fraction of U\mathcal{U} that is never probed during exploration. On the real LLMs, COMPASS avoids probing about 85% of U\mathcal{U} in every setting, and between 54% and 83% on the synthetic configurations, while still improving the estimation error over the non-iterative variant (Figure 4).

%∥E∥\|\mathcal{E}\|BalancedMajorityHeterogeneousLLaMa3.1-8bQwen3-8b
200.8010.5430.8000.8680.867
400.8020.6150.6700.8690.851
600.8330.5390.6850.8690.850
800.8230.6310.5520.8490.854

W2/Q3. In the revised version, we have extended the comparison along three directions (Section 4.2).

Acquisition strategy. We performed a sensitivity analysis on the UCB function defined in Eq. 11 by replacing it with (i) random sampling and (ii) sampling based only on the uncertainty (σ(z)\sigma(z)). The three strategies differ in where they spend queries: random and uncertainty sampling distribute them to reduce the estimation error, whereas the mean term of UCB also directs them toward regions of high predicted bias. The following table (Table 10 in the revised manuscript) shows the results in terms of clustering tendency. The results show that these strategies represent viable alternatives that can be suitably embedded within COMPASS, in place of the current implementation. The clustering tendency is computed in terms of Hopkins statistic (HH) and optimal number of clusters, retrieved via Elbow method (K∗K^*). The HH value is computed over 30 independent runs, with the mean and standard deviation reported. Moreover, the contribution of the iterative exploration loop itself is isolated by the comparison with the non-iterative variant (K = 1), which increases the estimation error in every configuration (Figure 4).

Sampling StrategyBalanced: HHBalanced: K∗K^*Majority: HHMajority: K∗K^*Heterogeneous: HHHeterogeneous: K∗K^*LLaMA3.1-8b: HHLLaMA3.1-8b: K∗K^*Qwen3-8b: HHQwen3-8b: K∗K^*
Random0.868 (±\pm 0.006)30.942 (±\pm 0.004)30.957 (±\pm 0.003)50.850 (±\pm 0.004)200.882 (±\pm 0.003)20
Uncertainty0.867 (±\pm 0.004)30.962 (±\pm 0.002)30.931 (±\pm 0.003)50.853 (±\pm 0.004)180.875 (±\pm 0.003)20
Uncertainty + Bias0.873 (±\pm 0.005)30.938 (±\pm 0.004)30.954 (±\pm 0.002)70.843 (±\pm 0.003)190.885 (±\pm 0.003)19

Representation learning. To assess the contribution of the trainable encoder, we replaced it with a frozen pre-trained BERT, keeping the GP surrogate and the acquisition rule unchanged. With BERT, as shown in the following table (Table 1 in the revised manuscript), the latent space shows a lower clustering tendency and the number of clusters does not reflect the expected structure. Neighboring prompts in the BERT space also differ more in both topic and bias (Figure 3). Without bias closeness, the GP cannot transfer bias information between neighboring prompts, and the exploration cannot be directed toward biased regions.

EmbedderBalanced: HHBalanced: K∗K^*Majority: HHMajority: K∗K^*Heterogeneous: HHHeterogeneous: K∗K^*LLaMA3.1-8b: HHLLaMA3.1-8b: K∗K^*Qwen3-8b: HHQwen3-8b: K∗K^*
BERT0.753 (±\pm 0.004)100.754 (±\pm 0.004)100.755 (±\pm 0.004)100.755 (±\pm 0.002)100.750 (±\pm 0.002)8
PΘP_\Theta0.873 (±\pm 0.005)30.938 (±\pm 0.004)30.954 (±\pm 0.002)70.843 (±\pm 0.003)190.885 (±\pm 0.003)19

Existing query-efficient auditing methods. The closest work is Hartmann et al. (2026) (already referenced in the paper), which reduces the number of queries needed to estimate an aggregate fairness metric on a given dataset. COMPASS addresses a different target: the distribution of bias over individual prompts, including prompts that are never queried. Since the two methods estimate different quantities, a direct comparison under the same budget is not well defined; we discuss the relation in the Introduction.

W3/Q1. (downstream tasks). COMPASS takes two inputs: a pool of prompts and an interpretation function fSf_{\mathcal{S}} that maps a response to a class of the sensitive attribute. Neither is tied to the “fill-in-the-blank” and “multiple-choice” formats used in the paper. Auditing a downstream task amounts to sampling the pool from the task’s input distribution and defining fSf_{\mathcal{S}} on its outputs: for example, candidate recommendation requests where fSf_{\mathcal{S}} extracts the gender of the recommended candidate, or reference-letter generation where it extracts the pronouns used. Because the surrogate predicts scores for prompts that were never probed, the pool acts as a sample of the task’s inputs rather than as a closed benchmark.

W3/Q2 (bias mitigation). We added a proof of concept in which the high-bias region identified by COMPASS is used to fine-tune the target model (Section 4.2, “Interventions”; details in the Appendix). We select the prompts on which LLaMa3.1-8B has an estimated bias above 0.8 (gender, fill-in-the-blank), fine-tune the model on 90% of them so that it re-balances the probability of the admissible responses while keeping the rest of its output distribution anchored to the original model, and measure the empirical bias on the remaining 10%. The average bias drops from 0.893 to 0.245 (−73%), and the median from 1.0 to 0.2 (Table 13). The training prompts are selected from the bias map alone, without probing the whole pool.

M\mathcal{M}Average (±\pm Std): b^\hat{b}Average (±\pm Std): bTVb_{\rm{TV}}Median: b^\hat{b}Median: bTVb_{\rm{TV}}
Vanilla0.893 (±\pm 0.234)0.697 (±\pm 0.289)1.00.809
Fine-tuned0.245 (±\pm 0.20)0.028 (±\pm 0.03)0.20.0
Bias reduction (%)72.696.080.0100.0

Data curation and targeted interventions. Since the predicted bias μ(z)\mu(z) can be computed for any prompt without querying the model, the same map supports other strategies, which we now describe in the Appendix. For data curation, it scores candidate training examples, so that counterfactual augmentation or re-weighting can be restricted to high-bias regions. At inference time, it acts as an inexpensive risk score: high-risk prompts can be routed to a debiasing strategy, such as a dedicated system prompt, while a high σ(z)\sigma(z) flags prompts that need a direct audit. We agree with the reviewer that a systematic validation of these strategies requires further work; the fine-tuning experiment shows that the bias map is directly usable for it.

Comments on the paper

2 comments

We thank the Reviewers for their valuable feedback, and for recognizing the importance and timeliness of the problem we address and the value of the framework we propose. We tried to address their concerns to the best of our possibilities, expanding the experimental evaluation accordingly. Specifically, we: (i) integrated quantitative results showing the clustering tendency and homophily of the induced latent representation; (ii) we compared our embedding component PΘ\mathcal{P}_\Theta with a pre-trained frozen embedder to assess the contribution of the representation learning component; (iii) we performed multiple sensitivity analyses on both synthetic and LLM-based configurations, showing the robustness of our approach; and finally (iv) we devised a fine-tuning procedure as a proof-of-concept for a targeted intervention strategy, enabled by the latent geometry induced by COMPASS. All the modifications are marked on the revised paper in blue. Point-by-point responses are further provided below.

Dear Reviewers, we reiterate our gratitude for your constructive feedback.

We put substantial effort to improve the paper according to your comments and suggestions. As the rebuttal deadline is approaching, we would greatly appreciate if you could let us know whether our additional experiments and responses have addressed your concerns. You can also find the revised version of our work incorporating all changes, marked in blue.

Thank you again.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.