- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a method for reconstructing latent bias of LLMs from their outputs. Different from traditional, singular bias scores, this approach characterizes how models systematically allocate probability mass across answer categories. Its main contribution is a Bayesian inverse-problem framework that decomposes observed candidate-answer log-probabilities into a shared latent preference bias, prompt-specific evidence shifts, and noises. The method uses MAP estimation and Laplace approximation to reconstruct a latent prior distribution. The paper validates the proposed method through three experiments: (1) Synthetic-data validation, which tests whether the algorithm can recover a known ground-truth distribution; (2) Caesar cipher experiments, which examine whether LLMs exhibit preferences over candidate shifts when informative input evidence is absent; and (3) BBQ experiments, which analyze social biases in ambiguous contexts and demonstrate how reconstructed distributions reveal behavioral differences that scalar bias scores may hard to imply.
Critical strengths
-
The paper provides a mathematically principled framework with explicit theoretical formulation. The authors formulate latent preference inference as a Bayesian inverse problem, explicitly modeling the shared latent prior, prompt-dependent evidence, and observation noise. They further develop a MAP-based reconstruction algorithm and use Laplace approximation to quantify uncertainty. This is important because the method is grounded in a clear mathematical formulation rather than relying solely on empirical scoring.
-
The paper offers a novel perspective on evaluating LLM biases beyond conventional benchmark scores. By reconstructing latent preference distributions, the method provides a more fine-grained characterization of how LLMs allocate probability mass across answer categories, revealing behavioral differences that may be obscured by scalar benchmark scores. The synthetic-data validation and experiments on Caesar ciphers and BBQ provide complementary evidence for the method's reconstruction capability and potential practical utility.
Critical weaknesses
-
The generalizability and validity of the proposed reconstruction framework remain insufficiently established. The evaluation is primarily conducted on a single social-bias benchmark, BBQ, with the original answer space mapped to three categories. It remains unclear whether the mapping preserves the relevant bias structure, whether alternative mappings or datasets would yield consistent conclusions, and whether the framework remains effective with larger or unrestricted answer spaces
-
The robustness and statistical reliability of the reconstructed preference distributions require more systematic evaluation. The paper does not systematically investigate how reconstructed distributions change with prompt templates, answer representations, sample sizes, or variations in observed output distributions. In particular, it remains unclear whether the inferred preferences are stable across semantically equivalent prompts and data subsets, and how reliably the method recovers latent preferences when the number and characteristics of observations change. If lack such robustness and sensitivity analyses, it is difficult to determine whether the observed differences reflect stable model behavior or artifacts of the experimental setup.
Questions for authors
- The BBQ experiments map the original answers to three categories: {S, A, U}. How sensitive are the reconstructed preference distributions to this mapping, and have the authors tested whether the method yields consistent conclusions on other bias benchmarks with the same {S, A, U} or a larger answer spaces?
- Have the authors evaluated how the reconstructed distributions change with the number of prompts, prompt templates, and answer representations? Specifically, do semantically equivalent prompts and different data subsets yield consistent preference estimates, and how does estimation uncertainty change as the sample size increases?
- The framework assumes constant prompt-dependent evidence in its main experiments. What evidence supports the framework's ability to distinguish latent preferences from prompt-specific effects when the latter vary across prompts? In other words, since the experiments assume constant prompt evidence, how do we know the method can still separate model preferences from prompt effects when the prompts differ?
2 comments
Reviewer Response
I appreciate the authors' responses and additional experiments, they address part of my concerns. While some questions regarding its generalizability and applicability to real-world settings remain, I find the revisions helpful. I'll keep my score.
Thank you for the clarifying questions regarding the answer space mapping as well as context sensitivity and prompt-specific evidence. We hope our responses below clarify the modeling choices we made as well as the organizational structure of the BBQ dataset.
Question 1 (Exploration of finer or larger answer spaces): Our {S,A,U} answer space for this experiment comes directly from the original organization of the BBQ dataset and is not an external mapping that we are imposing on the data. We are aligning the already provided answer choices with their semantic roles curated by the dataset.
We do acknowledge however that the exploration of finer and larger answer spaces is an excellent direction for future research. Our current contribution is to demonstrate how our proposed surrogate model can help interpret and further understand already established social bias benchmarks, of which BBQ is a prime example. Unfortunately, BBQ in its current form does not natively support finer/alternative groups and exploring such answer spaces would require an adaptation of this setup along with the use of additional benchmarks.
We do show, however, that our formulation does in fact support finer answer spaces with our shift-cipher experiments having 26 response categories. These experiments demonstrate that our method can be applied to a meaningful extent beyond a small set of 3 response categories as seen in our main BBQ experiment.
Question 2 (Stability of our reconstruction): We have incorporated an additional experiment to demonstrate the stability of our reconstruction to prompt variations. We have updated the manuscript and the new experiments can be found in the new Appendix C. The updated text is written in blue.
As a quick summary, we run our reconstruction algorithm on different random splits of the ambiguous BBQ dataset that contain different prompt variations intended to evaluate the same types of bias. We find that the reconstructed priors maintain stable under different randomized splits with a low mean absolute difference between the probabilities.
Question 3 (Nonconstant prompt evidence setting): Although the main LLM experiments use constant prompt evidence, Appendix B of our original submission explicitly evaluates nonconstant prompt evidence. We generate observations from a known multimodal prior and inject prompt-specific evidence over added Gaussian noise. Figure 4 shows that the reconstructed distribution closely follows the known prior while the empirical mean baseline is influenced by the injected prompt-specific evidence.