acceptodds
Under review as a conference paper at ICLR 2027

BINGO: Broadening Bias Discovery in LLM Judges via Diversity-Aware Optimization

Abstract

As Large Language Models (LLMs) continue to advance, they are increasingly used as evaluators, making it important to identify biases that influence their judgments beyond response quality. However, existing approaches to bias discovery often remain anchored to known bias patterns, limiting their ability to uncover biases beyond established categories. Moreover, primarily prioritizing strong bias effects can cause the search to repeatedly follow similar successful directions, limiting broader exploration of the search space and leaving other potentially effective bias factors undiscovered. To address these limitations, we introduce BINGO, an agentic framework for broadening automated bias discovery in LLM judges through diversity-aware optimization. Specifically, BINGO employs a dual-loop design: the inner loop promotes semantically and behaviorally diverse candidate bias factors, while the outer loop evaluates how strongly they influence judge preferences. In addition, a reflective feedback mechanism learns from successful, ineffective, and redundant search directions to guide subsequent exploration. Across four LLM judges, BINGO outperforms existing baselines in both bias strength and diversity, improving strength by 19.0% on average over the strongest baseline and reducing semantic and behavioral redundancy by up to 47.0%. Our qualitative analysis further shows that BINGO recovers known biases, refines broader categories, and uncovers novel bias directions.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM-as-a-Judge, Bias Discovery, Diversity-Aware Search, Agentic Search, Automated Evaluation, Prompt Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_bPSiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BINGO, an agentic framework for discovering strong and diverse bias factors in LLM judges. The method follows a dual-loop optimization process. The inner loop uses an optimizer LLM to generate candidate bias factors, filters semantically redundant candidates using embedding similarities, and selects behaviorally novel candidates by comparing their induced judge-decision vectors against previously discovered factors. The outer loop evaluates selected factors on a larger validation set and uses a reflector LLM to summarize successful, unsuccessful, and redundant directions to guide subsequent exploration.

Critical strengths

Existing bias discovery approaches may focus on previously known factors or concentrate on a limited range of successful candidates. Encouraging exploration of distinct judge vulnerabilities is a meaningful direction, particularly as LLM-based evaluation becomes increasingly widespread. The dual-loop architecture provides a clear separation between diversity-oriented candidate selection and strength-oriented validation. The combination of semantic filtering, behavioral comparison, and reflective feedback is intuitive and straightforward to implement.

Critical weaknesses
  1. The operational definition of bias does not fully distinguish undesirable bias from legitimate differences in response quality. The paper defines a bias factor as something that systematically influences an LLM judge's decision independently of response quality or correctness. However, its principal bias-strength measure is based on the preference of the judge for a modified response over an initially identical reference response. The experimental design assumes that because the injector preserves the original content and conclusion, any systematic preference for or against the injected response reflects bias. This assumption requires further justification. Preserving correctness or the main conclusion does not necessarily imply preserving overall response quality.
  2. Behavioral diversity may not adequately reflect the diversity of underlying bias mechanisms. A central contribution is the explicit optimization of semantic and behavioral diversity. Semantic novelty is measured using cosine similarity between factor descriptions, while behavioral novelty is measured using the absolute Pearson correlation between vectors of judge decisions. Although these measurements provide useful operational proxies, it is unclear whether they accurately capture diversity in the underlying bias mechanisms. In particular, low behavioral correlation can result from differences in which individual examples are affected, even when two candidate factors correspond to closely related underlying mechanisms. Conversely, distinct mechanisms might influence the same examples in similar ways and therefore exhibit high correlation.
Questions for authors
  1. Can the authors independently validate the diversity of the discovered bias factors?
  2. How does BINGO compare with simpler diversity-aware search methods under matched computational budgets?
  3. What constitutes a genuinely novel bias factor? Could the authors provide more systematic evidence that factors such as Personal Anecdotes represent previously uncharacterized judge vulnerabilities rather than variations of known style-related preferences?
7 comments
AuthorsOpenReview ↗

Response to Reviewer bPSi [1/4]

Dear Reviewer bPSi,

We greatly appreciate your thoughtful and detailed analysis of our work. Below, we provide a thorough response to each of the points you raised.


[W1] Differentiating Judge Bias from Genuine Quality Differences

We thank the reviewer for raising this important concern. While it is true that correctness may not guarantee to fully capture all aspects of reasoning quality, our correctness-based evaluation provides a meaningful and objective anchor for assessing reasoning quality. We consider this approach for the following reasons. 1) Reasoning quality is inherently difficult to define and assess comprehensively, and while LLM-based evaluation is a viable alternative, its assessments may vary across models and prompts.In contrast, 2) answer reconstruction provides a well-defined and objectively verifiable criterion for assessing whether the original answer remains derivable from the modified reasoning. For these reasons, constrained injection with answer reconstruction approach builds upon established practices in prior work [1, 2], and we carefully adopt this approach and further validate its reliability through additional experiments in Appendix I.1.

Nevertheless, to more comprehensively assess overall response quality beyond correctness, we conducted additional analyses.

(1) Neutral Rewriting Control. To separate the effects of bias factors from those of rewriting itself, we generate neutral rewrites rnr_n under identical injection settings but without a bias factor. Using GPT-5.1, we assess whether the overall response quality is similar or different for six factors (four from Appendix I and two additional factors), using 50 questions per factor. Evaluations are blinded and repeated with swapped response positions. Both orders yield similar quality judgments in 100% (rr vs. rnr_n), 97.3% (rnr_n vs. r~\tilde{r}), and 96.3% (rr vs. r~\tilde{r}) of cases, suggesting that rewriting alone does not introduce meaningful quality differences.

PairSimilarInconsistentDifferent
rr vs. rnr_n100.0%0.0%0.0%
rnr_n vs. r~\tilde{r}97.3%2.3%0.3%
rr vs. r~\tilde{r}96.3%3.3%0.3%

(2) Overall Quality Preservation. Beyond the correctness-focused checks in Appendix I, we directly assess overall response quality without mentioning correctness in the evaluation instructions. GPT-5.1 compares original and injected responses for the same six factors across 50 examples each, using blind, position-swapped evaluations. Additionally, seven human annotators each evaluate five pairs per factor, yielding 210 judgments. As the table below shows, GPT-5.1 detects a consistent quality difference in only 0.3% of comparisons, while human judgments confirm quality preservation in 94.8% of cases.

Bias FactorSimilar (GPT-5.1)Inconsistent (GPT-5.1)Different (GPT-5.1)Quality Preserved (Human)
Authority100.0%0.0%0.0%94.3%
Personal Anecdotes94.0%6.0%0.0%94.3%
Humor96.0%2.0%2.0%97.1%
Pop Culture References96.0%4.0%0.0%97.1%
Overly Simplistic Analogies98.0%2.0%0.0%91.4%
Aggressive Critique94.0%6.0%0.0%94.3%
Overall96.3%3.3%0.3%94.8%

Together, these complementary evaluations provide further support that the observed preference shifts primarily arise from sensitivity to injected presentation styles rather than substantive differences in response quality. We have these results in Appendix I.2 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer bPSi [2/4]

[W2] & [Q1] Validating Whether Diversity Proxies Capture Distinct Bias Mechanisms

We thank the reviewer for raising this question. To assess whether BINGO's diversity proxies capture meaningful differences beyond same-mechanism variation, we first compare two types of factor pairs: 1) same-mechanism pairs (A vs. A′), created by paraphrasing factor definitions while preserving their mechanisms and2) different-factor pairs (A vs. B), drawn from the final top-5 factors within each run. Using 15 factors from three runs with GPT-4o as the target judge and the same 500 test queries as Table 2, we find that different-factor pairs exhibit substantially lower semantic similarity and behavioral correlation.

Factor Pairs  Cosine Sim.     ∣ρ∣\lvert\rho\rvert  
Same-mechanism (AA vs. A′A')  0.931 ± 0.025  0.756 ± 0.065  
Different-factor (AA vs. BB)  0.375 ± 0.001  0.206 ± 0.045  

Furthermore, to examine whether BINGO's diversity measures capture meaningful differences beyond factor descriptions and judge decision patterns, we conduct an additional validation at the response level. We take each of the final top-5 factors in turn as a reference factor (A) and compare its response modifications against those induced by (1) semantically filtered candidates (description cosine similarity > 0.7 with A) and (2) the remaining four top-5 factors. We repeat this comparison for all five reference factors and report the average similarity for each group. To measure response-level similarity, we apply each factor to the same original responses and compute the cosine similarity between their response modification vectors (injected-response embedding − original-response embedding). Importantly, this metric is used only for validation and is not part of BINGO's optimization.

Run  Semantically Filtered Candidates  Final Top-5 Factors  
1  0.699  0.310  
2  0.735  0.318  
3  0.670  0.345  
Mean  0.701  0.324  

Across three runs using GPT-4o as the target judge, filtered candidates exhibit substantially higher response-modification similarity than the final top-5 factors. This suggests that semantic filtering removes redundant modifications, while behavioral novelty-based selection retains factors producing more distinct response modifications.

Finally, to independently validate the diversity of discovered bias mechanisms, we employ three external LLM raters (Grok 4.3, Qwen 3.7 Flash, and Gemini 3.8 Flash) to assess factor distinctness on a 1–5 scale, using both factor descriptions and examples of their modifications to the same original responses. Across three runs per method, we evaluate 1) pairs among the top-5 discovered factors (30 pairs), 2) discovered factors against five pre-discovered biases (75 pairs; Table 6), and 3) each factor against its own paraphrase as a control (15 pairs). As shown in the table, BINGO consistently outperforms both baselines across all raters, achieving distinctness scores of 4.30–4.72 within discovered factors and 4.38–4.66 against pre-discovered biases. Meanwhile, paraphrase controls uniformly score 1.00, indicating that raters distinguish genuinely different mechanisms from alternative descriptions of the same mechanism. These findings independently support the diversity of BINGO's discovered bias factors.

Judge  Method  Within-run  vs. Pre-discovered  Paraphrase Control  
GPT-4o  BiasScope  3.61  3.81  1.00  
OPRO  3.24  4.12  1.00  
BINGO  4.72  4.66  1.00  
Sonnet  BiasScope  3.40  4.12  1.00  
OPRO  3.89  4.20  1.00  
BINGO  4.37  4.53  1.00  
DeepSeek  BiasScope  3.53  3.90  1.00  
OPRO  3.75  4.19  1.00  
BINGO  4.37  4.56  1.00  
GPT-5.6  BiasScope  3.92  4.00  1.00  
OPRO  3.97  4.17  1.00  
BINGO  4.30  4.38  1.00  

Together, these results provide complementary evidence that BINGO's diversity proxies capture meaningful differences. These analyses are presented in Appendices K and L of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer bPSi [3/4]

[Q2] Comparison with Simpler Diversity-Aware Baselines Under Matched Computational Budgets

We thank the reviewer for this valuable suggestion. To the best of our knowledge, BINGO is the first to explicitly incorporate diversity into automated LLM-judge bias discovery, making direct comparisons difficult. However, to address this concern, we adapt simpler diversity-aware methods from other domains and compare them under matched budgets.

Baselines. All methods share BINGO's injector, judge (GPT-4o), data, and seed factor. While these baselines promote diversity through relatively simpler strategies, such as diversified generation or independent search trajectories, BINGO employs a more refined selection mechanism based on semantic and behavioral novelty.

  • OPRO [3]. Our original baseline, representing BINGO's simplest variant without its key components. For a fair comparison, we explicitly encourage diversity through the optimizer prompt (Appendix B.3).

  • Verbalized Sampling [4]. Originally developed to diversify LLM generation, this method produces multiple outputs with estimated probabilities. We adapt it to generate and evaluate five bias factors per iteration, updating the prompt with the strongest factors found so far. Then, we pick the strongest top-5 from all evaluated factors.

  • Island Model [5]. Originally developed for evolutionary program search, this method promotes diversity by maintaining multiple independent search groups (islands) each exploring different directions. We run four separate OPRO searches, each using only its own factor history. At each step, one island is randomly selected to generate and evaluate a factor. Every 12 evaluated candidates, the two weakest islands restart from the best factor of a randomly selected stronger island. Finally, we select the top-5 factors from all four searches.

We conduct two comparisons: (1) cost-matched, with the same per-run budget as BINGO (~$16.25), and (2) validated candidate-matched, where baselines are run until they evaluate the same number of candidates as BINGO. A validated candidate is considered as a bias candidate that undergoes injection-and-judgment evaluation, excluding initial seeds and duplicate proposals. BINGO evaluates three candidates per iteration during behavioral novelty selection, yielding 42 evaluated candidates (3 × 14 iterations, excluding iteration 1 that uses initial factor only). OPRO and Island Model evaluate one candidate per iteration; Verbalized Sampling evaluates up to five, but evaluates fewer if any duplicate previously proposed factors. We report averages over three runs per method.

Cost-matched

MethodCost/run ($)   S(b)\ ↑CosSim Avg ↓  CosSim Max ↓  ∣ρ∣\lvert\rho\rvert Avg ↓  ∣ρ∣\lvert\rho\rvert Max ↓  
OPRO16.66 ± 0.24  0.377 ± 0.006  0.545 ± 0.032  0.618 ± 0.025  0.458 ± 0.099  0.523 ± 0.082  
Verbal Sampling16.54 ± 0.15  0.346 ± 0.039  0.442 ± 0.054  0.486 ± 0.051  0.403 ± 0.021  0.482 ± 0.016  
Island Model16.38 ± 0.21  0.362 ± 0.019  0.537 ± 0.055  0.661 ± 0.059  0.465 ± 0.040  0.588 ± 0.061  
BINGO16.25 ± 0.20  0.416 ± 0.022  0.375 ± 0.001  0.448 ± 0.004  0.206 ± 0.045  0.297 ± 0.052  

Validated candidates matched

MethodCost/run ($)    S(b)S(b) ↑CosSim Avg ↓  CosSim Max ↓  ∣ρ∣\lvert\rho\rvert Avg ↓  ∣ρ∣\lvert\rho\rvert Max ↓  
OPRO33.27 ± 1.06  0.358 ± 0.031  0.550 ± 0.026  0.601 ± 0.009  0.447 ± 0.046  0.518 ± 0.046  
Verbal Sampling30.11 ± 0.23  0.371 ± 0.016  0.443 ± 0.036  0.518 ± 0.042  0.390 ± 0.050  0.444 ± 0.047  
Island Model31.09 ± 0.31  0.379 ± 0.010  0.463 ± 0.032  0.583 ± 0.060  0.389 ± 0.047  0.487 ± 0.055  
BINGO16.25 ± 0.20  0.416 ± 0.022  0.375 ± 0.001  0.448 ± 0.004  0.206 ± 0.045  0.297 ± 0.052  

Under a matched cost, BINGO achieves the highest bias strength and lowest redundancy across all four diversity metrics. Moreover, even when we increase the baselines' budgets by approximately 2× to match BINGO's 42 evaluated candidates, all three methods continue to underperform BINGO in both bias strength and diversity. These results demonstrate that BINGO's improvements cannot be replicated simply by allocating larger search budgets to simpler diversity-aware methods. We have added these analyses to Appendix J.

Reviewer_AbsNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BINGO, which combines diversity-based selection with reflective feedback to automate bias discovery in LLM judges. The main finding is that encouraging the search to explore different factors reduces redundancy while improving measured bias strength. This pattern holds across multiple judges. In the GPT-4o experiments, the discovered factors also retain an advantage on two unseen datasets. I understand the contribution as a method for auditing a broader range of judge preferences and finding factors that a search focused only on strength may overlook.

Critical strengths
  1. I find behavioral novelty selection a valuable part of the method. Auditing a judge requires finding different problems, and repeatedly finding variants of the same factor has limited value. In the GPT-4o ablation in Table 3, adding behavioral novelty selection to the variant with reflection reduces average absolute behavioral correlation from 0.337 to 0.217 while also increasing mean measured strength. This suggests that behavioral novelty selection helps the search find factors with more distinct effects on the judge.

  2. I also value the consistent improvements across judges. BINGO achieves higher mean measured strength than the baselines on all four judges. In the GPT-4o experiments, its discovered factors also retain an advantage on two unseen datasets. These results suggest that the discovered factors can transfer beyond the examples used during discovery.

Critical weaknesses
  1. Some factor-level results are not yet clearly interpretable as judge bias. In Table 7, the definition of Overly Simplistic Analogies permits analogies that distort the issue, while Aggressive Critique permits unnecessarily harsh criticism. If an edit reduces response quality, rejecting it may be reasonable, yet the bidirectional strength metric can still assign a high score. The checks in Appendix I provide useful support, but the QA human evaluation focuses on correctness and does not cover these two factors. This limits my confidence that the reported gains consistently reflect judge bias.

  2. Some steps that affect candidate selection remain unclear, which makes the component ablations harder to interpret. The behavioral stage in Section 3.2 and Algorithm 1 selects candidates by BN, but the reflector prompts mention cancelling behaviorally similar candidates. The optimizer prompts include regeneration within an iteration, and Appendix K mentions a regeneration budget, but Algorithm 1 does not explain how these steps enter candidate selection. The novelty scores also need clarification. Under Eq. (5), BN ranges from approximately 0.167 to 0.352 once five historical factors are available, and its upper bound shrinks mechanically from 0.731 to 0.352 in early iterations simply because the history size grows from 1 to 5. How this definition relates to the values in Figure 1 and the optimizer prompt’s instruction to drive novelty toward 1.00 is unclear. This leaves me unsure which scores informed the actual search. The description of QA strength as WR(b) on page 6 should also be made consistent with the definition on page 4.

Questions for authors
  1. For factors in Table 7 that may change response quality, how do the authors distinguish judge bias from a reasonable quality judgment? Please use representative edits and their win-rate directions to explain how the existing preservation checks support this interpretation.

  2. Did the experiments use behavioral deduplication beyond the BN selection described in Section 3.2 and Algorithm 1? Please explain how regeneration affected candidate selection and how these steps were used in the ablations. Please also clarify whether the BN scores supplied to the optimizer followed Eq. (5), and whether historical scores were recomputed or rescaled as the number of reference factors changed. A worked example from an actual iteration showing the correlations, computed scores, and selection decision would help establish which procedure was evaluated.

5 comments
AuthorsOpenReview ↗

Response to Reviewer AbsN [1/3]

Dear Reviewer AbsN,

We greatly appreciate your thoughtful and detailed analysis of our work. Below, we provide a thorough response to each of the points you raised.


[W1 & Q1] Distinguishing Judge Bias from Legitimate Quality

We thank the reviewer for highlighting this concern. Below, we address this by presenting our rationale about bias and quality and additional experimental results.

Distinguishing Quality from Bias. Given the nature of our ground-truth QA tasks, which have objectively verifiable answers, we adopt correctness-based evaluation to assess whether the injected responses preserve their original reasoning quality. Although this may not fully capture all aspects of reasoning quality, we believe it provides a meaningful and practical measure. Reasoning quality itself is difficult to define and assess, and while one could assess it by using LLM-based evaluation, its judgments may vary across models and prompts. In contrast, answer reconstruction allows us to assess reasoning quality against a specific, well-defined criterion that can be objectively verified. For these reasons, constrained injection with answer reconstruction has been widely adopted in prior work [1, 2], and we carefully build upon this established approach with additional validation experiments in Appendix I.

Nevertheless, to further support our claim that the observed preference shifts reflect judge bias rather than genuine quality differences, we conduct additional validation experiments and show qualitative examples.

(1) Correctness and Reasoning Preservation. To directly address the reviewer's concern particularly regarding Overly Simplistic Analogies and Aggressive Critique, we first extended the preservation evaluations in Appendix I.1 to include both factors, with consistent results across all evaluations. Notably, both factors achieve high masked-answer reconstruction accuracy, providing evidence that the original answers remain derivable from the injected reasoning. Substantial reasoning degradation would likely impair this ability, providing evidence of reasoning preservation beyond final-answer matching.

Bias FactorSame Conclusion ↑Answer Reconstruction ↑Bias Presence ↑Human Eval Correctness ↑
Overly Simplistic Analogies100%98%100%94.3%
Aggressive Critique100%100%100%94.3%

(2) Overall Quality Preservation. While the evaluations in Appendix I.1 focused on whether injected responses preserve correctness, including factual accuracy and reasoning validity, we further assess whether overall response quality is maintained beyond correctness. Specifically, we evaluate whether the overall quality of the original and injected responses is similar or different, without explicitly mentioning correctness. For the same six factors, GPT-5.1 evaluates 50 examples per factor through blind evaluation with position swapping, while seven human annotators each assess five pairs per factor (210 judgments in total). GPT-5.1 identifies a consistent 0.3% quality difference, and human annotators find overall quality preserved in 94.8% of judgments. These findings further support that the preference shifts discovered by BINGO primarily reflect the injected bias factors rather than genuine differences in response quality.

Bias FactorSimilar (GPT-5.1)Inconsistent (GPT-5.1)Different (GPT-5.1)Quality Preserved (Human)
Authority100.0%0.0%0.0%94.3%
Personal Anecdotes94.0%6.0%0.0%94.3%
Humor96.0%2.0%2.0%97.1%
Pop Culture References96.0%4.0%0.0%97.1%
Overly Simplistic Analogies98.0%2.0%0.0%91.4%
Aggressive Critique94.0%6.0%0.0%94.3%
Overall96.3%3.3%0.3%94.8%
AuthorsOpenReview ↗

Response to Reviewer AbsN [2/3]

[W1 & Q1] Distinguishing Judge Bias from Legitimate Quality (CONTD)

(3) Neutral Rewriting Control. To verify that the observed preference shifts arise from the injected bias factors rather than LLM rewriting itself, we introduce a neutral rewrite rnr_n, produced using the same injector, system prompt, and constraints, but without applying any bias factor. We use GPT-5.1 to assess whether the overall quality of two responses is similar or different across six factors (four from Appendix I and two highlighted by the reviewer) on the same 50 questions. Evaluations are conducted blindly with position swapping to control for order effects. The responses are judged to have similar quality in both orders in 100% (rr vs. rnr_n), 97.3% (rnr_n vs. r~\tilde{r}), and 96.3% (rr vs. r~\tilde{r}) of comparisons. These findings suggest that the substantial preference shifts observed in target judges stem primarily from the injected bias factors rather than underlying quality differences.

PairSimilarInconsistentDifferent
rr vs. rnr_n100.0%0.0%0.0%
rnr_n vs. r~\tilde{r}97.3%2.3%0.3%
rr vs. r~\tilde{r}96.3%3.3%0.3%

(4) Representative Edits and Win-Rate Directions. Lastly, we provide representative examples of the two factors the reviewer requested (Overly Simplistic Analogies and Aggressive Critique) to illustrate how their injected modifications influence judge preferences.

Example 1: Overly Simplistic Analogies (WR=0.301WR = 0.301)

Question: Is the Yellow Pages the fastest way to find a phone number?

ResponseContent
Original (rr)The Yellow Pages is a physical directory that lists businesses and services, which can be slow to navigate compared to digital options. In the modern era, online search engines or digital directories provide quicker access to phone numbers, making them faster than the Yellow Pages. So the answer is "false".
Injected (r~\tilde{r})The Yellow Pages is like that old friend who takes forever to find their keys—it's a physical directory that lists businesses and services, but navigating it can feel like searching for a needle in a haystack! In today's fast-paced world, online search engines or digital directories are like the speedy delivery service of phone numbers, zipping past the Yellow Pages with ease. So the answer is "false".

Example 2: Aggressive Critique (WR=0.040WR = 0.040)

Question: Would four shoes be insufficient for a set of octuplets?

ResponseContent
Original (rr)Each of the eight babies in a set of octuplets would typically need one pair of shoes, which means they would require a total of 8 pairs of shoes, or 16 individual shoes. Therefore, four shoes would be insufficient to provide for all eight babies. So the answer is "true".
Injected (r~\tilde{r})It's quite clear that each of the eight babies in a set of octuplets would typically need one pair of shoes. This means they would require a total of 8 pairs of shoes, or 16 individual shoes. Therefore, it's utterly ridiculous to think that four shoes could possibly suffice for all eight babies. So the answer is "true".

In the Overly Simplistic Analogies example, the injection introduces everyday analogies while preserving the original factual comparison and conclusion. Similarly, Aggressive Critique introduces harsh language without altering the underlying calculation or reasoning. These examples further illustrate how both factors influence judge preferences through stylistic changes while preserving the underlying response quality.

The additional results have been incorporated into the revised manuscript (Appendices F and I).

AuthorsOpenReview ↗

Response to Reviewer AbsN [3/3]

[W2]& [Q2] Clarification points and further explanations

  • Clarification q1: Behavioral Deduplication and Regeneration

We thank the reviewer for identifying this ambiguity and apologize for the confusion. The main experiments did not use behavioral deduplication, relying only on semantic filtering followed by BN-based selection. The behavioral-duplicate check (∣ρ∣=1.0\lvert\rho\rvert = 1.0) was introduced solely in the Behavioral + Reflector ablation, where semantic filtering was absent. In this variant, candidates perfectly correlated with existing factors were treated as redundant and provided to the reflector as feedback. However, this check was never triggered and had no effect on the reported results. In contrast, the main reflector used only semantically rejected candidates as feedback. To avoid confusion, we have updated the reflector prompt in the manuscript to reflect only the main experimental setting. Regarding regeneration, additional candidates are generated only when fewer than (M) survive semantic filtering, for up to five rounds. All regenerated candidates undergo the same semantic filtering and SN ranking before BN-based selection. If fewer than (M) survive, selection proceeds with the remaining candidates; if none survive, the search terminates.

  • Clarification q2. Novelty scores

We thank the reviewer for this careful observation and confirm that the stated bounds are correct. Given that the attainable upper bound of BN depends on (K), we recognize that “higher is better” would be a more precise instruction. Our original wording to drive novelty toward 1.00 was intended to convey the same optimization direction, using 1.00 as a fixed ideal reference regardless of (K), rather than as an attainable target. Both expressions encourage higher novelty and reflect the same optimization objective. Importantly, BN is primarily used for candidate selection, serving only as a qualitative signal of aggregate novelty to the optimizer. Within each iteration, all (M) candidates are evaluated against the same reference set, and the highest-BN candidate is selected. Thus, changes in the attainable BN range across iterations do not affect this within-iteration comparison. The BN values in Figure 1 are illustrative rather than actual experimental results, and we have revised them to align with the bounds in Eqs. (3) and (5).

  • Clarification q3. S(b) metric

Thank you for catching this typo. The correct definition is S(b)=∣WR(b)−0.5∣S(b) = \lvert WR(b) - 0.5 \rvert, which has been consistently used in all experiments and reported results for the QA setting. We have corrected it and apologize for the confusion.

We have clarified these implementation details and added an illustrative example of a complete iteration in Appendix R of the revised manuscript. We sincerely appreciate the reviewer's careful feedback.


[1] Lai et al., BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation, ICLR 2026

[2] Ye et al., Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge, ICLR 2025


We would like to express our sincere appreciation for the reviewer's insightful comments and careful examination of our work.

We hope that the additional analyses and clarifications have adequately addressed your concerns and provided a stronger basis for evaluating our contributions.

We remain grateful for your time and consideration and would warmly welcome any further questions or suggestions.

 

Thank you very much,
Authors

Reviewer_pDn1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BINGO, a diversity-aware optimization framework for discovering biases in LLM judges. The framework adopts a dual-loop architecture, where the inner loop promotes semantic and behavioral diversity among candidate bias factors, while the outer loop evaluates their bias strength. A reflection mechanism further guides the discovery process based on previously explored bias factors. Experiments across four LLM judges demonstrate that BINGO achieves higher bias strength and greater diversity compared to existing baselines.

Critical strengths
  1. Behavioral Novelty: One notable strength of this paper is the introduction of behavioral novelty into the bias discovery process. Unlike conventional approaches that primarily rely on semantic similarity, the proposed method considers how different bias factors influence the actual decisions of LLM judges. This provides a meaningful distinction between semantic and behavioral diversity, allowing the framework to identify bias factors that may be semantically different but behaviorally redundant.

  2. Strong Experimental Results: The experimental results are convincing. BINGO consistently outperforms the baselines in both bias strength and diversity across four LLM judges. The ablation study further demonstrates the contributions of the proposed components, while the experiments on unseen datasets provide additional evidence of generalizability.

Critical weaknesses

The proposed framework demonstrates strong empirical performance. However, its methodological novelty appears relatively limited, as the core components, including semantic similarity filtering, correlation-based behavioral diversity measurement, and reflection-guided optimization, largely build upon established techniques. Although their integration is effective, the paper could better establish what fundamentally distinguishes the proposed dual-loop design from existing quality-diversity optimization approaches.

Questions for authors

In Section 3.1, Equation (1) defines bias strength as (S(b) = |\mathrm{WR}_D(b) - 0.5|), measuring the absolute deviation of the win rate from 0.5. However, Section 4.1 states that bias strength in the main QA setting is measured as (\mathrm{WR}(b)). These two definitions have substantially different interpretations and potentially different evaluation outcomes. Could the authors clarify which definition is actually used in the experiments reported in Table 2 and confirm whether the reported results consistently follow Equation (1)?

2 comments
AuthorsOpenReview ↗

Response to Reviewer pDn1

Dear Reviewer pDn1,

We appreciate your insightful comments and the time dedicated to evaluating our work. We have addressed each point below.


[Weakness] Novelty and contribution of BINGO

We thank the reviewer for recognizing BINGO's strong empirical performance and appreciate the opportunity to clarify its novelty. To the best of our knowledge, BINGO is the first work to address open-ended LLM-judge bias discovery without relying on predefined bias categories. Specifically, we introduce a novel diversity-aware optimization framework designed specifically for LLM judges, leveraging judge-specific behavioral diversity to discover strong and distinct biases.

Existing LLM judge bias discovery approaches. For the automated discovery of biases in LLM judges, existing methods primarily expand from known bias seeds or extract factors from observed failures and human–judge disagreements (BiasScope [1]; HUB-J [2]). While effective, these approaches can remain anchored to known bias patterns or observed failure cases. To the best of our knowledge, BINGO is the first framework to explicitly incorporate diversity into automated LLM-judge bias discovery, encouraging exploration beyond known bias patterns.

LLM-driven optimization. Meanwhile, optimization methods such as OPRO [3] and GEPA [4] primarily optimize task performance through iterative search. However, optimizing bias strength alone can repeatedly favor similar successful factors. Prompt-level diversity instructions alone are insufficient: even our OPRO baseline, which explicitly encourages diversity (Appendix B.3), produces more redundant factors than BINGO. BINGO instead introduces a behavioral diversity measure designed specifically for LLM-judge bias discovery, systematically quantifying diversity based on how different bias factors influence the target judge's decision patterns. BINGO also explicitly considers computational efficiency, as individually evaluating the target judge's behavior for all candidate factors would incur substantial cumulative costs, and thus incorporates lightweight semantic filtering to eliminate highly redundant candidates and avoid unnecessary behavioral evaluations

Quality-diversity approaches. QD methods explicitly incorporate diversity, but traditional approaches often organize diversity around predefined feature descriptors. For example, Rainbow Teaming [5], developed for adversarial red-teaming, maintains an archive indexed by predefined dimensions of risk category and attack style. However, in open-ended bias discovery, the relevant bias dimensions are unknown, making predefined descriptors restrictive. BINGO instead measures diversity relative to factors proposed in earlier iterations of its own search, using these accumulated proposals to guide exploration and progressively expand the search space. By leveraging the target judge's actual decision patterns, BINGO identifies distinct behavioral vulnerabilities without requiring predefined bias dimensions. This enables the discovery of new bias mechanisms beyond predefined categories.

Together, these distinctions highlight BINGO's methodological contribution through a judge-specific behavioral diversity objective and an efficient two-stage selection strategy, enabling systematic discovery of strong and diverse judge vulnerabilities beyond existing bias patterns, predefined descriptors, or prompt-level diversity instructions alone.


[Question] Clarification for S(b) metric

We thank the reviewer for catching this typo in lines 306-307. The correct definition is S(b)=∣WR(b)−0.5∣S(b) = \lvert WR(b) - 0.5 \rvert, consistent with page 4. We have corrected this on page 6 of the revised manuscript and apologize for the confusion.


[1] Lai et al., BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation, ICLR 2026

[2] Koo et al., Auditing the Judge: Human-Grounded Bias Discovery, Quantification, and Mitigation in LLM Judges, ICML AI4GOOD Workshop 2026

[3] Yang et al., Large Language Models as Optimizers, ICLR 2024

[4] Agrawal et al., GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning, ICLR 2026

[5] Samvelyan et al., Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts, NeurIPS 2024


We sincerely appreciate the reviewer's thoughtful questions, which have provided us with a valuable opportunity to further clarify our contributions. We hope that our explanations have adequately addressed your concerns.

We would be happy to provide any further clarification or address any remaining questions.

Thank you again for your time and consideration.

 

Thank you very much,
Authors

AuthorsOpenReview ↗

Kind Invitation to Review Our Responses (30 Hours Remaining)

Dear Reviewer pDn1,

We are very grateful for the time and care you have devoted to evaluating our work.

Following your helpful comments, we provided further explanations to clarify the methodological contributions of our framework, particularly its distinctions from existing LLM-judge bias discovery, LLM-driven optimization, and quality-diversity approaches. We also clarified the definition of our bias strength metric and corrected the inconsistency in the revised manuscript.

We hope these explanations offer a clearer picture of our contributions and help resolve the questions you raised. Should you have a chance to revisit our response, we would warmly welcome any thoughts or suggestions you might wish to share. We would also be glad to elaborate on any points that may still need clarification.

We are truly thankful for your consideration and for the opportunity to improve our work through your feedback.

Sincerely,
Authors

Reviewer_4mjtOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BINGO, an automated framework for discovering strong and diverse factors that influence the preferences of LLM judges. Its dual-loop design combines semantic filtering and behavioral novelty selection with bias-strength evaluation and reflective feedback to guide subsequent exploration. Across four judges, the authors report improvements over predefined bias factors, BiasScope, and OPRO, with additional experiments examining transfer to unseen datasets and an open-ended preference setting. Component ablations, initialization and top-K sensitivity analyses, and automated and human validation provide further evidence about the framework’s behavior and the preservation of response content.

Critical strengths
  1. The method addresses a meaningful limitation of automated bias discovery. Searching only for high-strength factors can repeatedly produce related interventions, limiting the usefulness of the resulting audit. BINGO explicitly combines semantic and behavioral diversity with strength-based feedback, providing a well-motivated approach to discovering a broader collection of judge sensitivities (Section 3).
  2. The reported improvements are consistent across the four main judges. BINGO achieves the highest mean strength and the lowest mean values for all four redundancy metrics in Table 2. For example, its strength increases from the strongest baseline’s 0.364 to 0.416 on GPT-4o and from 0.118 to 0.179 on Sonnet. Reporting results over three runs and evaluating on a separate test set strengthens this empirical comparison.
  3. Experimental results match their claims and partially support their findings.
Critical weaknesses
  1. The main evaluation does not sufficiently distinguish judge bias from legitimate preferences about response quality. Equation (1) rewards any systematic deviation from a 50% win rate, including consistent rejection of the injected response. However, the judge is asked which answer is “better” (Appendix L, Figure 6), whereas the QA human validation primarily checks whether correctness is preserved (Appendix I.1). Factors such as irrelevant personal anecdotes, overly simplistic analogies, and unwarranted aggressive criticism can reduce relevance, clarity, or usefulness without changing the final answer. Consequently, high measured strength can reflect an appropriate quality judgment. I acknowledge the open-ended human preference check in Appendix I.2, but its 20 examples do not resolve this concern for the main QA setting. A broader assessment of overall quality, including neutral rewriting controls and signed effects, is needed to substantiate the central interpretation.
  2. The computational comparability of the baselines is unclear. Using the same examples and iteration limit does not establish comparable search budgets. BINGO proposes ten candidates per iteration, probes three behaviorally, evaluates a selected candidate on the validation set, and additionally uses reflection and candidate regeneration (Section 4.1; Appendix K).
  • Appendix B does not provide corresponding candidate-evaluation counts, judge calls, token usage, or total costs for OPRO and BiasScope. This prevents a clear assessment of whether the gains come from diversity-aware selection or differences in the amount of search performed. Results under matched query or computational budgets would substantially strengthen the comparison.
  1. The evidence for meaningfully broader bias discovery relies heavily on the optimized diversity proxies. Held-out improvements in cosine similarity and absolute decision correlation demonstrate generalization of those metrics, but do not independently establish that the factors represent distinct bias phenomena. In particular, two factors that both lose on almost every query can have low correlation when their rare exceptions differ; this warrants examination given the near-maximal strength scores for some judges in Table 2. The qualitative analysis mainly provides factor names and definitions, leaving limited evidence about whether ostensibly new factors introduce distinct mechanisms or combine familiar properties. Representative original/injected examples, an independent assessment of factor distinctness, and checks on the stability of low-variance behavioral vectors would make the breadth claim more convincing.
Questions for authors

See Weakness above

5 comments
AuthorsOpenReview ↗

Response to Reviewer 4mjt [1/4]

Dear Reviewer 4mjt,

We sincerely appreciate your thoughtful and constructive review. We have reviewed your comments carefully and provide detailed responses below.


[W1] Separating bias effects from quality changes (neutral rewriting & overall quality)

We thank the reviewer for raising this important concern. As the reviewer correctly notes, we assess reasoning quality preservation primarily through correctness-based evaluation. While this may not fully capture all aspects of reasoning quality, we believe it offers a meaningful and practical alternative for the following reasons.

  • Reasoning quality itself is inherently difficult to define and assess. LLM-based evaluation is considerable, but it may vary across models and prompts.
  • In contrast, answer reconstruction allows us to define reasoning quality in terms of a specific, well-defined criterion and objectively verify it, thereby avoiding these issues.

For these reasons, constrained injection with answer reconstruction has been widely adopted in prior work [1, 2], and we carefully build upon this established approach with additional validation experiments in Appendix I.

Nevertheless, to further address the reviewer's concern, we conduct additional evaluations of overall response quality.

(1) Neutral rewriting control. To isolate the effect of bias injection from LLM rewriting, we introduce a neutral rewrite rnr_n, generated with the same injector, system prompt, and constraints as r~\tilde{r} , but without the bias factor. Following the reviewer’s suggestion, GPT-5.1 evaluates overall quality for six factors (four from Appendix I and two additional reviewer-highlighted factors) on the same 50 questions, using blind evaluation with position swapping to mitigate order bias. The pairs are judged Similar in both orders in 100% (rr vs. rnr_n), 97.3% (rnr_n vs. r~\tilde{r}), and 96.3% (rr vs. r~\tilde{r}) of cases. Thus, the strong preference shifts observed in target judges are primarily attributable to the injected bias factors rather than genuine quality differences.

PairSimilarInconsistentDifferent
rr vs. rnr_n100.0%0.0%0.0%
rnr_n vs. r~\tilde{r}97.3%2.3%0.3%
rr vs. r~\tilde{r}96.3%3.3%0.3%

(2) Overall quality preservation. The evaluations in Appendix I explicitly asked whether the injected response remains correct, i.e., whether it reaches the same answer and contains no factual or reasoning errors. To check that injection also preserves quality beyond correctness, we ran a new evaluation with both LLM and human annotators, by asking about overall response quality without any mentioning of correctness. Using the same six factors, GPT-5.1 judged whether the overall quality of r and r is similar or different on 50 examples per factor, using blind evaluation with position swapping, and seven human annotators each assessed five pairs per factor (210 judgments in total). GPT-5.1 found a consistent quality difference in only 0.3%, and human annotators judged overall quality to be preserved in 94.8% of judgments. These results indicate that the preference shifts identified by BINGO are driven by the injected factors rather than by changes in overall quality.

Bias FactorSimilar (GPT-5.1)Inconsistent (GPT-5.1)Different (GPT-5.1)Quality Preserved (Human)
Authority100.0%0.0%0.0%94.3%
Personal Anecdotes94.0%6.0%0.0%94.3%
Humor96.0%2.0%2.0%97.1%
Pop Culture References96.0%4.0%0.0%97.1%
Overly Simplistic Analogies98.0%2.0%0.0%91.4%
Aggressive Critique94.0%6.0%0.0%94.3%
Overall96.3%3.3%0.3%94.8%

We have incorporated the neutral rewriting results and overall quality assessment into Appendix I.2 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 4mjt [2/4]

[W2] Computational comparability under matched budgets

We thank the reviewer for this constructive suggestion. We conduct cost-matched and validated candidate-matched comparisons to assess whether BINGO's gains arise from additional computation. The results show that BINGO consistently outperforms both baselines under matched budgets, and that substantially increasing their computational budgets does not close the performance gap.

Following Sec. 4.3, we use GPT-4o as the target judge, with the same datasets as our main experiments. Each method is run three times, with its top-5 factors evaluated on the held-out test set. We define a validated candidate as a bias factor that has undergone injection-and-judgment evaluation, excluding initial seeds. For BINGO, three candidates are evaluated during behavioral novelty selection per iteration, yielding 3 x 14 iter. (excluding iteration 1 that uses initial factor) = 42 unique candidate evaluations. OPRO evaluates one per iteration, and BiasScope counts only new factors after merging.

We first compare all methods under the same budget (~$16.25 per run, matching BINGO’s average cost), allowing OPRO and BiasScope to search until the budget is exhausted. BINGO achieves the highest bias strength and lowest redundancy across all four diversity metrics.

MethodCost/run ($)  S(b)S(b) ↑  CosSim Avg ↓  CosSim Max ↓  ∣ρ∣\lvert\rho\rvert Avg ↓  ∣ρ∣\lvert\rho\rvert Max ↓  
BiasScope16.96 ± 0.13  0.136 ± 0.049  0.576 ± 0.057  0.646 ± 0.072  0.430 ± 0.036  0.537 ± 0.022  
OPRO16.66 ± 0.24  0.377 ± 0.006  0.545 ± 0.032  0.618 ± 0.025  0.458 ± 0.099  0.523 ± 0.082  
BINGO16.25 ± 0.20  0.416 ± 0.022  0.375 ± 0.001  0.448 ± 0.004  0.206 ± 0.045  0.297 ± 0.052  

We then increased the baselines’ budgets to match BINGO’s 42 candidates that underwent validation per run. Despite additional computation, neither baseline closes the gap: OPRO requires ~2× the cost without closing the gap, while BiasScope consumes ~7× the cost over 50 iterations but reaches only 21 unique candidates and continues to underperform across all metrics. Given this cost, we report the extended BiasScope experiment from a single run, as indicated in the table. Together, these results show that BINGO’s performance cannot be obtained simply by allocating more computation to the baselines.

MethodCost/run ($)  S(b)S(b) ↑  CosSim Avg ↓  CosSim Max ↓  ∣ρ∣\lvert\rho\rvert Avg ↓  ∣ρ∣\lvert\rho\rvert Max ↓  
BiasScope114.32 (21 cand.)  0.195  0.669  0.732  0.590  0.647  
OPRO33.27 ± 1.06  0.358 ± 0.031  0.550 ± 0.026  0.601 ± 0.009  0.447 ± 0.046  0.518 ± 0.046  
BINGO16.25 ± 0.20  0.416 ± 0.022  0.375 ± 0.001  0.448 ± 0.004  0.206 ± 0.045  0.297 ± 0.052  

We added these analyses to Appendix J.

AuthorsOpenReview ↗

Response to Reviewer 4mjt [3/4]

[W3] Validating the Distinctness, Diversity, and Stability of Discovered Bias Factors

We sincerely thank the reviewer for the thoughtful feedback on our diversity metrics and factor distinctness, which we address through the additional analyses below.

(1) Independent assessment of factor distinctness. To evaluate distinctness beyond our optimized metrics, we use three LLM raters not involved in the main experiments (Grok 4.3, Qwen 3.7 Flash, Gemini 3.8 Flash) to score how different two bias mechanisms are from 1 (same) to 5 (completely different). Each rater is also provided with examples of how each factor modifies the same original responses. We evaluate three pair types across three runs: (1) within-run top-5 pairs, comparing every two factors (C(5,2)=10 pairs/run, 30 total); (2) top-5 vs. pre-discovered biases, pairing each of the five factors with five pre-discovered biases (5 × 5 = 25 pairs/run, 75 total); and (3) paraphrase controls, pairing each factor with a paraphrase of its own definition (5 pairs/run, 15 total) to verify that raters recognize the same mechanism.

Judge  Method  Within-run  vs. Pre-discovered  Paraphrase Control  
GPT-4o  BiasScope  3.61  3.81  1.00  
OPRO  3.24  4.12  1.00  
BINGO  4.72  4.66  1.00  
Sonnet  BiasScope  3.40  4.12  1.00  
OPRO  3.89  4.20  1.00  
BINGO  4.37  4.53  1.00  
DeepSeek  BiasScope  3.53  3.90  1.00  
OPRO  3.75  4.19  1.00  
BINGO  4.37  4.56  1.00  
GPT-5.6  BiasScope  3.92  4.00  1.00  
OPRO  3.97  4.17  1.00  
BINGO  4.30  4.38  1.00  

BINGO achieves the highest distinctness across all raters, both among its discovered factors and against pre-discovered biases, outperforming both baselines. The paraphrase control consistently scores 1.00, confirming that raters recognize equivalent mechanisms despite different wording. These results provide further evidence that BINGO discovers diverse underlying bias mechanisms. The per-judge results and corresponding discussion have been included in Appendix K of the revised manuscript.

(2) Diversity in actual response modifications. We examine whether description-level diversity translates into actual response differences. For each final top-5 factor A, we compare semantically filtered candidates (description cosine > 0.7 with A) against the other four top-5 factors, and report the averaged results across all five factors.We apply each factor to the same original responses and measure cosine similarity between their response modification vectors (injected-response embedding − original-response embedding). This response-level metric is not used during BINGO’s optimization.

Run  Semantically Filtered Candidates  Final Top-5 Factors  
1  0.699  0.310  
2  0.735  0.318  
3  0.670  0.345  
Mean  0.701  0.324  

Across three GPT-4o runs, candidates removed as duplicates of A induce far more similar edits than the other final selected factors. This suggests that BINGO’s semantic filtering captures redundancy in actual response modifications, not merely similarity between factor descriptions. This experiment is added in Appendix L in the revised manuscript.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response

Dear reviewers and AC,

We are grateful for the thoughtful time and attention you have given to evaluating our work.

We are encouraged that the reviewers recognized BINGO's meaningful contribution to automated bias discovery (4mjt, bPSi), particularly its behavioral novelty selection that goes beyond semantic similarity by considering actual judge decisions (pDn1, AbsN). Reviewers also appreciated its intuitive dual-loop design (bPSi), consistent improvements in bias strength and diversity across four LLM judges (4mjt, pDn1, AbsN), and generalization to unseen datasets (pDn1, AbsN).

We appreciate each reviewer’s constructive and valuable feedback on our manuscript. We have carefully updated and improved the manuscript in response to the feedbacks, including the following additional discussions and experiments:

1. Distinguishing bias from quality changes (4mjt, AbsN, bPSi)

  • Extended correctness preservation checks (Appendix I.1)
  • Neutral rewriting and LLM/human overall-quality evaluations (Appendix I.2)
  • Representative examples with win-rate directions (Appendix F)

2. Computational comparability (4mjt, bPSi)

  • Matched-budget comparisons and simpler diversity-aware baselines (Appendix J)

3. Validating diversity and novelty of discovered factors (4mjt, bPSi)

  • Independent factor-distinctness assessment (Appendix K)
  • Mechanism- and response-level diversity validation (Appendix L)
  • Behavioral stability analysis (Appendix N)
  • Novelty beyond known-style bias combinations (Appendix O)
  • Qualitative examples (Appendix F)

4. Methodological clarity and typo corrections (pDn1, AbsN) (Sec. 4.1; Appendices A, R, S, T)

In the revised manuscript, these updates are temporarily highlighted in green\textbf{\color{green}green} for your convenience to check.

We sincerely believe that these revisions strengthen our work and better highlight BINGO's contributions to the ICLR community. We sincerely thank the reviewers once again for their thoughtful feedback and would be happy to address any further questions or concerns.

Sincerely,
Authors

AuthorsOpenReview ↗

Gentle Reminder of Our Rebuttal

Dear Reviewers and AC,

We sincerely thank you for your time and constructive feedback on our submission.

As the discussion period closes in three days, we would like to kindly follow up on our responses. We have replied to each review individually, and we would be grateful if you could let us know whether our responses address your concerns. If any questions remain, we would be glad to provide further clarification or additional results in the remaining time.

Thank you again for your effort and consideration.

Best regards,
Authors

The reviewers have been reminded

Best,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.