acceptodds
Under review as a conference paper at ICLR 2027

Beyond Demographics: Counterfactual VQA for Auditing Fairness in Black-Box LVLMs

Abstract

Recent advances in large vision–language models (LVLMs) have amplified concerns about fairness, yet existing evaluations remain largely centered on demographic attributes (e.g., gender and race) and often conflate fairness with refusal behavior. In this study, we reveal an overlooked risk for LVLM fairness: decision stability degrades substantially under visual changes (e.g., social behaviors and aesthetic elements), even in proprietary systems. To audit this risk, we introduce ABCDE-Bench, a paired-image VQA benchmark that tests across five attribute categories: Aesthetics, Behavior, Culture, Demography, and Environment. We further propose refusal-aware metrics for black-box evaluation that disentangle decision instability from refusal asymmetry. Experiments show that across diverse LVLMs, counterfactual inconsistencies are widespread, and LVLMs consistently show lower stability under non-demographic than demographic changes, whereas human performance shows the opposite tendency. Finally, we examine lightweight mitigation strategies and find that human-norm exemplars increase consistency more uniformly across models than instruction prompting alone. Refusal-aware analysis further shows that the observed consistency gains under these interventions are accounted for by increased symmetric refusal, while the contribution from matching non-refusal answers decreases. Overall, our findings highlight an underexplored non-demographic source of instability and motivate broader, refusal-aware audits of LVLM fairness beyond demographics.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Fairness, Large Vision-Language Models, Counterfactual Invariance

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wuhGOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ABCDE-Bench, a counterfactual VQA benchmark for auditing fairness in black-box LVLMs. It covers five categories of visual attributes and uses human-validated image pairs to evaluate decision consistency. The authors also propose refusal-aware metrics to distinguish inconsistent answers from refusal behavior. Experiments across 11 LVLMs reveal lower consistency under non-demographic changes, while prompting-based interventions mainly improve consistency by increasing symmetric refusals.

Critical strengths

1.The benchmark extends fairness evaluation beyond standard demographic attributes. The broad attribute coverage and human validation make it a useful resource for studying contextual sensitivity in LVLMs. 2.The evaluation covers a wide range of proprietary and open-weight models. The refusal-aware analysis is particularly useful, as it shows that higher consistency does not necessarily mean more stable substantive answers.

Critical weaknesses

1.I'm not fully convinced by the comparison between demographic and non-demographic attributes. The categories involve different questions and decision targets, so the consistency gap may partly reflect differences in task difficulty rather than sensitivity to the manipulated attributes. A more controlled comparison using matched questions would help clarify this. 2.The connection between decision consistency and fairness remains somewhat unclear. Many questions ask models to infer subjective properties such as trustworthiness, income, or competence from images, where no reliable answer may exist. In these cases, inconsistent responses do not necessarily indicate unfair treatment. The paper would benefit from a clearer distinction between general response instability and fairness-related failures. 3.The human comparison needs stronger support. Human performance is measured on only 30% of the benchmark, while the reported model results cover the full dataset. The claimed opposite trend between humans and models would be more convincing with results on the same subset and uncertainty estimates.

Questions for authors

1.Does the demographic versus non-demographic consistency gap remain when question types and difficulty are controlled across categories? 2.Can the authors compare human and model consistency on exactly the same evaluation subset, with confidence intervals?

3 comments
AuthorsOpenReview ↗

Response to Reviewer wuhG

Dear Reviewer wuhG,

Thank you for the positive assessment of our benchmark and refusal-aware analysis. We address the concerns through additional controlled comparisons and a matched human–model analysis.

W1 and Q1. Question composition and task difficulty

To examine whether question composition explains the observed gap, we grouped the benchmark questions into 15 question-target types and standardized the demographic and non-demographic groups to a common target distribution. The model-average gap remains +6.5 points after adjustment, compared with +6.4 points before adjustment. For example, the gaps within income/SES, education, and competence are +5.0 (positive for all 11 models), +6.0 (10/11 models), and +6.4 points (10/11 models), respectively.

We also conducted a controlled experiment using 60 base images, each independently edited to change either a demographic or a non-demographic attribute, yielding 120 counterfactual pairs in total. For each base image, both counterfactual pairs were evaluated using the same question and answer options. Non-demographic edits were balanced across the four categories, with questions covering the three targets above. The gap remains positive for both GPT-5 (+4.4 points) and Claude Sonnet 5 (+7.2 points), averaging +5.8 points (95% CI: [0.6, 11.4]).

These results further support our main finding when base images and VQA tasks are held fixed, suggesting that differences in question composition alone do not explain the observed gap. Both analyses have been added to Appendix E.7 (highlighted in magenta).

W2. Decision consistency and fairness

Counterfactual invariance has a well-established role in fairness evaluation [1], and related counterfactual probing approaches have also been used in NLP and vision-language fairness research [2–4]. Our work extends this invariance-based perspective to black-box LVLMs, examining changes to attributes that human annotators judge inappropriate as evidence for the specific question (Sec. 3.3, Step 2). Unlike general robustness tests under arbitrary visual perturbations, our evaluation is grounded in human judgments about which visual attributes should not influence a particular decision.

For subjective questions such as income or competence, a uniquely verifiable answer cannot be established from an image alone, and our benchmark does not prescribe a desirable response to each question. Counterfactual invariance can be evaluated without determining whether an individual prediction is correct. We therefore adopt a ground-truth-free evaluation that measures consistency across counterfactual image pairs, without judging individual answers as correct or refusals as incorrect (Sec. 3.3, L249–250).

[1] Kusner+, Counterfactual Fairness, NeurIPS2017.
[2] Zhao+, Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods, NAACL2018.
[3] Howard+, SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples, CVPR2024.
[4] Xiao+, GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing, IJCV2025.

W3 and Q2. Human-model comparison

To ensure a direct comparison, we recomputed model consistency on exactly the same 30% subset used for human evaluation and calculated pair-level bootstrap confidence intervals. On these identical pairs, the model-average demographic–non-demographic gap remains positive at +7.7 points (95% CI: [3.5, 11.9]), whereas the human gap is −3.8 points (95% CI: [−14.1, 6.6]). The paired model–human difference in gaps is +11.5 points (95% CI: [0.4, 22.3]). We have added the matched comparison results to Finding 1 (highlighted in magenta).

The matched comparison thus supports a larger demographic–non-demographic gap in models than in humans. The human point estimate shows an opposite tendency, although its confidence interval includes zero. We therefore interpret this trend descriptively rather than as a statistically established reversal. We have revised the corresponding statements in the abstract, introduction, and Finding 1 (highlighted in magenta) to avoid implying a statistically established opposite trend. As noted in our Limitations (L1510–1511), human evaluation serves as a behavioral reference rather than a normative fairness baseline.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer wuhG,

Thank you again for your time and constructive feedback on our submission. We have carefully considered your comments and addressed them in our rebuttal.

We understand that you may be busy, but we would appreciate it if you could review our response before the discussion period ends. If any points remain unclear, we would be happy to clarify them.

Best regards,
The Authors

Thank you for the detailed response and additional experiments. They directly address my concerns, so I will maintain my positive rating and increase my confidence.

Reviewer_A8k1OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces ABCDE-Bench, a paired-image VQA benchmark designed to audit large vision-language models (LVLMs) for decision stability under human-validated "irrelevant" visual changes across five categories (Aesthetics, Behavior, Culture, Demography, and Environment). The authors propose refusal-aware stability metrics—such as Consistency-Coverage AUC (CC-AUC) and Symmetric Refusal AUC (SR-AUC)—to disentangle true decision consistency from refusal asymmetries. Experiments across multiple proprietary and open-weight models show that models are generally less stable under non-demographic variations than demographic ones. Additionally, the authors test mitigation strategies like instruction prompting and exemplar retrieval, showing that consistency gains are heavily driven by increased symmetric abstention.

Critical strengths
  1. This work moves beyond traditional demographic attributes (e.g., gender and race) to examine non-demographic contextual elements (such as social behaviors and aesthetic cues) addresses a tiny blind spot in current VLMs' fairness evaluations.

  2. The introduction of CC-AUC and SR-AUC provides a diagnostic framework to prevent models from masking underlying instability through safe or asymmetric refusal behaviors.

Critical weaknesses
  1. The paper's core premise, changing any non-demographic attribute deemed "irrelevant" by humans should never alter an LVLM's output, is conceptually fragile and overly simplistic. In open-world visual reasoning, many non-demographic cues (e.g., clothing style, setting, or background objects) legitimately serve as probabilistic soft evidence or statistical hints for human and machine inference. Treating any sensitivity to these factors as a "fairness violation" or "undesired contextual sensitivity" lacks rigorous theoretical grounding.

  2. The construction pipeline relies heavily on LLM-generated proposals and open-world questions that are inherently underspecified and lack ground-truth labels. Because the questions are speculative (e.g., asking about a person's probable socioeconomic status, trustworthiness, or professional credibility based on a single image), there is no objective standard for what the "correct" model behavior should be. Measuring "consistency" on arbitrary, subjective, or speculative queries tests general robustness rather than concrete algorithmic fairness.

  3. The automated pipeline uses LLMs and text-to-image generators to synthesize counterfactual pairs. Despite heuristic fidelity diagnostics, synthetic image editing frequently introduces unintended confounding artifacts, style shifts, or semantic drift outside the intended attribute. It remains unproven whether the observed decision instabilities stem from genuine bias or simply from out-of-distribution artifacts and visual noise inherent in synthetic image generation.

  4. The empirical comparisons across models, categories, and mitigation strategies lack formal statistical significance testing (e.g., confidence intervals, p-values, or variance across multiple distinct prompt formulations). Given that models exhibit high sensitivity to prompt formatting, minor score fluctuations are over-interpreted as definitive performance gaps (e.g., the demographic vs. non-demographic gap).

Questions for authors

Given that many queries in ABCDE-Bench involve subjective or speculative inferences (such as judging trustworthiness or income from visual appearance), how do the authors separate genuine fairness violations from justified model uncertainty or rational probabilistic use of contextual soft cues? Furthermore, what statistical guarantees ensure that the reported decision instabilities are caused by the targeted attributes rather than uncontrolled visual artifacts introduced during synthetic image generation?

3 comments
AuthorsOpenReview ↗

Response to Reviewer A8k1 (1/2)

Dear Reviewer A8k1,

Thank you for the detailed review. We respectfully disagree with the reviewer's assessment of the conceptual basis of our fairness evaluation. We clarify the underlying fairness principle and provide additional statistical evidence.

W1 and Q1. The fairness principle and contextual evidence

Counterfactual invariance has a well-established role in fairness evaluation [1], where predictions are expected to remain invariant under specified counterfactual changes. Related counterfactual probing approaches have also been used in NLP and vision-language fairness evaluations [2–4]. Our work extends this invariance-based perspective to behavioral auditing of black-box LVLMs, covering both demographic and non-demographic attributes.

Importantly, a cue's statistical predictive value does not by itself determine whether it is appropriate evidence for an individual-level judgment, a distinction that also motivates counterfactual fairness [1]. Rather than assuming that all non-demographic attributes should be ignored, we validate each attribute–question pair through human judgments of whether the attribute should influence the particular decision (Sec. 3.3, Step 2). Indeed, this validation excluded approximately 34% of generated tasks (L256–257), e.g., laboratory equipment modernity as evidence for research funding (mean validity rating 4.8; Appendix C.1).

The demographic–non-demographic gap also persists under stricter irrelevance thresholds, ranging from +5.4 to +6.4 points as the cutoff is tightened to ratings below 2.0 (Appendix E.4). As discussed in Sec. 3.1 (L189–202), this criterion does not require statistical independence or causal irrelevance. We thus evaluate violations of a human-validated, task-specific invariance criterion, without claiming to establish causal counterfactual fairness or to assess fairness comprehensively.

[1] Kusner+, Counterfactual Fairness, NeurIPS2017.
[2] Zhao+, Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods, NAACL2018.
[3] Howard+, SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples, CVPR2024.
[4] Xiao+, GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing, IJCV2025.

W2. Subjective questions and fairness evaluation

Auditing a model's response to an income question does not assume that income can or should be inferred from a photograph. For subjective questions such as income or trustworthiness, a uniquely verifiable answer cannot be established from an image alone, and our benchmark does not prescribe a desirable response to each question. Counterfactual fairness assesses whether predictions remain invariant under specified counterfactual changes, independently of whether either prediction is correct [1]. We therefore adopt a ground-truth-free evaluation that measures consistency across counterfactual image pairs, without judging individual answers as correct or refusals as incorrect (Sec. 3.3, L249–250). Symmetric refusals count as consistent, whereas asymmetric refusals contribute to inconsistency. Our refusal-aware metrics further distinguish refusal-driven consistency from stable answering.

The distinction from general robustness evaluation lies in the attributes being changed. Rather than testing arbitrary visual perturbations, we examine changes to attributes that human annotators judge inappropriate as evidence for the specific question (Sec. 3.3, Step 2). This makes our evaluation a task-specific audit of fairness-related sensitivity.

W3 and Q2. Synthetic image confounding

Natural image pairs that differ only in a targeted fine-grained attribute are difficult to obtain at scale. As in prior image-based counterfactual studies [3,4], we therefore use synthetic editing to construct such pairs. We also acknowledge that image generation can introduce unintended changes (Sec. 3.3, L241–244). To minimize this risk, we manually inspect all pairs and employ automatic diagnostics assessing attribute realization and common-content preservation (Sec. 3.3, Step 3 and Appendix B).

Furthermore, Appendix E.5 examines the demographic–non-demographic gap across image-fidelity strata. The gap persists across all strata, including the highest-fidelity subsets, suggesting that the finding is not solely driven by lower-fidelity pairs. While these checks cannot guarantee the absence of all confounding, they provide evidence that the observed pattern is not limited to pairs with substantial editing discrepancies.

AuthorsOpenReview ↗

Response to Reviewer A8k1 (2/2)

W4. Statistical significance of the consistency gap

To assess the reliability of the reported gap, we additionally computed pair-level stratified bootstrap confidence intervals using 10,000 resamples. The model-average demographic–non-demographic consistency gap is +6.4 points (95% CI: [4.5, 8.1]). All 11 models show positive gaps, with 95% CIs excluding zero for 10 models, including all nine proprietary models. The only exception is LLaVA-OneVision-1.5-8B (95% CI: [−0.5, 6.5]). These results provide additional statistical support for the observed category gap. We have added the corresponding model-average 95% CI to Finding 1 (highlighted in magenta).

We also examined sensitivity to answer-option order in Appendix E.6. Comparing within-image and cross-image agreement under matched option-order combinations, we observe a larger agreement drop for non-demographic than demographic changes (10.1 vs. 5.3 points; demographic minus non-demographic: −4.8 points, 95% CI: [−6.2, −3.5]), indicating greater sensitivity to non-demographic visual changes relative to within-image response variability. Although this analysis does not test question or instruction rewording, the demographic–non-demographic pattern persists under matched option-order variation.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer A8k1,

Thank you again for your time and constructive feedback on our submission. We have carefully considered your comments and addressed them in our rebuttal.

We understand that you may be busy, but we would appreciate it if you could review our response before the discussion period ends. If any points remain unclear, we would be happy to clarify them.

Best regards,
The Authors

Reviewer_2RRWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at whether vision-language models change their judgments when visual details that should not matter are changed, such as makeup or the background. The authors build a benchmark of paired images to test this across five categories. They find that non-demographic changes often affect model decisions more than changes in gender or other demographic attributes. They also show that higher consistency can mainly come from refusing to answer both versions, so it does not necessarily mean the model is making better or fairer judgments.

Critical strengths
  1. The paper broadens the scope of fairness evaluation. It looks beyond gender and race to study appearance, behavior, culture, and environment, bringing attention to factors that existing evaluations can overlook.

  2. Paired images and human judgments about attribute relevance provide a practical way to study unwanted sensitivity, even when reliable ground-truth answers are unavailable.

  3. It separates giving consistent answers from consistently refusing to answer, showing why a high consistency score alone can be misleading.

  4. The experiments support a clear empirical story. The evaluation across 11 models shows a recurring demographic versus non-demographic gap, while the prompting experiments demonstrate how apparent improvements can mainly come from increased refusal.

Critical weaknesses
  1. Many questions ask models to judge income or trustworthiness from a photo, where refusing may be the most reasonable response. A model can also give the same unsupported answer twice, so consistency alone does not establish good judgment.

  2. The demographic versus non-demographic comparison could be better controlled. Differences in questions, scenes, or editing difficulty could contribute to the reported gap. Comparing different attribute changes using the same question and base image would make the main finding more convincing.

  3. The image edits may change more than the intended attribute. The quality checks help, but small changes in expression, pose, or lighting could still affect model responses. This matters especially if some attribute categories require more substantial edits than others.

  4. What counts as irrelevant is somewhat subjective. This is particularly tricky for behavior and culture, where people may disagree about what should influence a judgment. The retention threshold includes average ratings at the scale midpoint, so retained examples do not necessarily reflect strong agreement that the attribute is irrelevant.

Questions for authors

Could you compare demographic and non-demographic changes using the same base images and question targets?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 2RRW

Dear Reviewer 2RRW,

Thank you for the constructive review and for recognizing the value of our benchmark and refusal-aware evaluation.

W1. Consistency under subjective questions

Questions about income or trustworthiness do not have a uniquely verifiable answer from an image alone, and our benchmark does not prescribe a desirable response to each question. Accordingly, our ground-truth-free evaluation assesses decision stability under human-validated irrelevant visual changes, without judging individual answers as correct or refusals as incorrect. Our refusal-aware metrics further distinguish stable answering from symmetric refusal.

We also agree that a model may consistently produce unsupported answers. As explicitly acknowledged in Sec. 3.1 (L187–188), satisfying counterfactual invariance alone does not establish fairness in a broader sense. We therefore interpret consistency as a measure of one fairness-related property, not as evidence that individual answers are correct or appropriate.

W2 and Q1. Controlled category comparison

To examine whether question composition explains the observed gap, we grouped the benchmark questions into 15 question-target types and standardized the demographic and non-demographic groups to a common target distribution. The model-average gap remains +6.5 points after adjustment, compared with +6.4 points before adjustment. For example, the gaps within income/SES, education, and competence are +5.0 (positive for all 11 models), +6.0 (10/11 models), and +6.4 points (10/11 models), respectively.

We also conducted a controlled experiment using 60 base images, each independently edited to change either a demographic or a non-demographic attribute, yielding 120 counterfactual pairs in total. For each base image, both counterfactual pairs were evaluated using the same question and answer options. Non-demographic edits were balanced across the four categories, with questions covering the three targets above. The gap remains positive for both GPT-5 (+4.4 points) and Claude Sonnet 5 (+7.2 points), averaging +5.8 points (95% CI: [0.6, 11.4]).

These results strengthen our main finding while controlling for differences in base images and questions. Both analyses have been added to Appendix E.7 (highlighted in magenta).

W3. Counterfactual image fidelity

As in prior image-based counterfactual studies [1,2], we use synthetic images to obtain controlled attribute variations at scale. We also acknowledge that such edits can introduce unintended changes (Sec. 3.3, L243–244). To minimize this risk, we manually inspect all pairs and employ automatic diagnostics assessing attribute realization and common-content preservation (Sec. 3.3 Step 3 and Appendix B). The demographic–non-demographic gap persists across all fidelity strata, including the highest-fidelity subsets (Appendix E.5), suggesting that the finding is not solely driven by lower-fidelity pairs.

[1] Howard+, SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples, CVPR2024.
[2] Xiao+, GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing, IJCV2025.

W4. Borderline irrelevance ratings

To assess the impact of borderline irrelevance ratings, Appendix E.4 provides a threshold sensitivity analysis. The demographic–non-demographic gap remains between +5.4 and +6.4 points as the retention threshold is progressively tightened to mean ratings below 2.0, indicating that our main finding is not solely driven by borderline judgments.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer 2RRW,

Thank you again for your time and constructive feedback on our submission. We have carefully considered your comments and addressed them in our rebuttal.

We understand that you may be busy, but we would appreciate it if you could review our response before the discussion period ends. If any points remain unclear, we would be happy to clarify them.

Best regards,
The Authors

Thank you for your comments, I would like to maintain the score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Reviews and Rebuttal

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their thoughtful feedback and constructive suggestions. We would like to briefly summarize the strengths highlighted in the reviews and how we addressed the main concerns raised.

1. Strengths Highlighted in the Reviews

  • Broader scope of fairness evaluation: Reviewers recognized the value of extending LVLM fairness audits beyond conventional demographic attributes to a wider range of contextual factors.

  • Clear empirical findings: Reviewer 2RRW highlighted the clear empirical story, including the demographic–non-demographic consistency gap observed across models.

  • Refusal-aware evaluation: Reviewers 2RRW and wuhG appreciated the distinction between decision consistency and symmetric refusal, particularly the finding that higher consistency does not necessarily indicate more stable substantive answers. Reviewer A8k1 also recognized the value of the refusal-aware metrics.

2. Summary of Rebuttal

We have carefully addressed the reviewers' concerns through additional experiments, analyses, and manuscript revisions (highlighted in magenta). We summarize the key points below.

  • Controlled category comparison (2RRW, wuhG): We conducted two additional analyses, including question-target standardization and a controlled experiment using the same base images and questions. Both results support the original demographic–non-demographic consistency gap, even after controlling for these differences.

  • Statistical reliability (A8k1): We added bootstrap confidence intervals, confirming that the model-average demographic–non-demographic gap is positive with a 95% CI excluding zero. We also highlighted the within-image variability analysis already included in the original manuscript (Appendix E.6), providing further evidence that the observed category difference is not simply due to general response instability.

  • Matched human–model comparison (wuhG): We conducted a matched comparison on the same evaluation subset with confidence intervals, confirming a larger demographic–non-demographic gap in models than in humans. We also revised the manuscript to avoid implying a statistically established opposite trend in human responses.

  • Fairness interpretation (A8k1, wuhG): We clarified that our evaluation builds on counterfactual invariance, which has a well-established role in fairness research. We explained that our use of task-specific human judgments of irrelevance distinguishes our approach from general robustness evaluation, and that statistical associations alone do not determine whether a cue is appropriate evidence for individual-level judgments. We further clarified that our ground-truth-free evaluation does not judge individual answers as correct or refusals as incorrect. Instead, consistency captures one fairness-related property without establishing answer correctness or comprehensive fairness.

  • Image fidelity (2RRW, A8k1): We explained the difficulty of obtaining natural image pairs with controlled fine-grained attribute changes at scale, which motivates the use of synthetic images as in prior counterfactual studies. We also highlighted our manual inspection and automatic fidelity diagnostics. The fidelity-stratified analysis further suggests that the observed gap is not solely driven by substantial editing discrepancies.

  • Irrelevance validation (2RRW): We highlighted the threshold sensitivity analysis already included in the original manuscript (Appendix E.4), which shows that the demographic–non-demographic gap persists under stricter irrelevance criteria and is not solely driven by borderline judgments.

We believe these additional analyses and clarifications substantially strengthen the empirical evidence and address the main concerns raised in the reviews. We sincerely appreciate your continued consideration of our work.

Best regards,
The Authors

AuthorsOpenReview ↗

Final Update Following Reviewer Discussion

Dear Area Chair and Reviewers,

We sincerely appreciate the reviewers' follow-up responses. Reviewer wuhG confirmed that our additional experiments directly addressed their concerns and maintained their positive rating with increased confidence. Reviewer 2RRW also maintained their positive rating.

Several of Reviewer A8k1's concerns overlap with those raised by the other reviewers. We also provided new statistical evidence supporting our main finding, addressing their specific concern about statistical reliability. We hope these points will be taken into account in the final assessment.

Thank you again for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.