acceptodds
Under review as a conference paper at ICLR 2027

EAGLE: Grounding Multimodal LLM Judges for AI-Generated Image Assessment via Evidence Anchors

Abstract

Multimodal large language models (MLLMs) provide a promising foundation for AI-generated image assessment, yet existing MLLM-based evaluators typically derive assessment scores from holistic multimodal representations. This overlooks a fundamental distinction: different judgments require different evidence dependencies, e.g., text-image semantic correspondence depends on cross-modal evidence, whereas perceptual quality or authenticity relies primarily on visual evidence. To adaptively characterize these distinct evidence dependencies across different assessment tasks, we introduce evidence anchors that explicitly ground different assessment judgments in their relevant evidence. Building on this idea, we propose EAGLE (Evidence-Anchored Grounding for Multimodal LLM Evaluation), an MLLM judging framework that couples holistic MLLM scoring with anchor-guided evidence decomposition. EAGLE learns a set of latent queries divided into shared and anchor-specific groups: shared queries capture evidence reusable across judgments, while anchor-specific queries adaptively extract visual or cross-modal evidence to ground their corresponding judgments. This design preserves the broad assessment capability of pretrained MLLMs while providing each judgment with evidence tailored to its underlying dependency. Extensive experiments on popular benchmarks show that EAGLE achieves substantially improved within-dataset and cross-dataset generalization, validating the importance of explicitly grounding MLLM judges in their underlying evidence dependencies.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
MLLM-as-Judge, AIGC Image Assessment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_sY5NOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EAGLE, an MLLM-based evaluator for AI-generated images. It introduces cross-modal and visual evidence anchors and uses shared and anchor-specific latent queries to extract evidence for different assessment criteria. Experiments show strong within-dataset and cross-dataset performance across several image assessment tasks.

Critical strengths

The paper provides a clear and practical framework for modeling task-dependent evidence in MLLM judges. The empirical results are strong, particularly in cross-dataset evaluation, and the ablations support the main architectural choices.

Critical weaknesses

The two evidence-anchor categories are manually defined and may oversimplify assessment criteria that depend on mixed evidence. In addition, experiments use only one MLLM backbone, making it unclear how well the proposed design generalizes across different MLLM architectures.

Questions for authors

How sensitive is EAGLE to the choice of MLLM backbone? Have the authors tested whether the same evidence-anchor structure transfers to another MLLM family?

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer sY5N

We sincerely appreciate your positive comments and insightful suggestions. Below, we provide our point-by-point responses to each of your comments.

W(1): The two evidence-anchor categories are manually defined and may oversimplify assessment criteria that depend on mixed evidence.

Response: We agree that some assessment criteria may depend on multiple types of evidence. The cross-modal and visual anchors are therefore not intended to form an exhaustive or mutually exclusive taxonomy, but rather to represent two common forms of dominant evidence dependence in image assessment.

In EAGLE, shared queries allow information to be reused across criteria, while anchor-specific queries emphasize the evidence most relevant to each judgment. This design can therefore accommodate mixed evidence while still providing a structured inductive bias for criterion-specific grounding, rather than enforcing a hard separation between evidence types. We will clarify this scope in the revised manuscript.

W(2): In addition, experiments use only one MLLM backbone, making it unclear how well the proposed design generalizes across different MLLM architectures.

Q: How sensitive is EAGLE to the choice of MLLM backbone? Have the authors tested whether the same evidence-anchor structure transfers to another MLLM family?

Response: EAGLE is not highly sensitive to the choice of MLLM backbone. To verify this, we instantiate the same evidence-anchor structure on InternVL3-8B without changing the overall training design.

Table: Results are reported as (SRCC, PLCC).

SettingQwen2.5-VL-7B (Zero-shot)InternVL3-8B (Zero-shot)EAGLE (Qwen2.5-VL-7B)EAGLE (InternVL3-8B)
AGIQA-3K Corr.(68.7, 65.7)(71.1, 71.5)(87.1, 91.5)(86.8, 92.0)
AGIQA-3K Quality(71.4, 69.3)(61.3, 66.6)(90.4, 93.3)(91.9, 94.1)
AIGCIQA2023 Corr.(67.0, 71.3)(71.8, 64.5)(85.2, 84.6)(86.2, 85.4)
AIGCIQA2023 Quality(64.1, 71.6)(65.8, 59.5)(88.2, 89.9)(90.7, 90.2)
AIGCIQA2023 → AGIQA-3K Corr.(68.7, 65.7)(71.1, 71.5)(81.7, 86.3)(82.7, 86.9)
AIGCIQA2023 → AGIQA-3K Quality(71.4, 69.3)(61.3, 66.6)(82.0, 86.9)(81.2, 87.0)
AGIQA-3K → AIGCIQA2023 Corr.(67.0, 71.3)(71.8, 64.5)(80.6, 79.6)(80.4, 78.9)
AGIQA-3K → AIGCIQA2023 Quality(64.1, 71.6)(65.8, 59.5)(81.8, 82.0)(81.8, 81.4)

Under standard evaluation, EAGLE with InternVL3-8B performs comparably to or better than the Qwen2.5-VL-7B version on most metrics. Under cross-dataset evaluation, the two backbones remain closely comparable, with only small variations across settings. Notably, although Qwen2.5-VL-7B and InternVL3-8B exhibit substantially different zero-shot behavior, applying EAGLE brings both backbones to a similar strong performance level.

These results show that the proposed evidence-anchor design is not tied to Qwen2.5-VL and can be transferred effectively to a different MLLM family. We will include this additional backbone evaluation in the revised manuscript.

Thank your comment. I keep my score.

Thank you for your positive assessment and continued support of our work. We sincerely appreciate your time and constructive feedback.

Reviewer_1tRAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes EAGLE, an evaluator for AI-generated images based on Qwen2.5-VL-7B-Instruct. Its motivation is that assessment criteria depend on different evidence: text–image correspondence requires cross-modal relationships, whereas quality and authenticity primarily require visual evidence. EAGLE combines a holistic score-token prediction branch with an evidence-decomposition branch containing shared, cross-modal, and visual-specific latent queries. Criterion-dependent query activation and attention pathways retrieve evidence, and learned gates aggregate query-level scores. Training combines soft score-distribution supervision, decomposition score supervision, ranking loss, and query-diversity regularization; inference averages the two branch predictions. Evaluation spans six datasets and nine dataset–criterion settings, with transfer experiments between AGIQA-3K and AIGCIQA2023. The strongest results are in correspondence assessment and cross-dataset transfer. Ablations examine the branches, query structure, interaction pathways, losses, and joint-criterion training; additional analyses report input sensitivity and computational cost.

Critical strengths
  1. The cross-dataset results are strong and hold against a trained same-backbone baseline. EAGLE has the best SRCC and PLCC in every column of Table 2, which covers both transfer directions and both criteria. The more informative comparison is the LoRA-adapted holistic baseline in Table 5, which shares the frozen backbone and the supervision; EAGLE beats it on both criteria in both transfer directions. This supports an improvement in transfer between the two datasets tested.
  2. The ablations cover components, losses, and query structure. Table 5 separates the two branches, removes anchor interaction, removes structured queries, and trains criteria individually; Table 3 separates the ranking and diversity losses. Appendix C.1 reports trainable parameters and per-sample training and inference time, so the cost of the added module is visible.
Critical weaknesses
  1. The paper does not show that the anchors cause each criterion to rely on its intended evidence.
  • Table 4 perturbs only EAGLE; no trained baseline or ablated variant is tested under the same interventions.
  • Shared queries interact with text even for visual criteria, and Equations 6 and 7 read final-layer MLLM states, so evidence sources are not strictly separated.
  • Figure 4 is a qualitative example, and Table 5 shows predictive gains rather than which evidence the scores rely on.
  1. Run-to-run variance is unreported, and the cross-dataset gain stated in Section 4.2 does not match Table 2.
  • Tables 1-5 report no run-to-run variance, although several margins are 0.1-0.2 points (e.g., HPDv2 SRCC 85.3 vs. 85.1).
  • Section 4.2 reports gains of "up to 7.4 SRCC and 8.8 PLCC" over M3-AGIQA, but Table 2 gives maxima of 8.1 and 11.5.
Questions for authors
  1. If the Table 5 LoRA baseline receives the Table 4 interventions with the same inputs and instructions, is its criterion-specific pattern weaker than EAGLE's?
  2. How large is run-to-run variation relative to the 0.1-0.2-point margins in Tables 1-5?
  3. Do the AGIQA-3K and AIGCIQA2023 baseline numbers taken from the original works (Section 4.1) use the same test splits as EAGLE?
  4. Which Table 2 entries produce the "up to 7.4 SRCC and 8.8 PLCC" gains reported in Section 4.2?
  5. What values are used for σ (Equation 3), δ (Equation 12), and the similarity function in Equation 13?
4 comments
AuthorsOpenReview ↗

Author Response to Reviewer 1tRA(1/2)

We thank Reviewer 1tRA for the constructive feedback and insightful suggestions. Below, we address the main concerns and provide additional results where needed.

W1(1): Table 4 perturbs only EAGLE; no trained baseline or ablated variant is tested under the same interventions.

W1(3): Figure 4 is a qualitative example, and Table 5 shows predictive gains rather than which evidence the scores rely on.

Q1: If the Table 5 LoRA baseline receives the Table 4 interventions with the same inputs and instructions, is its criterion-specific pattern weaker than EAGLE's?

Response: Thanks for your valuable comments. We extend the intervention study to the LoRA baseline and the two structural ablations using identical inputs and instructions.

The answer to Q1 is yes. Full EAGLE exhibits stronger criterion-specific selectivity than the holistic LoRA baseline across both textual and visual interventions. The additional structural comparisons further show that the observed evidence-dependence pattern is closely related to the proposed evidence-aware design.

Table 5 establishes the contribution of the proposed components to predictive performance. The new controlled intervention results further show how the proposed design affects the evidence dependence of different judgments. Please refer to our response to Reviewer DdSe, W4 for the complete quantitative results and analysis.

W1(2): Shared queries interact with text even for visual criteria, and Equations 6 and 7 read final-layer MLLM states, so evidence sources are not strictly separated.

Response: We would like to clarify that our evidence anchors are not intended to impose a strict separation between modalities. As described in the submitted paper, the visual evidence anchor characterizes judgments that rely primarily on information within the visual modality, whereas the cross-modal anchor characterizes judgments that depend on the relationship between the conditioning prompt and the generated image.

This is also reflected in our design. For a visual criterion, EAGLE activates both the shared queries and the visual-specific queries. The shared queries capture evidence reusable across judgments and may interact with textual representations, while the visual-specific queries bypass the textual interaction and directly enter the visual pathway. In addition, (TτT^\tau) in Eq. 6 and (VτV^\tau) in Eq. 7 are extracted from the task-conditioned final-layer MLLM representation and are therefore not intended to be strictly modality-pure.

Thus, the proposed evidence decomposition is designed to emphasize different evidence dependencies, rather than enforce complete modality isolation. We will clarify this distinction in the revised manuscript.

W2(1): Tables 1-5 report no run-to-run variance, although several margins are 0.1-0.2 points (e.g., HPDv2 SRCC 85.3 vs. 85.1).

Q2: How large is run-to-run variation relative to the 0.1-0.2-point margins in Tables 1-5?

Response: Thank you for this insightful comment. We additionally repeat EAGLE with five random seeds and report the mean and standard deviation for representative standard evaluation and cross-dataset generalization settings.

SettingReported5-run Mean ± Std
Standard Evaluation (SRCC, PLCC)
AGIQA-3K Corr.(87.1, 91.5)(87.5 ± 0.2, 91.6 ± 0.1)
AGIQA-3K Quality(90.4, 93.3)(90.6 ± 0.2, 93.3 ± 0.1)
AIGCIQA2023 Corr.(85.2, 84.6)(85.1 ± 0.3, 84.5 ± 0.2)
AIGCIQA2023 Quality(88.2, 89.9)(88.6 ± 0.4, 90.0 ± 0.2)
Cross-Dataset Generalization (SRCC, PLCC)
AGIQA-3K → AIGCIQA2023 Corr.(80.6, 79.6)(80.4 ± 0.2, 79.8 ± 0.2)
AGIQA-3K → AIGCIQA2023 Quality(81.8, 82.0)(81.7 ± 0.3, 82.3 ± 0.4)
AIGCIQA2023 → AGIQA-3K Corr.(81.7, 86.3)(81.6 ± 0.3, 86.3 ± 0.2)
AIGCIQA2023 → AGIQA-3K Quality(82.0, 86.9)(82.2 ± 0.5, 87.1 ± 0.3)

The run-to-run standard deviation is approximately 0.1–0.5 points. Therefore, individual differences of 0.1–0.2 points can be comparable to stochastic variation and should be interpreted cautiously. However, EAGLE's main improvements, particularly under cross-dataset generalization, are substantially larger than the observed variation, indicating that the main performance gains are stable across random seeds rather than driven by stochastic variation. The originally reported results are also consistent with the five-run averages, with no systematic tendency to overestimate performance. We will report mean ± standard deviation across multiple runs for all methods in the revised manuscript.

AuthorsOpenReview ↗

Author Response to Reviewer 1tRA(2/2)

W2(2): Section 4.2 reports gains of "up to 7.4 SRCC and 8.8 PLCC" over M3-AGIQA, but Table 2 gives maxima of 8.1 and 11.5.

Q4: Which Table 2 entries produce the "up to 7.4 SRCC and 8.8 PLCC" gains reported in Section 4.2?

Response: Thank you for carefully checking this point. The values in Sec. 4.2 correspond to an earlier version of our experiments and were not updated to reflect the final results reported in Table 2. Based on the current table, the maximum improvement over M3-AGIQA is 8.1 SRCC points for AIGCIQA2023 → AGIQA-3K correspondence (81.7 vs. 73.6), and 11.5 PLCC points for AIGCIQA2023 → AGIQA-3K quality (86.9 vs. 75.4). We will update Sec. 4.2 accordingly in the revised manuscript.

Q3: Do the AGIQA-3K and AIGCIQA2023 baseline numbers taken from the original works (Section 4.1) use the same test splits as EAGLE?

Response: Yes. Our experiments on AGIQA-3K and AIGCIQA2023 follow the exact data partitions used by DPGF-Net. For the baselines we reproduce on these datasets, including DeQA-Score and Qwen2.5-VL-7B, we evaluate them on the same splits as EAGLE.

For the remaining baselines, we report the results from DPGF-Net, which evaluates these methods on the same dataset partitions using official implementations or the authors' released code, with separate optimization for quality and correspondence. We thank the reviewer for bringing our attention to an imprecise statement in Sec. 4.1. The current manuscript states that these results are taken from the corresponding original works, which does not accurately describe our evaluation protocol, and we will correct this in the revision. M3-AGIQA is the only exception, for which we report the officially published results because the required MLLM-generated intermediate data are not publicly available.

In addition, for the other datasets, we construct fixed train/test splits and reproduce all accessible baselines on exactly the same splits as EAGLE. We will clarify the evaluation protocol in the revised manuscript and release our code, evaluation scripts, and dataset split files for reproducibility.

Q5: What values are used for σ (Equation 3), δ (Equation 12), and the similarity function in Equation 13?

Response: Thank you for noting this missing detail. We use σ=0.5\sigma=0.5 in Eq. (3), δ=0.2\delta=0.2 in Eq. (12), and cosine similarity in Eq. (13). We appreciate the reviewer for bringing this to our attention and will explicitly include these settings in the revised manuscript.

Five runs do help with the cross-dataset worry, but some margins in Table 1 remain too small to be convincing. I will keep my score.

AuthorsOpenReview ↗

Author Response to Reviewer 1tRA – On Performance Margins

Thank you for the follow-up. We compare EAGLE with the two strongest representative competitors, M3-AGIQA and DPGF-Net.

Comparison with M3-AGIQA. We would like to emphasize that M3-AGIQA operates under a different information setting from EAGLE. It relies on external online MLLMs to generate intermediate image descriptions and distills this capability into a local model for multi-round aspect-specific evaluation. In contrast, EAGLE does not use external MLLM-generated intermediate information, but explicitly models the evidence dependencies of different judgments through shared and anchor-specific queries.

Without the additional information available to M3-AGIQA, EAGLE remains highly competitive under standard evaluation, showing that the proposed evidence-anchor design can achieve comparable or better performance. The difference becomes more pronounced under cross-dataset evaluation, where EAGLE consistently outperforms M3-AGIQA on both correspondence and quality in both transfer directions, with gains of up to 8.1 SRCC and 11.5 PLCC points. One possible explanation is that description-based intermediate information may introduce additional source-domain dependence, weakening the transferability of M3-AGIQA on the target datasets. The substantially stronger transfer results further suggest that EAGLE's evidence-anchor formulation provides a more robust basis for cross-dataset generalization.

Comparison with DPGF-Net. DPGF-Net follows the same information setting as EAGLE and therefore provides a more direct comparison. EAGLE achieves clear improvements on several representative tasks, including correspondence assessment on AGIQA-3K and AIGCIQA2023, as well as CompBench, GenAI-Bench, and AGIN. Across these five tasks, EAGLE improves over DPGF-Net by an average of approximately 4.9 SRCC and 4.8 PLCC points, with the largest gains reaching 9.0 SRCC and 9.3 PLCC points.

Standard Evaluation (SRCC, PLCC)

MethodAGIQA-3KAIGCIQA2023CompBenchGenAI-BenchAGINHPDv2
Corr.QualityCorr.QualityAuthenticityCorr.Corr.NaturalnessPreference
M3-AGIQA(85.2, 91.4)(90.5, 93.2)(80.6, 79.7)(86.2, 89.2)(82.8, 81.7)––––
DPGF-Net(80.1, 87.0)(90.1, 92.9)(82.8, 81.2)(88.1, 89.8)(80.9, 79.8)(84.1, 84.1)(74.4, 74.2)(81.6, 82.5)(85.1, 85.1)
EAGLE(87.1, 91.5)(90.4, 93.3)(85.2, 84.6)(88.2, 89.9)(80.5, 80.4)(88.7, 88.6)(83.4, 83.5)(83.3, 84.6)(85.3, 85.5)

Cross-Dataset Generalization (SRCC, PLCC)

MethodAGIQA-3K → AIGCIQA2023AIGCIQA2023 → AGIQA-3K
Corr.QualityCorr.Quality
DPGF-Net(62.3, 61.4)(76.8, 78.4)(65.4, 73.3)(77.1, 82.4)
M3-AGIQA(74.2, 72.5)(74.9, 74.6)(73.6, 77.2)(74.3, 75.4)
EAGLE(80.6, 79.6)(81.8, 82.0)(81.7, 86.3)(82.0, 86.9)

Taken together, these results show that EAGLE's empirical gains are not limited to a few close entries in Table 1. Under the same information setting, EAGLE achieves substantial improvements across multiple tasks, while under cross-dataset evaluation it consistently outperforms M3-AGIQA by large margins. We believe these results provide convincing empirical support for the effectiveness and generalization of EAGLE.

Reviewer_DdSeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes EAGLE, an MLLM-based framework for AI-generated image assessment. The central idea is that different assessment criteria require different types of evidence. To capture these differences, the authors introduce evidence anchors and a decomposition framework consisting of shared and anchor-specific learnable queries. These queries extract relevant information through cross-attention, and their predictions are combined with holistic MLLM scoring. Experiments across several image assessment benchmarks demonstrate improved performance in both within-dataset evaluation and cross-dataset generalization.

Critical strengths
  1. Interesting motivation. The paper identifies a reasonable distinction between the evidence required for different image assessment tasks. Explicitly considering these evidence dependencies is an interesting direction for improving the reliability of MLLM-based judges.

  2. Clear presentation. The paper is generally well organized, and the proposed architecture and experimental setup are clearly explained.

Critical weaknesses
  1. Limited practical applicability due to excessive methodological complexity. I am not convinced this is a promising direction for practical MLLM-as-a-Judge research. The proposed method introduces multiple trainable components, including shared and anchor-specific queries, cross-attention modules, scoring heads, gating networks, and several training objectives. Such a complex framework requires substantial training and hyperparameter tuning efforts. In the experiments, the authors use thousands of labeled training examples. However, in many real-world MLLM-as-a-Judge applications, the situation is often the opposite: task-specific human annotations are scarce, and users cannot afford repeated training and extensive hyperparameter searches. In such settings, simpler approaches such as prompt optimization or straightforward supervised fine-tuning may be more practical.

  2. Missing comparisons with simpler and more practical baselines. Although the paper compares against several existing image assessment methods and includes a LoRA-based holistic scoring baseline, I believe some important and intuitive alternatives are missing. In particular, the authors do not compare against stronger or larger MLLM judges, or prompt optimization methods. A judge with optimized instructions might achieve competitive performance without introducing additional trainable modules.

  3. Insufficient investigation of data efficiency. The proposed framework is evaluated primarily in supervised settings with thousands of training examples. However, one of the most important practical challenges in adapting MLLM judges is the limited availability of task-specific annotations. The paper does not investigate how the proposed method performs with substantially fewer labeled examples, such as 50, 100, or 500 samples. It remains unclear whether the proposed evidence decomposition framework is effective in low-resource settings or whether its advantages depend on relatively large amounts of supervised data.

  4. Unclear necessity of the proposed evidence decomposition. The method combines several established components, including learnable queries, cross-attention, task-specific scoring, gating, and auxiliary losses. Although the ablation studies show that these components can improve performance, it remains unclear whether the improvements specifically result from meaningful evidence grounding or simply from increased model capacity and additional supervised optimization. More direct comparisons with simpler architectures would help establish the necessity of the proposed design.

Questions for authors

See above.

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer DdSe(1/4)

We thank Reviewer DdSe for the careful evaluation of our work and the constructive feedback. Below, we address the concerns in detail and provide additional clarifications and experimental results where appropriate.

W1: Limited practical applicability due to excessive methodological complexity.

Response: We thank the reviewer for raising the practical considerations of training cost, hyperparameter tuning, and annotation efficiency. We clarify these points below.

Method complexity and training cost. EAGLE follows a unified design in which the additional components are organized by the proposed evidence-anchor principle and optimized jointly in a single end-to-end procedure. The pretrained MLLM backbone remains largely frozen, with adaptation performed through LoRA together with the evidence-decomposition modules. In the submitted manuscript, Appendix C.1 already provides a detailed comparison of trainable parameters, per-GPU training time, and inference time with representative methods. The results show that EAGLE achieves a reasonable trade-off between assessment performance and computational cost.

Hyperparameter tuning. EAGLE does not rely on repeated dataset-specific hyperparameter searches. We use the same model architecture and optimization configuration across datasets without dataset-specific tuning.

Supervision and data efficiency. We agree that annotation scarcity is an important practical consideration for adapting MLLM judges. Our standard evaluation follows the supervised protocols commonly used by existing image-assessment benchmarks for fair comparison. Importantly, Table 2 of the main text already evaluates cross-dataset transfer, where a model trained on a source dataset is directly applied to unseen target datasets without any target-domain annotations or adaptation. EAGLE substantially outperforms competing methods in this setting, showing strong generalization beyond the training domain.

To further address the reviewer’s concern about limited source-domain supervision, we additionally evaluate EAGLE with only 50, 100, and 500 labeled images in W3, under both standard and cross-dataset settings. EAGLE already achieves clear gains over zero-shot evaluation with 50–100 labeled images and remains competitive with full-data supervised baselines in several settings, particularly for correspondence assessment and cross-dataset generalization.

Together, these results show that EAGLE remains effective both when target-domain annotations are unavailable and when source-domain supervision is limited.

Suggestion on simple alternatives. We agree that simpler adaptation strategies are important comparisons. Our holistic LoRA baseline represents straightforward supervised fine-tuning under the same backbone and training data. In our response to W2, we further compare EAGLE with automatic instruction optimization and a substantially larger 72B MLLM judge. The results show that prompt optimization improves the zero-shot judge in several settings, while scaling to 72B does not consistently improve performance. Importantly, neither approach nor their combination closes the gap to EAGLE.

AuthorsOpenReview ↗

Author Response to Reviewer DdSe(2/4)

W2: Missing comparisons with simpler and more practical baselines.

Response: To directly address this concern, we additionally evaluate two simpler alternatives: zero-shot judging with a substantially larger MLLM, and automatic instruction optimization. We use Qwen2.5-VL-7B-Instruct and Qwen2.5-VL-72B-Instruct-AWQ as the 7B and 72B zero-shot judges, respectively. For instruction optimization, we perform an iterative search inspired by OPRO [1], where candidate instructions are repeatedly proposed and selected according to their performance on a development subset drawn from the training split. Specifically, following [1], we use 300 MOS-stratified training images per dataset/criterion and run 12 rounds with 8 candidate instructions per round. We further apply the optimized instruction to the 72B judge to examine whether combining larger backbone capacity with prompt optimization can close the gap to EAGLE.

The results are as follows:

Table: Standard Evaluation (SRCC, PLCC).

Setting7B7B + Opt.72B72B + Opt.EAGLE (7B)
AGIQA-3K Corr.(68.7, 65.7)(74.0, 77.6)(64.9, 64.5)(74.2, 75.3)(87.1, 91.5)
AGIQA-3K Quality(71.4, 69.3)(72.2, 75.7)(70.8, 76.0)(72.7, 78.4)(90.4, 93.3)
AIGCIQA2023 Corr.(67.0, 71.3)(68.7, 63.9)(67.9, 62.5)(67.3, 61.8)(85.2, 84.6)
AIGCIQA2023 Quality(64.1, 71.6)(79.0, 75.9)(68.7, 68.0)(79.4, 80.7)(88.2, 89.9)

Table: Cross-Dataset Generalization (SRCC, PLCC)

Setting7B7B + Opt.72B72B + Opt.EAGLE (7B)
AIGCIQA2023 → AGIQA-3K Corr.(68.7, 65.7)(68.1, 67.7)(64.9, 64.5)(67.6, 66.4)(81.7, 86.3)
AIGCIQA2023 → AGIQA-3K Quality(71.4, 69.3)(77.7, 77.8)(70.8, 76.0)(74.5, 80.1)(82.0, 86.9)
AGIQA-3K → AIGCIQA2023 Corr.(67.0, 71.3)(67.7, 64.4)(67.9, 62.5)(67.4, 65.3)(80.6, 79.6)
AGIQA-3K → AIGCIQA2023 Quality(64.1, 71.6)(74.8, 72.4)(68.7, 68.0)(77.0, 76.5)(81.8, 82.0)

The results lead to three clear observations. First, prompt optimization helps the zero-shot 7B judge in some cases, but the improvements are inconsistent across criteria and remain well below EAGLE. Second, simply scaling the judge from 7B to 72B does not yield consistent gains; in several settings, the 72B zero-shot judge performs similarly to or worse than the 7B judge. Third, combining the 72B backbone with optimized instructions still does not close the gap to EAGLE in the standard setting.

The same conclusion holds for cross-dataset generalization. When the instruction optimized on the source dataset is directly transferred to an unseen target dataset without target-domain tuning, it improves some results but still remains clearly below EAGLE. Overall, these comparisons show that EAGLE's gains cannot be reproduced simply by stronger prompting, a larger MLLM backbone, or their combination.

[1] Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., & Chen, X. Large Language Models as Optimizers. ICLR 2024.

AuthorsOpenReview ↗

Author Response to Reviewer DdSe(3/4)

W3: Insufficient investigation of data efficiency.

Response: To directly investigate the low-resource setting raised by the reviewer, we additionally evaluate EAGLE using only 50, 100, and 500 labeled training images, together with the full training set, on AGIQA-3K and AIGCIQA2023. Each labeled image provides both correspondence and quality annotations. DeQA-Score and DPGF-Net are included as full-data supervised references.

Table: Standard Evaluation (SRCC, PLCC)

Method / # LabelsAGIQA-3K Corr.AGIQA-3K QualityAIGCIQA2023 Corr.AIGCIQA2023 Quality
EAGLE, 0(68.7, 65.7)(71.4, 69.3)(67.0, 71.3)(64.1, 71.6)
EAGLE, 50(76.3, 82.1)(78.4, 83.7)(73.4, 73.3)(81.2, 81.7)
EAGLE, 100(79.1, 84.6)(82.5, 87.1)(75.5, 75.3)(82.3, 83.4)
EAGLE, 500(80.8, 86.8)(86.0, 90.2)(76.4, 76.3)(84.5, 86.2)
EAGLE, Full(87.1, 91.5)(90.4, 93.3)(85.2, 84.6)(88.2, 89.9)
DeQA-Score [2] (Full)(69.9, 81.2)(90.3, 92.7)(70.7, 70.6)(86.8, 88.9)
DPGF-Net [3] (Full)(80.1, 87.0)(90.1, 92.9)(82.8, 81.2)(88.1, 89.8)

Table: Cross-Dataset Generalization (SRCC, PLCC)

Method / # LabelsAIGCIQA2023→AGIQA-3K Corr.AIGCIQA2023→AGIQA-3K QualityAGIQA-3K→AIGCIQA2023 Corr.AGIQA-3K→AIGCIQA2023 Quality
EAGLE, 0(68.7, 65.7)(71.4, 69.3)(67.0, 71.3)(64.1, 71.6)
EAGLE, 50(72.6, 73.6)(76.9, 81.4)(72.4, 71.8)(74.8, 73.9)
EAGLE, 100(75.5, 77.2)(77.0, 82.1)(73.9, 73.1)(78.3, 77.5)
EAGLE, 500(77.6, 81.1)(81.4, 86.2)(77.2, 75.5)(80.4, 80.2)
EAGLE, Full(81.7, 86.3)(82.0, 86.9)(80.6, 79.6)(81.8, 82.0)
DeQA-Score [2] (Full)(51.5, 58.2)(75.0, 79.4)(58.7, 59.2)(77.8, 79.3)
DPGF-Net [3] (Full)(65.4, 73.3)(77.1, 82.4)(62.3, 61.4)(76.8, 78.4)

EAGLE already improves clearly over the zero-shot baseline with only 50–100 labeled images, although performance naturally decreases relative to full-data training. Compared with full-data supervised baselines, low-resource EAGLE remains competitive, especially for correspondence assessment and cross-dataset generalization. With only 50 labeled source images, EAGLE already matches or outperforms DeQA-Score and DPGF-Net on correspondence in both transfer directions. Quality assessment is more sensitive to the amount of supervision, but also improves consistently as more labeled data are provided.

These results show that EAGLE can operate effectively with substantially fewer annotations, while its best performance still benefits from additional supervision.

[2] You, Z., Cai, X., Gu, J., Xue, T., & Dong, C. Teaching Large Language Models to Regress Accurate Image Quality Scores Using Score Distribution. CVPR 2025.

[3] Li, T., Liao, X., & Zhou, M. DPGF-Net: Dual-Prior Guided Fusion Network for Joint Assessment of Perceptual Quality and Semantic Consistency in AI-Generated Images. CVPR 2026.

AuthorsOpenReview ↗

Author Response to Reviewer DdSe(4/4)

W4: Unclear necessity of the proposed evidence decomposition.

Response: We further examine the necessity of the proposed design from two complementary aspects: whether the evidence-aware structure contributes to predictive performance, and whether it leads to criterion-specific dependence on the intended evidence sources. Our results show that both structured queries and anchor interaction are important for effective evidence organization, and that removing either component weakens both predictive performance and evidence selectivity.

1. The proposed structure is important for predictive performance.

Table 5 of the paper provides evidence structure-controlled ablations that retain the evidence-decomposition branch while removing key components of the proposed design. EAGLE$_D$ w/o Structured Queries replaces the shared/anchor-specific query organization with a single query set shared across criteria, while EAGLE$_D$ w/o Anchor Interaction removes anchor-dependent interaction. Both variants consistently underperform EAGLE under both standard and cross-dataset evaluation.

These results show that the evidence-decomposition branch alone is not sufficient. Structured queries help organize evidence according to different assessment criteria, while anchor interaction guides the acquisition of criterion-relevant evidence. Together, these components enable more effective criterion-specific evidence modeling.

2. Controlled interventions show that EAGLE exhibits stronger criterion-specific evidence dependence.

We further apply the same interventions to the LoRA baseline, EAGLE$_D$, and the two structural ablations, and compare them with full EAGLE using identical inputs and instructions. Prompt Swap replaces the conditioning prompt with an unrelated one, while Null Prompt removes the textual condition; both perturb textual semantic evidence while leaving the image unchanged. Gaussian Blur, Gaussian Noise, and JPEG Compression perturb visual evidence while keeping the conditioning prompt fixed. For prompt interventions, stronger criterion selectivity means a larger correspondence change relative to quality; for visual degradations, it means a larger quality change relative to correspondence.

Table: Each entry reports Correspondence (|Δ\Delta|) / Quality (|Δ\Delta|) on AGIQA-3K, , where (|Δ\Delta|) denotes the mean absolute change in prediction.

InterventionLoRA BaselineEAGLE$_D$EAGLE$_D$ w/o Anchor InteractionEAGLE$_D$ w/o Structured QueriesEAGLE
Prompt Interventions
Prompt Swap0.957 / 0.1530.810 / 0.0700.018 / 0.0190.154 / 0.1171.095 / 0.125
Null Prompt0.349 / 0.0570.273 / 0.0540.023 / 0.0160.111 / 0.0670.471 / 0.057
Visual Degradations
Gaussian Blur0.336 / 0.6290.305 / 0.4740.335 / 0.4800.431 / 0.5910.282 / 0.669
Gaussian Noise0.361 / 0.5160.431 / 0.5540.473 / 0.6420.445 / 0.6150.287 / 0.578
JPEG Compression0.197 / 0.3020.226 / 0.2870.241 / 0.3290.253 / 0.2920.188 / 0.325

Full EAGLE consistently exhibits stronger criterion-specific selectivity than the holistic LoRA baseline for both textual and visual interventions. In contrast, the structural ablations show the clearest effect on semantic textual evidence: removing either anchor interaction or structured query organization causes a substantial drop in selectivity. Under Prompt Swap, for example, the correspondence-to-quality selectivity ratio drops from 11.6 for EAGLE$_D$ to 1.0 without anchor interaction and 1.3 without structured queries.

Together, Table 5 and the controlled intervention results show that the proposed structure contributes not only to predictive performance, but also to how different judgments depend on different evidence sources.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Responses to Reviewers

Dear Reviewers and Area Chair,

We sincerely thank all three reviewers (DdSe, 1tRA, and sY5N) for their valuable time and constructive feedback on our paper. We appreciate the reviewers' recognition of the motivation behind criterion-dependent evidence modeling, the proposed evidence-anchor decomposition framework, and the strong empirical results, particularly in cross-dataset generalization.

In response to the reviewers' comments, we have carefully revised and strengthened the manuscript with additional experiments, analyses, and clarifications. The major updates are summarized below:

  • Additional comparisons with stronger and simpler MLLM judges (Reviewer DdSe): We introduce new baselines using Qwen2.5-VL-7B and Qwen2.5-VL-72B, with and without OPRO-inspired automatic instruction optimization. The results show that neither backbone scaling nor instruction optimization, nor their combination, consistently closes the performance gap to EAGLE. (Section 4.4, Table 7)

  • Low-resource evaluation (Reviewer DdSe): We conduct additional experiments using only 50, 100, and 500 labeled training images on AGIQA-3K and AIGCIQA2023, covering both standard evaluation and cross-dataset generalization. The results demonstrate that EAGLE achieves meaningful improvements with limited supervision and remains competitive with full-data supervised baselines, particularly in correspondence assessment and cross-dataset transfer. (Section 4.4, Table 6)

  • Criterion-Specific Evidence Sensitivity (Reviewer 1tRA): We further evaluate the holistic LoRA baseline, EAGLE$_D$, and the two structural ablations under the same Prompt Swap, Null Prompt, and visual interventions, using identical inputs and instructions. The results demonstrate stronger criterion-specific evidence sensitivity in EAGLE and show that removing anchor interaction or structured queries substantially weakens the selective response to textual evidence. These findings further support that the proposed evidence-anchor structure contributes not only to predictive performance but also to criterion-specific evidence utilization. (Appendix, Table 11)

  • Clarification of evidence-anchor design (Reviewers 1tRA and sY5N): We clarify that the proposed evidence anchors emphasize different evidence dependencies rather than enforce strict modality isolation. Shared queries may capture reusable cross-modal information, while anchor-specific queries prioritize evidence relevant to individual assessment criteria. (Section 3)

  • Generalization across MLLM backbones (Reviewer sY5N): We instantiate the same evidence-anchor structure on InternVL3-8B without changing the overall training design. The results demonstrate substantial improvements over the corresponding zero-shot baseline and performance comparable to EAGLE with Qwen2.5-VL-7B, supporting the transferability of the proposed design across MLLM families. (Section 4.4, Table 7)

  • Run-to-run variance analysis (Reviewer 1tRA): We conduct five-seed experiments for EAGLE on representative standard evaluation and cross-dataset generalization settings. The observed standard deviations range from 0.1 to 0.5 points, suggesting that small differences of 0.1–0.2 points should be interpreted cautiously, while the main performance improvements are substantially larger than the observed variation. We also plan to provide more comprehensive mean ± standard deviation statistics for all compared methods in the final version, subject to computational feasibility. (Appendix; further statistics planned for the final version)

  • Additional implementation and experimental clarifications (Reviewers DdSe and 1tRA): We clarify the unified end-to-end training design, the use of consistent optimization settings across datasets, and the specific hyperparameters and similarity function used in the method. We also correct the numerical inconsistency in the cross-dataset improvement reported in Section 4.2 and clarify the evaluation protocol and dataset splits. (Section 4.1 and 4.2)

All modifications included in the current revised manuscript are colored in blue for ease of inspection.

We believe these revisions substantially strengthen the empirical validation, methodological clarity, and reproducibility of EAGLE. We sincerely appreciate the reviewers' thoughtful suggestions, which have helped us improve the paper.

Thank you very much for your time and consideration.

Best regards,
Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.