acceptodds
Under review as a conference paper at ICLR 2027

PrismIQA: Capability-Factorized Multi-Agent Image Quality Assessment with Falsification and Region-to-Global Attribution

Abstract

Reasoning-capable vision–language models have substantially advanced image quality assessment (IQA), including high-resolution assessment through active crop-and-zoom operations. Nevertheless, most existing systems assign global scoring, defect localization, degradation discrimination, and perceptual-impact estimation to one general-purpose policy. Recent agentic systems alleviate part of this burden through workflow-level roles such as planner–executor–summarizer, but the planner still commonly performs broad perception and task allocation. We instead study capability-level factorization: each agent has a distinct objective, evidence space, and action set. Our framework, PrismIQA, comprises four stages: a Global Assessor estimates a calibrated holistic prior; a high-recall Region Proposer identifies candidate defect regions; a Falsification Agent compares each crop with full-image context and rejects pseudo-degradations such as intentional depth-of-field and natural texture; and an Impact Assessor estimates the severity and semantic importance of verified defects and attributes their residual effect on the global score. We analyze this architecture as a policy over semantic interaction trajectories. Through Lipschitz–transport analysis and a KL-chain diagonal constraint, we prove that the factorized four-agent system is superior to a shared model: four independently optimized models yield a strictly smaller deviation upper bound than one shared model. On seven IQA datasets, PrismIQA achieves SOTA performance and reaches 0.854 SRCC and 0.892 PLCC on Vista-Bench.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Eb2bOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PrismIQA, a no-reference IQA system for high-resolution images that splits assessment into four sequential stages: a Global Assessor, a Region Proposer, a Falsification Agent, and an Impact Assessor. The stages exchange a fixed structured state and are trained independently by GRPO or SFT followed by GRPO with no joint training. The paper argues, via a KL chain-rule decomposition and a diagonal-constraint argument, that four independently optimized models reach a lower class-optimal loss, and therefore a tighter reward-deviation bound. Empirically it reports SRCC/PLCC on seven datasets. There are also ablations to show the importance of each agent.

Critical strengths

The manuscript is generally well written, with clear and well-presented figures. The experiments follow the standard IQA evaluation protocol, including the main benchmark comparison and ablation studies.

Critical weaknesses

The paper devotes several pages to theoretical analysis, but the experimental comparison is not set up fairly.

  1. What is the training set? The paper does not state it. Sec. 5.1 only says that protocols “follow prior work”. Since the four-stage design closely mirrors how Vista-Bench is annotated, I suspect the models were trained on Vista-Bench or on data built with the same pipeline. The authors should state this explicitly, together with how overlap with the Vista-Bench test set was excluded.
  2. Is the comparison fair? For most baselines, they correspond to models trained on KADID-10K or KonIQ-10k and evaluated cross-dataset. The paper does not specify its training protocol, so it is impossible to judge whether the comparison is fair.
  3. What drives the improvement? PrismIQA uses four Qwen3-VL-8B models, but Table 1 includes only Qwen2.5-VL-7B as a base-model baseline, not Qwen3-VL-8B. The strongest MLLM-based baselines (Q-Insight, VisualQuality-R1, Q-Probe) are also built on Qwen2.5-VL-7B. It is therefore unclear whether the gains come from the stronger base model, from using four models instead of one, or simply from the training data.
Questions for authors

Is the theoretical analysis of training the four agents separately necessary? I am not an expert in this type of proof, but the main result appears to hold essentially by construction: four independent parameter vectors can always fit at least as well as one shared vector. This question would be answered more convincingly by an empirical comparison, just a single shared model trained on all four roles and run through the same pipeline. Appendix C.6 already trains such a model but reports only role losses. Reporting its SRCC/PLCC would directly test the claimed benefit of factorization. More generally, the benefit should be weighed against the practical cost of deploying four separate 8B models.

In my view, an image quality assessment paper can make a strong contribution through an insightful framework design and comprehensive experiments. The current theory adds length without strengthening the paper’s claims.

2 comments

1. Data Source and Training Setup

For every entry in Table 1, our data are drawn from VisualQuality-R1. Our model is trained on the training set of KADID-10K. This training setup is intended to demonstrate the model’s generalization ability, which is a common and accepted setting in IQA tasks.

2. Fair Comparison and Role-Specific Training

Our SFT trajectories are extracted from KADID-10K and partitioned into four role-specific subsets, so that each of the four agents learns its corresponding role. To address the reviewer’s concern about fairness, we additionally train a single Qwen3-VL-8B model on the same training data without role-specific extraction—i.e., a shared model trained jointly on all four roles. As shown below, this shared model underperforms both Q-Probe and our full model on Vista and average metrics, indicating that the gains are not merely due to the stronger base model or the training data alone; the role-specific factorization is important.

ModelVista SRCCAvg SRCCVista PLCCAvg PLCC
Single Qwen3-VL-8B (shared, no role extraction)0.7800.7900.8210.806
Q-Probe0.8010.8010.8500.810
FullPrismIQA (Ours)0.8540.8350.8920.841

Thanks for the response and additional results. However, my core concern remains unaddressed: the authors have not sufficiently justified why training four separate agents is necessary for this quality task.

So I will keep my rating as weak rejection now.

Reviewer_nBhUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an IQA agent with a capability-level decomposition that mitigates the potential interference in capabilities that arise when a single shared model performs different tasks by training four dedicated models separately. It supports this design with mathematical descriptions and controlled experiments.

Critical strengths
  1. The paper proposes using four separate specialized models to handle different tasks, reducing training issues in a single shared model caused by conflicting requirements to some extent.
  2. The paper considers the interference that local cropping regions themselves cause on the IQA model’s judgments and takes this into account in the design of the evaluation phase.
  3. It compares the method with similar evaluation approaches and demonstrates more accurate results.
Critical weaknesses
  1. The main paper and existing appendices do not adequately report the sources, scale, annotation processes, and partitioning of the training data for the four functional models, making it difficult to assess the training setup of these models.
  2. “A model trained primarily on defective crops…. ” This sentence requires a clearer description to demonstrate the motivation.
  3. Table 2 should be supplemented with experiments replacing a specific agent with a shared generalist model instead of directly removing it. This would help assess the importance of each functional model.
  4. Given that this paper introduces four models for evaluation, could the authors provide the average evaluation time compared to other methods to know the time cost of the evaluation?
Questions for authors

See weaknesses.

2 comments

1. Data Source and Training Setup

For every entry in Table 1, our data are drawn from VisualQuality-R1. Our model is trained on the training set of KADID-10K. This training setup is intended to demonstrate the model’s generalization ability, which is a common and accepted setting in IQA tasks.

2. Context-Aware Cropping

The paper points out that traditional visual zoom-in or region-level supervision usually benefits from the diversity of target objects: SFT trajectories only need to cover the target regions. In IQA, however, if the training data always focus on degraded regions, the model tends to overfit a spurious causal relationship that “cropping implies low quality” (Figure 1(d)). This further leads to a systematic low-score bias across all local views, a loss of content discriminability, and misinterpretation of natural depth-of-field blur as degradation. To address this, Q-Probe adopts context-aware cropping: the SFT trajectories mix genuine degradations, natural DoF, and pristine foregrounds, and the cropped regions are required to cover degraded patches while also preserving natural blur or clear areas. This severs the spurious correlation between “local cropping” and “low quality,” and enables the model to decide when and how to invoke tools based on both global and local context.

3. Replacing a Specific Agent with a Shared Generalist Model

We thank the reviewer for this constructive suggestion. To assess the importance of each functional model more rigorously, we supplemented the “remove-one-agent” ablation with experiments that replace a specific agent with a shared generalist model, Qwen3-VL-8B, while keeping the overall pipeline intact. As shown in Table X, the full model FullPrismIQA achieves 0.854 Vista SRCC and 0.836 Avg SRCC, outperforming all replacement variants: replacing FilterAgent, ImpactAssessor, GlobalAssessor, and RegionProposer with Qwen3-VL-8B yields 0.774/0.785, 0.794/0.818, 0.740/0.787, and 0.761/0.800, respectively. Although some replacement configurations improve slightly over direct removal (e.g., replacing ImpactAssessor raises Avg SRCC from 0.802 to 0.818), they remain clearly below the full model; replacing FilterAgent even lowers Avg SRCC below direct removal, and replacing GlobalAssessor or RegionProposer lowers Vista SRCC below direct removal. These results indicate that a shared generalist model cannot effectively substitute for the specialized functional models, further supporting the necessity of the proposed agent-specific design.

ConfigurationVista SRCCAvg SRCC
FullPrismIQA0.8540.836
w/o FilterAgent0.7630.789
Replace FilterAgent with Qwen3-VL-8B0.7740.785
w/o ImpactAssessor0.7810.802
Replace ImpactAssessor with Qwen3-VL-8B0.7940.818
w/o GlobalAssessor0.7450.776
Replace GlobalAssessor with Qwen3-VL-8B0.7400.787
w/o RegionProposer0.7690.794
Replace RegionProposer with Qwen3-VL-8B0.7610.800

4. Inference Time

Other 8B models require about 12.8 seconds to process one image, while our method takes 14.5 seconds. Although our method incurs a somewhat longer inference time, this overhead is acceptable given the performance gains.

Thank you for the clarifications and additional experiments, which partially address my concerns. The agent-replacement ablations and runtime comparison are helpful, and the responses clarify the training data source and scale. However, I still find the annotation procedure for the role-specific training data insufficiently explained. This leaves uncertainty about the training setup and reproducibility. On balance, I maintain my original rating.

Reviewer_63S5OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

PrismIQA splits no-reference IQA into four separately trained agents: a global assessor, a high-recall region proposer, a falsification agent that rejects pseudo-degradations, and an impact assessor that maps verified defects back to the global score. The authors claim that four independently optimized models have a tighter reward-deviation bound than one shared model. Reported validation is a comparison on seven IQA datasets, with 0.854 SRCC and 0.892 PLCC on Vista-Bench, plus ablations that remove individual agents. Reported validation is a comparison on seven IQA datasets, with 0.854 SRCC and 0.892 PLCC on Vista-Bench, plus ablations that remove individual agents.

Critical strengths

The useful design choice is to make falsification its own decision, with an abstain option. Crop-and-zoom IQA often treats a requested crop as evidence of poor quality, and this stage is aimed at separating intentional blur, bokeh, and natural texture from real defects. Defining each agent by its objective, visible evidence, and allowed actions is also clearer than assigning those roles by prompt on one shared model.

Critical weaknesses
  1. Experimental reproducibility and credibility. (a) Table 1 cannot be reconciled with results reported in multiple published papers on the same public models and datasets. For example, under the published KonIQ-train protocol, the SRCC values for MUSIQ, Q-Align, and Q-Insight are 0.929, 0.940, and 0.916, while this paper reports 0.473, 0.573, and 0.806. The paper does not say whether these entries were copied, evaluated from official checkpoints, or retrained, and it does not provide exact split manifests. Therefore, the comparison may use incompatible protocols. (b) The training pipeline is seriously underspecified. The paper does not explain how the SFT demonstrations or role-specific trajectories were obtained, nor how proposal boxes, ternary falsification labels, severity, importance, and regional-impact targets were annotated. Only the global and falsification rewards are written down; the GRPO rewards for the proposer and impact assessor are absent. Dataset sizes, role-wise splits, training steps, and validation procedures are also missing. Thus the proposed system cannot currently be reproduced.

  2. Limited evidence for innovation, stability, and practicality. (a) The architecture mainly composes established global IQA, crop/zoom region proposal, contextual verification, and score aggregation. The dedicated falsification role is potentially useful, but the paper does not isolate its benefit under a matched budget. Four independent full-parameter 8B models are compared with one shared 8B model, confounding factorization with roughly fourfold model capacity and additional optimization/inference compute. The reported gain over the single-agent control is only 0.020 average SRCC. (b) A sequential agent pipeline can propagate early localization and verification errors, yet the decoding configuration, repeated inference results, and training-seed variance are not reported. Mean and standard deviation over training seeds and, if sampling is used, repeated decoding runs are needed. (c) Efficiency is central for IQA metrics and reward models, which are often called repeatedly during evaluation or optimization. PrismIQA uses four 8B checkpoints and up to five regional evaluations, but reports no latency, throughput, FLOPs, number of VLM calls, peak memory, or early-exit rate. The practical cost relative to Q-Probe, the single-agent control, and conventional IQA models is therefore unknown.

Questions for authors
  1. For every Table 1 entry, was the number copied, obtained from official weights, or produced by retraining? Please provide the exact split manifest for every dataset, explain the large disagreement with the cited KonIQ results, and rerun all methods on identical test images if needed.

  2. What datasets and annotations supervise each role? How were the SFT demonstrations/trajectories constructed, and what are the exact SFT targets and GRPO rewards for the proposer and impact assessor? Please provide role-wise sample counts, splits, training steps, and validation procedures.

  3. Does the advantage remain under matched total parameters and training/inference compute? Please report mean and standard deviation across training seeds and, if applicable, repeated decoding runs. Please also compare per-image latency, throughput, FLOPs, VLM calls, peak memory, and early-exit rate with Q-Probe and the single-agent control.

1 comment

1. Data Source and Training Setup for Table 1

For every entry in Table 1, the evaluation data are drawn from VisualQuality-R1. Our model is trained on the training set of KADID-10K. This training setup is intended to demonstrate the model’s generalization ability, which is a commonly adopted setting in IQA tasks.

2. Data Construction and GRPO Reward Design

We mainly use KADID-10K to construct the data. Specifically, the 8k training images in KADID-10K are used to generate 8k trajectories, and each trajectory runs through 4 roles. The GRPO rewards for the Proposer and the Impact Assessor are, respectively, a classification reward (1 for correct, 0 for incorrect) and a Continuous Fidelity Reward.

The Continuous Fidelity Reward is not a discrete binary reward. Instead, it evaluates the ranking correctness of each quality estimate. For the k-th quality estimate of image i, its reward is the average fidelity after pairing it with all other images j in the current minibatch, i.e., over B−1 images, where B is the minibatch size and K is the number of quality estimates per image. For each pair (i, j), the fidelity is 1 minus the absolute difference between the model’s pairwise comparison probability and the human preference. The comparison probability is the Thurstone probability that the k-th quality estimate of image i is higher than that of image j, and the human preference is given by human MOS. This reward measures the agreement between the comparison probability and human preference: when human preference indicates i is better than j, a higher probability for this ordering yields a higher reward; when it indicates the opposite, a lower probability yields a higher reward. If B equals 1, the reward is set to 0.

The Thurstone comparison probability is obtained by applying the standard Gaussian CDF to a standardized score difference: the numerator is the k-th predicted quality score of image i minus the mean of the K quality predictions for image j, and the denominator is the square root of the sum of the sample variances of the K predictions for images i and j plus a small positive constant gamma that prevents division by zero.

The human preference is 1 if the MOS of image i is greater than that of image j; it is 0.5 if the two MOS values are equal; otherwise it is 0.

We collect the K rewards for each image and normalize them within the group into advantages using the group mean and standard deviation.

Key points:

  • The Continuous Fidelity Reward uses the continuous consistency between the model comparison probability and the human preference, providing a finer-grained ranking supervision signal.
  • Compared with the binary reward commonly used in GRPO, it can capture subtle differences in quality ranking, thereby improving generalization across distortion scenarios.
  • The reward computation explicitly uses the mean and variance obtained from multiple GRPO samples, so it can model prediction uncertainty.

3. Theoretical Proof and Experimental Advantage

In fact, a substantial part of our theoretical proof addresses exactly this point: under the same parameter setting, our theoretical analysis shows that, under the stated assumptions, our scheme outperforms joint training and other multi-agent approaches. Empirically, using the same set of parameters but only different prompts as different roles achieves only 0.855, 0.928, 0.919, 0.503, 0.824, 0.808, and 0.822 across the seven reported metrics in the same order, which is overall lower than PRISMIQA: 0.892, 0.931, 0.923, 0.515, 0.884, 0.898, and 0.846. The gaps are especially clear on the first, fifth, sixth, and seventh metrics, while the second and third metrics are close. This demonstrates the clear advantage of our approach over prompt-based role separation under the same parameter budget.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.