acceptodds
Under review as a conference paper at ICLR 2027

MIEScore: A Human-Aligned Benchmark for Multi-Source Image Editing

Abstract

Recent advances in unified multimodal models have significantly improved text-guided image editing abilities. In particular, models such as Nano-Banana-Pro and GPT-Image-2 demonstrate emerging capabilities in multi-source image editing (MIE), including tasks such as object synthesis, person-background composition, and cross-image style fusion. However, existing benchmarks and image editing assessment (IEQA) methods remain primarily focused on single-image editing tasks and largely overlook the more challenging setting of MIE. This highlights the urgent need for a comprehensive and human-aligned benchmark for MIE. To this end, we introduce **MIE-Bench**, the first large-scale multiple image editing benchmark with fine-grained human preference annotations. Specifically, MIE-Bench includes 3,000 editing instances across 16 tasks, each involving more than two source images and an editing prompt, together with 36K edited images produced by 12 state-of-the-art editing models and over 108K mean opinion scores (MOSs) covering visual quality, instruction following, and attribute preservation. Based on MIE-Bench, we propose **MIEScore**, a multimodal large language model (MLLM)-based evaluation model enhanced with skill optimization and multi-dimensional supervised fine-tuning, to provide human-aligned feedback for MIE. Extensive experiments show that MIEScore achieves state-of-the-art performance in aligning with human preferences and generalizes well across other IEQA datasets. Both the dataset and model will be released.

Primary area:
datasets and benchmarks
Keywords:
multi-source image editing, benchmark, image quality assessment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ctuxOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MIE-Bench and MIEScore for evaluating multi-source image editing. The benchmark contains 3,000 editing instances across 16 tasks, 36,000 outputs from 12 editing models, and human scores for visual quality, instruction following, and attribute preservation. At inference, MIEScore takes the source images, editing instruction, and a candidate edited image, and predicts these three scores. It combines a natural-language evaluation skill optimized using scored trajectories and validation feedback with a Qwen3-VL-8B evaluator trained through LoRA and a regression head.

Critical strengths

The multi-source evaluation problem is reasonable.

The benchmark could be a useful community resource. Its task and generator coverage, together with separate quality, instruction-following, and preservation labels, could support training and comparing evaluators independently of the proposed model.

The experiments provide encouraging evidence for the training combination. A same-backbone ablation reports a gain from adding skill optimization, and fixed-parameter transfer tests explore its applicability to other evaluation datasets. The appendix also provides several concrete SFT hyperparameters.

Critical weaknesses
  1. The reliability of the human supervision is difficult to assess. Conflicting descriptions of annotation and filtering leave the actual data-collection protocol unclear, and agreement among annotators is not established. This weakens confidence in the benchmark's central contribution. Also, calibration of score-based annotations is important, which has been shown in some technical reports (e.g., QwenImage2).

  2. The reported overall scores are inconsistent with their component scores. The paper combines visual quality, instruction following, and attribute preservation using a weighted geometric mean. Under the same samples and score scale, this total cannot exceed the corresponding weighted arithmetic mean, even when scores are averaged across images. Yet, for BAGEL on Background Transfer, the reported component scores imply an upper bound of 32.317, while the reported overall score is 44.07. This cannot be explained simply by changing the order of averaging and aggregation. The paper needs to clarify whether different samples or scales were used, or correct the scores, before the resulting rankings can be trusted.

  3. Generalization to unseen multi-source editing problems is not sufficiently established. The split description does not rule out shared source content and instructions across training and testing. Pooled correlations and model-level rankings also leave performance on choosing among outputs for the same task, and on unseen editors, uncertain.

4.The benefit of automatic skill optimization is not clearly isolated. Adding the final skill improves the same-backbone model, but a matched comparison with a carefully written fixed multi-source rubric is missing. The evidence therefore does not establish how much automatic search adds beyond better task-specific instructions. Comparably trained joint multi-image evaluators would also clarify the broader methodological increment.

  1. The methodological contribution specific to multi-source editing is limited. The evaluation dimensions, human-rating protocol, skill optimization, and supervised score prediction are largely generic and could be applied to other image-editing evaluation settings. Although the rubric includes source-role instructions, the paper does not establish a substantial evaluation mechanism or annotation design tailored to cross-source interactions. Restricting the study to multi-source editing therefore narrows its coverage without a clearly demonstrated methodological gain; the stronger contribution appears to be the dataset's coverage.

  2. RL result of MIEScore is more convinced.

Questions for authors

Which annotation protocol was actually used? Please reconcile the conflicting rater, source-image, and generator counts, and provide rating counts, exclusion logs, filtering denominators, and inter-rater reliability.

Can Tables 7–10 be regenerated from the same per-image scores using the stated aggregation rule? Please explain the Background Transfer discrepancies, report whether corrections affect Tables 2–5 or model rankings, and clarify the differences between the fine-tuned DeepSeekVL2/InternVL3.5 entries in Tables 2 and 4.

Are all outputs sharing a source-image/instruction group kept in one split, with reused source images prevented from crossing splits? Please provide group-disjoint, within-instance, unseen-editor, and source-count results with uncertainty estimates.

How much does automatic skill optimization improve over a carefully written fixed multi-source rubric and a comparable jointly multi-image evaluator under the same backbone, training data, and compute budget?

Which parts of the annotation protocol or evaluation method specifically address cross-source interactions beyond a generic editing evaluator with multiple image inputs? Can controlled experiments demonstrate a benefit from these multi-source-specific choices?

16 comments
AuthorsOpenReview ↗

W1&Q1. Clarification of Human Annotation Reliability and Score Calibration

W1&Q1. Clarification of Human Annotation Reliability and Score Calibration

We sincerely thank the reviewer for raising this important concern regarding the reliability of human supervision and the calibration of score-based annotations. We agree that rigorous quality control, reliable human annotations, and proper score calibration are essential for ensuring the credibility of the benchmark. To address these concerns, we have further clarified our subjective data collection and processing protocols in the revised manuscript.

1. Annotation protocol

We recruited 15 human subjects to independently evaluate the edited images, following the ITU-R BT.500 [1] recommendations for subjective image quality assessment. Before the formal evaluation, all subjects received standardized training to familiarize themselves with the evaluation protocol, the three assessment dimensions (visual quality, instruction following, and attribute preservation), and the corresponding scoring criteria. They then completed a preliminary test to ensure their understanding of the evaluation requirements and scoring procedure before proceeding to the formal annotation.

The evaluation consisted of 36,000 edited images, generated from 3,000 source images using 12 editing models. All edited images were randomly shuffled to avoid potential ordering effects, and each subject independently evaluated the entire set of 36,000 images across all three dimensions. Thus, each subject provided 108,000 ratings, resulting in 1,620,000 raw subjective ratings in total.

To ensure annotation reliability, we followed the recommended procedures to identify abnormal ratings and screen unreliable subjects. An individual rating for an image is considered an outlier if it deviates from the mean rating of that image by more than 2 standard deviations for normally distributed ratings or 20\sqrt{20} standard deviations otherwise. Following this procedure, 5.24% of the total subjective ratings were identified as outliers and removed. Specifically, among the 1,620,000 raw subjective ratings, 26,856, 28,739, and 29,295 ratings were identified as outliers and removed for visual quality, instruction following, and attribute preservation, respectively. The numbers and percentages of outlier ratings removed for each subject are reported in table below.

SubjectRemoved RatingsRemoval Rate
Human13,4213.17%
Human23,7863.51%
Human34,1293.82%
Human44,4674.14%
Human54,8134.46%
Human65,1624.78%
Human75,4895.08%
Human85,7945.36%
Human96,1275.67%
Human106,4185.94%
Human116,6836.19%
Human126,9756.46%
Human137,2466.71%
Human147,4986.94%
Human156,8826.37%
Total84,8905.24%

We further report both Kendall's coefficient of concordance (W) and the intraclass correlation coefficient (ICC(2,15)) to evaluate inter-subject agreement and the reliability of aggregated subjective scores, respectively. As shown in table below, the three evaluation dimensions achieve Kendall's W values of 0.712, 0.734, and 0.728, and ICC(2,15) values of 0.923, 0.941, and 0.938, respectively, with average values of 0.725 and 0.934. This consistently high values across all three dimensions indicate good inter-subject agreement and high reliability of the aggregated MOS scores.

DimensionKendall's WICC(2,15)
Visual Quality0.7120.923
Instruction Following0.7340.941
Attribute Preservation0.7280.938
Average0.7250.934
AuthorsOpenReview ↗

W1&Q1. Clarification of Human Annotation Reliability and Score Calibration

2. Score calibration

To mitigate inter-subject differences in rating tendencies and scoring-scale usage, we perform subject-wise Z-score normalization as a score calibration before aggregation following the ITU-R BT.500 [1]. Similar score calibration have been adopted in existing image quality assessment studies, such as ARIQA [2] and AGIQA-3K [3], and LMM4Edit [4]. This calibration reduces systematic biases caused by individual differences in score means and variances, making ratings from different subjects more comparable while preserving their relative scoring patterns. Specifically, each subject's valid ratings are normalized using their own mean and standard deviation, followed by linear scaling to [0,100][0,100]. The final Mean Opinion Score (MOS) is obtained by averaging the calibrated ratings for each image. The computation is formulated as:

\qquad z_j=\frac{1}{N_j}\sum_{i=1}^{N_j}z_{ij}, \qquad MOS_j=\frac{100(z_j+3)}{6},$$ where $r_{ij}$ denotes the raw rating assigned by the $i$-th subject to the $j$-th image, $\mu_i$ and $\sigma_i$ represent the mean and standard deviation of the ratings provided by the $i$-th subject, respectively, and $N_j$ denotes the number of valid ratings for the $j$-th image. **References** [1] H. Duan et al., "Confusing Image Quality Assessment: Toward Better Augmented Reality Experience," *IEEE Transactions on Image Processing*, vol. 31, pp. 7206–7221, 2022. [2] C. Li et al., "AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment," *IEEE Transactions on Circuits and Systems for Video Technology*, vol. 34, no. 8, pp. 6833–6846, 2024. [3] Z. Xu et al., "LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMs," Proceedings of the ACM International Conference on Multimedia, pp. 6908–6917, 2025.
AuthorsOpenReview ↗

W2&Q2. Correction of Column Ordering Errors in Tables

W2&Q2. Correction of Column Ordering Errors in Tables

Thank you for pointing out this inconsistency. We carefully checked the original results and identified a column-ordering issue in the second subtable of Tables 9–11, rather than an error in the score calculation. Specifically, the data columns were displayed in a different order from their corresponding column headers. The original data were arranged in the following order: Assisted Drawing, Background Transfer, Filter Composition, Style Transfer, Physical Understanding, Spatial Composition, Story Generation, and Interaction. However, the column headers did not follow this order, resulting in incorrect associations between task names and reported scores.

We have corrected the column ordering in Tables 9-11 in revised PDF to ensure that all scores are properly aligned with their corresponding tasks. Importantly, the underlying calculations and experimental results remain unchanged. This issue is confined to the presentation of Tables 9–11 and does not affect any other figures, tables, or conclusions of the paper. We apologize for the confusion and thank the reviewer for helping us identify this presentation issue.

Reviewer_hk4XOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces MIE-Bench a large-scale multiple image editing benchmark with fine-grained human preference annotations. The dataset contains 3000 editing sets and covers 16 tasks. Besides the data, the dataset also provides human annotations within three metrics: visual quality, instruction following and attribute preservation. Last, the paper based on the human evaluation, finetuned a vlm for generate vlm judge score named MIE-score. The score can be applied to several editing benchmarks.

Critical strengths

1 The dataset covers a relative diverse tasks, including composition, attribute transfer, style transfer, and complex reasoning. 2. The dataset provides human feedback across three complementary evaluation dimensions.

Critical weaknesses

This paper has several critical issues:

  1. The definition of successful preservation remains insufficiently precise. Pixel similarity, identity preservation, and attribute preservation are distinct requirements. An edit may preserve identity despite large pixel changes, or change identity while retaining high pixel similarity. 2. After checking the proposed metric on Figure 12 to 14, the reviewer has found that those metric dose not align with the numerical scores, specially when different source images supply identity, pose, style, or background.

The line 951 mentioned "The scoring annotation task involved 5 participants rating each image on a 0-5 Likert scale, assessing visual quality, instruction following and attribute preservation." The Figure 1 mentioned 15 annotator.

Questions for authors

The baseline methods are not fairly chosen. Single-source metric model process references separately and average their scores, while MIEScore receives all references jointly. Can we use all the reference images and result for open/closed source models? Such as opus 5.5, Qwen 3.5/3.8 models.

Can you provide controlled examples separating pixel preservation from identity preservation, with both human and MIEScore ratings?

13 comments
AuthorsOpenReview ↗

W1&Q2. Clarification of the Definition and Evaluation Criteria for Attribute Preservation

W1&Q2. Clarification of the Definition and Evaluation Criteria for Attribute Preservation

We thank the reviewer for this insightful comment. We agree that pixel-level similarity and identity preservation represent distinct aspects of image preservation. In our work, attribute preservation is defined as follows:

"This dimension assesses how well the edited image maintains the original visual attributes of the unedited regions in the source images. It focuses on preserving the context, structure, and appearance of unchanged areas, ensuring they are not unintentionally modified or degraded. This includes maintaining consistent lighting, texture, identity (in the case of humans or objects), and spatial relationships, contributing to the overall coherence and realism of the final result."

As reflected in this definition, our attribute preservation criterion is not limited to pixel-level similarity or identity preservation. Instead, we adopt a broader criterion that captures the overall preservation of visual attributes. This design is motivated by three considerations:

  1. Better alignment with human subjective evaluation. Human annotators naturally consider pixel-level appearance and identity-level consistency together when judging preservation. Evaluating them separately may cause confusion, as these aspects are interrelated and jointly contribute to the overall perception of whether the original visual attributes have been preserved.

  2. Broader applicability and feasibility across diverse editing scenarios. The relative importance and existence of pixel-level similarity and identity preservation varies across different editing tasks. For example, style transfer typically involves substantial pixel-level changes, making identity preservation more important than pixel similarity. In contrast, background transfer aims to retain the original foreground and background appearance, making pixel-level consistency more important. Treating these two aspects as independent criteria makes it difficult to balance and integrate them appropriately when assessing visual consistency across different editing tasks. In contrast, our broader attribute preservation criterion provides a holistic assessment of visual consistency by naturally integrating pixel-level and identity-level information, making it more effective across diverse editing scenarios.

  3. More effective reward supervision for downstream optimization. Treating pixel-level similarity and identity preservation as independent reward components may produce conflicting optimization signals. For example, in style transfer, substantial pixel-level changes are necessary to achieve the target style. Optimizing pixel-level similarity independently may therefore penalize desirable style transformations, even when the content identity is well preserved. Such conflicting signals can mislead the optimization process and hinder effective improvement. In contrast, our unified attribute preservation criterion integrates both aspects into an overall assessment, providing more coherent and effective reward supervision for downstream optimization.

We have added the detailed explanation of our attribute preservation criterion, together with illustrative examples, to Section A4.1 and Figure 7 of the revised manuscript.

AuthorsOpenReview ↗

W2. Clarification of the Visualization Order in Figures

W2. Clarification of the Visualization Order in Figures

Thank you for pointing this out. We would like to clarify that Figures 13–15 present results based on human annotations. The ranking is based on the models' average performance across all editing instances, with all models arranged in descending order. Consequently, the results for individual editing instances are not necessarily presented in descending order. To improve clarity and facilitate a more direct comparison, we have added the corresponding human-annotated mean opinion scores (MOSs) to each individual image in the Figure 13-15 in revised PDF, allowing readers to better understand the relationship between the visual results and their numerical scores.

AuthorsOpenReview ↗

W3. Correction of the Number of Human Annotators

W3. Correction of the Number of Human Annotators

We thank the reviewer for pointing out this inconsistency. The correct number of annotators is 15, as stated in Figure 1. The mention of 5 participants on Line 951 was a typographical error, which has been corrected in the revised manuscript.

Reviewer_vRELOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the limitations of traditional image quality metrics in understanding editing intentions, the weak alignment between direct MLLM-based scoring and human judgments, and the focus of existing image editing evaluation models on single-reference image settings. To address these challenges, the authors construct a benchmark for multi-source image editing (MIE) and develop a human-aligned evaluation model for MIE.

Critical strengths
  1. Constructed a high-quality multi-source image editing manual scoring dataset, using human real scoring as the supervisory signal. Compared to using MLLM automatic scoring as the supervisory signal, this data can reduce scoring bias;
  2. Using Skill Optimization to automatically optimize evaluation rules, ablation experiments have demonstrated its effectiveness;
  3. Numerous experiments have demonstrated the effectiveness of MIEScore and validated its generalization ability on different datasets.
Critical weaknesses
  1. The authors demonstrate the effectiveness of Skill Optimization but do not compare it with carefully crafted human-designed prompts. Moreover, key implementation details of Skill Optimization, such as the number of iterations and candidate skills, are insufficiently specified.
  2. The evaluation relies heavily on correlation-based metrics, including SRCC, PLCC, and KRCC, while lacking an analysis of absolute score accuracy and calibration.
Questions for authors
  1. Does Skill Optimization offer significant advantages over carefully crafted human-designed prompts in terms of time efficiency and evaluation accuracy?
  2. What is the rationale behind the selection of the λ coefficients in the optimization loss function?
  3. Could the authors provide additional comparisons of the absolute differences between model-predicted scores and human ratings?
7 comments
AuthorsOpenReview ↗

W1&Q1. Comparison of Human-Designed, LLM-Generated, and Optimized Skills

W1&Q1. Comparison of Human-Designed, LLM-Generated, and Optimized Skills

We thank the reviewer for suggesting a comparison with carefully crafted human-designed prompts. To provide a comprehensive comparison, we have conducted experiments involving three types of skills: human-written skills, directly LLM-generated skills using GPT-5 and Gemini-3.1-Pro, and skills optimized by our proposed method. For human-written skills, we invited three domain experts to independently develop skills for the multi-source editing evaluation task. Before skill development for both human experts and LLM, we provided them with a detailed introduction to our evaluation objectives and task definitions, along with access to our curated dataset. The experts were asked to carefully examine the source images, edited results, and corresponding editing instructions to identify task-specific evaluation considerations, critical visual attributes, and potential pitfalls that should be explicitly addressed in the skills. Based on these observations, each expert independently formulated skills incorporating the relevant evaluation criteria and task-specific considerations to guide the assessment of editing quality.

We evaluate the impact of these different skills on the performance of various editing evaluators. Notably, our method independently optimizes the evaluation skills for each evaluator, rather than applying a shared optimized skill across different evaluators. This enables us to investigate whether automatic skill optimization can consistently improve evaluator performance under different model architectures and capabilities.

As shown in Table below, our optimization strategy consistently achieves substantial performance improvements across different evaluators, demonstrating its effectiveness in enhancing editing evaluation through skill optimization. Experimental results and analysis have also been added in Section A7.2 and Table 16 of the revised manuscript.

AuthorsOpenReview ↗

W1&Q1. Comparison of Human-Designed, LLM-Generated, and Optimized Skills

|Visual Quality|Instruction Following|Attribute Preservation|Overall Score
ModelSkills|SRCCPLCC|SRCCPLCC|SRCCPLCC|SRCCPLCC
ChatGPT-5No Skill|0.54160.3512|0.50320.3953|0.43350.3712|0.53490.3892
Expert 1|0.68730.7128|0.75160.7793|0.76380.7715|0.75620.7738
Expert 2|0.65250.6816|0.73920.7458|0.73140.7583|0.72260.7415
Expert 3|0.63180.6614|0.69650.7282|0.70860.7393|0.70480.7217
GPT-5 Generated|0.63430.6678|0.69750.7336|0.71180.7484|0.70260.7263
Gemini-3.1 Generated|0.62850.6592|0.69630.7251|0.70620.7368|0.69840.7195
Our Method|0.70680.7315|0.76840.8002|0.78630.8105|0.77950.7873
Gemini-3.1-ProNo Skill|0.53720.4473|0.45090.3358|0.42610.3671|0.52440.3214
Expert 1|0.67450.7018|0.73820.7654|0.75260.7813|0.74680.7642
Expert 2|0.63830.6695|0.73160.7238|0.74350.7512|0.71540.7326
Expert 3|0.60280.6354|0.67150.7018|0.68840.7196|0.68050.6983
GPT-5 Generated|0.60850.6417|0.67420.7085|0.69160.7218|0.68240.7053
Gemini-3.1 Generated|0.61170.6438|0.67950.7096|0.69430.7252|0.68650.7084
Our Method|0.69180.7263|0.76420.7952|0.77160.7983|0.76580.7836
Qwen3-VL-7BNo Skill|0.38440.3379|0.36420.3934|0.38650.3781|0.41140.3678
Expert 1|0.64850.6773|0.71480.7416|0.72850.7594|0.72360.7415
Expert 2|0.62520.6518|0.71630.7035|0.72920.7368|0.70240.7196
Expert 3|0.58270.6135|0.64940.6786|0.66530.6974|0.65850.6762
GPT-5 Generated|0.58740.6195|0.65180.6842|0.66830.7015|0.65940.6827
Gemini-3.1 Generated|0.60180.6326|0.66530.6978|0.68150.7136|0.67380.6954
Our Method|0.68890.7210|0.75420.7896|0.77130.8038|0.76450.7761
LMM4EditNo Skill|0.48880.5928|0.61040.6326|0.37980.4377|0.58220.6209
Expert 1|0.51250.5468|0.58360.6124|0.59630.6258|0.58840.6052
Expert 2|0.49380.5215|0.59120.5786|0.60250.6173|0.57460.5918
Expert 3|0.45160.4823|0.51840.5462|0.52950.5583|0.52380.5416
GPT-5 Generated|0.46280.4915|0.52760.5538|0.53620.5664|0.53150.5493
Gemini-3.1 Generated|0.47830.5036|0.53950.5642|0.54860.5793|0.54370.5618
Our Method|0.55620.5815|0.61930.6478|0.63250.6594|0.62580.6442
EditRewardNo Skill|0.34230.3743|0.49420.4944|0.34540.3461|0.46410.4387
Expert 1|0.47160.5083|0.54820.5765|0.56130.5902|0.55380.5716
Expert 2|0.48650.4972|0.53680.5893|0.55240.5786|0.56520.5594
Expert 3|0.43820.4615|0.50730.5238|0.51670.5412|0.50160.5273
GPT-5 Generated|0.45270.4796|0.51950.5482|0.53580.5574|0.52630.5438
Gemini-3.1 Generated|0.44350.4862|0.52940.5376|0.54320.5498|0.51850.5362
Our Method|0.51840.5416|0.57830.6052|0.58760.6194|0.58250.6013
AuthorsOpenReview ↗

W1&Q1. Comparison of Human-Designed, LLM-Generated, and Optimized Skills

|Visual Quality|Instruction Following|Attribute Preservation|Overall Score
ModelSkills|SRCCPLCC|SRCCPLCC|SRCCPLCC|SRCCPLCC
Qwen3-VL-7B (Fine-tuned)No Skill|0.80510.8136|0.85390.8762|0.86840.8719|0.86270.8758
Expert 1|0.81260.8163|0.85910.8952|0.88940.8971|0.88730.8832
Expert 2|0.80560.8092|0.85150.8894|0.88230.8916|0.88120.8775
Expert 3|0.76840.7975|0.83360.8542|0.84450.8613|0.84560.8462
GPT-5 Generated|0.77150.7864|0.82160.8613|0.85320.8774|0.85030.8586
Gemini-3.1 Generated|0.78320.7958|0.83450.8736|0.86540.8842|0.86250.8691
Our Method|0.82390.8257|0.87030.9077|0.90060.9075|0.90000.8954

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Our Responses & Paper Revisions

We sincerely thank all reviewers for their valuable and constructive feedback. We appreciate that the reviewers recognized the importance and potential value of MIE-Bench in addressing the underexplored multi-source image editing evaluation problem (Reviewers vREL, hk4X, ctux), acknowledged its diverse task coverage and fine-grained human annotations across three complementary dimensions (Reviewers hk4X, ctux), and recognized the effectiveness and generalization capability of MIEScore, as well as the promising results of Skill Optimization (Reviewers vREL, ctux).

We have made our best effort to address the concerns and revise the paper accordingly. The major modifications are summarized as follows:

1. More comparisons with editing-specific evaluators: We have included additional comparisons with fine-tuned editing-specific evaluators, as reported in Table 2, to provide a more comprehensive assessment of MIEScore under comparable settings.

2. Absolute score accuracy and calibration: We have conducted additional analyses of absolute score accuracy and calibration, including MAE, RMSE, calibration slope, and calibration intercept, as reported in Table 3, to further assess the accuracy and reliability of MIEScore.

3. Selection of optimization coefficients: We have conducted ablation studies on the λ coefficients in the optimization loss function, as reported in Table 12, and provided further analysis to justify our design choices.

4. Comparison of evaluation skills: We have added comparisons with human-written and LLM-generated evaluation skills, as reported in Table 17, to further investigate the effectiveness of automatic Skill Optimization and its advantages over manually designed evaluation rules.

5. Additional RL experiments: We have conducted additional reinforcement learning experiments by using MIEScore as the reward model to optimize Qwen-Image-Edit. We further evaluate the RL-enhanced editing results using both MIEScore and human annotations, as detailed in Section A.8.

The major revised contents in the manuscript are highlighted in blue. Point-for-point responses to specific comments are given in the following reviewer-specific responses. We welcome any further discussions and will address any remaining concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.