- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes two specialized reward modeling pipelines. FIRM-Edit uses a difference-first procedure to evaluate editing execution and source consistency, while FIRM-Gen uses a plan-then-score procedure for complex text-to-image prompts. The paper further proposes CME and QMA reward formulations to mitigate reward hacking during RL.
Critical strengths
The paper covers reward data construction, human evaluation, reward model training, benchmarking, and downstream RL in a relatively complete system. The specialized critics consistently improve performance on both editing and generation tasks. The discussion of the interaction between reward formulation and reward hacking is also practically relevant for RL-based image generation and editing.
Critical weaknesses
-
The principles behind CME and QMA are still largely heuristic, and the evaluation covers only a limited set of reward-hacking behaviors. The paper mainly studies copying the source image and generating low-quality black silhouettes. Other realistic failure modes include modifying an easy region while ignoring the main instruction and so on. A systematic adversarial reward-hacking suite would help determine whether CME and QMA generalize beyond the specific shortcuts studied in the paper. Comparisons with weighted sums, multiplicative formulations, constrained optimization, and Pareto-based approaches would also be useful.
-
The generalization of the critic to reference-based editing is not fully established. Although FIRM-Edit receives both source and edited images, the training distribution and generator diversity may limit generalization. The authors should test unseen editing models, local edits, multi-step edits, identity preservation, text editing, and different image resolutions.
-
The data construction pipeline relies heavily on MLLM-generated difference descriptions, scoring plans, and training labels. FIRM-Bench contains only 807 samples, and the paper does not sufficiently report inter-annotator agreement or how teacher errors propagate into the final critic.
Questions for authors
See Critical Weaknesses