acceptodds
Under review as a conference paper at ICLR 2027

Search Finds Blind Spots: Stress-Testing Edit Reward Models Under Optimization Pressure

Abstract

Image editing reward models guide candidate selection and iterative refinement by scoring images. While evaluations measure agreement with human ratings, they provide limited evidence about reliability under sustained optimization. To address this gap, we present **EditStress-Bench**, a benchmark for evaluating editing rewards under search pressure. It comprises 300 human-object editing tasks, matched search controls, and independent human assessment of quality and editing defects. Experiments evaluate twelve reward settings from six reward families using Best-of- selection and iterative refinement. The strongest static reward becomes the weakest Best-of- selector at a budget of 64 candidates, revealing a mismatch between *static agreement* and *search-time quality*. Rewards with source-image access remain 10.6 Overall Human Quality (OHQ) points below the human oracle on average. Motivated by these findings, we introduce **factor-guided reward patching**, which profiles defect sensitivity, predicts selected failures, and targets weak factors during training. Targeted updates improve OHQ by 1.6 to 2.1 points, compared with 0.3 to 0.4 for equal-budget random updates. Repeated patching reduces monitored failures, while a broader audit identifies persistent appearance drift. Code, evaluation data, and human annotations will be released.

Primary area:
datasets and benchmarks
Keywords:
image editing, reward model evaluation, optimization pressure, reward overoptimization, human evaluation, failure analysis

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_298BOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose EditStress-Bench, a benchmark for evaluating the reliability of image editing reward models under increasing search budgets. Using HICO-DET-derived human–object editing tasks, it compares static reward–human agreement with the human-rated quality of outputs selected through Best-of-N and iterative refinement, revealing that the agreement does not necessarily translate into effective selection. The authors also introduce factor-guided reward patching, which uses controlled perturbations to identify weaknesses in detecting specific editing defects, predict failures amplified by search. The approach is evaluated through search with Overall Human Quality (OHQ), a holistic score assessed by a separate three scorers.

Critical strengths
  • Separating holistic quality assessment from defect annotation also reduces overlap between the evidence used to evaluate quality and diagnose failures.
Critical weaknesses
  • It seems that the discussed problem can be mitigated simply by adopting stronger reward models. Beyond GPT-4o judge (Table 1), evaluating recent strong VLMs such as Claude Opus 5.5 or GPT-6 Astra as reward model would help establish whether the mismatch between static agreement. The authors should demonstrate that their approach addresses a persistent practical limitation and provides gains beyond those obtainable simply by enhancing the underlying reward model.

  • EditStress-Bench’s reliance on HICO-DET limits both its task coverage and its evaluation framework to human-centric editing scenarios. The five factors is tailored to human–object interactions, most explicitly through human fidelity and the emphasis on action completion and contact consistency. This specialization limits the framework’s applicability to broader image editing tasks, weakening its representativeness as a general well-known image editing benchmark such as PIE-Bench [1]. More diverse tasks and evidence of transfer beyond human-centric scenarios are needed to support broader claims.

[1] Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Code (Ju et al., ICLR 2024)

Questions for authors
  1. The practical relevance of the problem setting is insufficiently established. In particular, the paper provides limited justification for best-of-N scaling with large N in image editing tasks. Without supporting references or concrete use cases, the motivation for making this setting a central focus of the benchmark remains weak.

  2. The distinction between the default (D) and alternative (A) settings in Table 1 is unclear and hard to understand. Could the authors explain (or define) precisely on Section 4 how these settings differ and why both are considered?

4 comments
AuthorsOpenReview ↗

Responses to Reviewer 298B (Part I)

To reviewer 298B: Thanks for the constructive comments. Responses to specific comments are given below, and the corresponding revisions are highlighted in blue in the revised paper.

  1. W1: We respectfully clarify that stronger reward models do not remove the mismatch. Following the suggestion, we have evaluated GPT-6 Astra and Claude Opus 5.5 as judges under the two protocols of the GPT-4o judge. All configurations share the same nested candidate pools on Core-120 (Table R5, Appendix B.7, Sec. 4.3).
    ConfigurationStatic SRCC (rank)N = 16 OHQ (rank)N = 64 OHQ (rank)Oracle gap
GPT-6 Astra (D)0.861 (1)82.1 (1)79.3 (11)12.3
Claude Opus 5.5 (D)0.847 (2)81.5 (3)78.8 (12)12.8
GPT-6 Astra (A)0.836 (3)80.3 (5)83.1 (3)8.5
Claude Opus 5.5 (A)0.819 (5)79.9 (7)82.8 (6)8.8

The two frontier judges lead static agreement and small-budget search, yet fall to 11th and 12th among sixteen configurations at N = 64. With joint 95% intervals, rank transfer at N = 64 is −0.389 [−0.653, −0.051] among the fourteen source-aware configurations. For all sixteen configurations it is −0.176 [−0.403, 0.138], following the same pattern as Table 6. On the same backbones, the alternate settings score five rubric items and select outputs 3.8 and 4.0 points higher at N = 64. A stronger backbone therefore does not prevent search-time failures, whereas scoring composition changes selected quality. This is the gap that factor-guided patching addresses, raising OHQ by 1.6 to 2.1 points for a fixed backbone (Table 4).

  1. W2: EditStress-Bench is designed as a stress test of editing rewards under optimization pressure, and it is not intended as a general editing benchmark like PIE-Bench. Human-object interaction editing is chosen because target, action, contact and preservation constraints must hold jointly, which makes reward errors most consequential under search (Sec. 3.2). To show that the findings extend beyond human-centric editing, we have added the following results.

(1) Transfer to PIE-Bench. We sample 100 non-human tasks from PIE-Bench (Ju et al., 2024) with a fixed seed, covering 29 object, 27 attribute, 23 background and 21 style edits. All tasks use the official editing instructions. Qwen-Image-Edit generates 64 candidates per task, the twelve configurations select from shared nested pools, and OHQ follows the same three-rater blinded protocol (Sec. 4.3, Appendix B.8).

      Family    D − A, HOID − A, PIE-BenchD, N = 16 → 64, HOID, N = 16 → 64, PIE-Bench
InternVL-Reg−5.5 [−7.1, −3.9]−2.5 [−3.6, −1.4]−0.6 [−1.8, 0.5]−1.4 [−2.2, −0.6]
EditReward−2.1 [−4.1, 0.0]−0.8 [−1.8, 0.2]+4.5 [3.5, 5.5]+2.1 [1.3, 2.9]
EditScore−6.7 [−8.4, −5.0]−2.8 [−3.8, −1.7]−1.3 [−2.4, −0.1]−1.0 [−1.9, −0.2]
VIEScore−3.2 [−4.9, −1.5]−1.9 [−3.0, −0.8]−1.9 [−3.0, −0.7]−1.6 [−2.5, −0.8]
GPT-4o judge−5.3 [−7.0, −3.5]−2.3 [−3.3, −1.3]−2.7 [−3.8, −1.5]−1.9 [−2.8, −1.1]

On PIE-Bench, D selects outputs 1.9 to 2.8 points below A in four families, and these D settings lose 1.0 to 1.9 points from N = 16 to N = 64. All eight intervals exclude zero (Table R8). EditReward behaves the same way on both datasets. Rank transfer over the twelve configurations falls from −0.007 at N = 16 to −0.182 at N = 64 (Table R7).

(2) Factor set. The five factors are specific to human-object interaction by design. Profiling, enrichment and targeted patching only require a factor set with calibration pairs. In the initial version, Appendix F.5 already applied the same regression to five categories identified in the residual audit. The sensitivity-enrichment slope remains negative for these categories alone (−3.799 [−4.254, −3.294] under Best-of-N) and for the union of ten factors (−4.109 [−4.433, −3.774]) (Table 25). The framework can therefore be instantiated with factors suited to other editing domains.

AuthorsOpenReview ↗

Responses to Reviewer 298B (Part II)

  1. Q1: We have added references and use cases to the introduction (Sec. 1). Reward scores are used for Best-of-N reranking of image editing candidates (Luo et al., 2026) and to guide inference-time search in image generation (Ma et al., 2025). They also provide group-relative rewards for reinforcement learning (Liu et al., 2025; Xue et al., 2025). In each case, the reward repeatedly selects among many candidates, which is the pressure that EditStress-Bench measures. We are also encouraged that Reviewers g2x7 and EJ17 found this search-time setting practically relevant.

Importantly, the finding does not depend on a large N. In the initial version, Appendix B.5 (Table 10) already showed that source-aware Best-of-N rank transfer falls from 0.479 at four calls to 0.006 at eight calls. The revised Sec. 4.2 now states this in the main text. Larger budgets show how far selection can drift, for example the decline of three default settings between N = 16 and N = 64 (Table R3).

  1. Q2: We have defined the D/A settings in Sec. 4.1 and added a Composition column to Table 1. D and A share the backbone of each family and differ in scoring head, rubric weighting or aggregation. In five families, D uses the native or single scalar output and A adds factor information. For VIEScore, D weights the rubric items unequally and A weights them equally. Full specifications are given in Appendix A.2. Both settings are considered because they hold the backbone fixed and change only the scoring composition, which isolates its effect on search-time quality. At N = 64, D selects outputs 2.1 to 6.7 points below A under Best-of-N in all five source-aware families, with four intervals excluding zero (Table R2).

We hope these responses address the reviewer's concerns, and we would be glad to discuss any remaining questions.

Thank you for the detailed response and additional experiments. The evaluations with stronger judges substantially address my concern that the observed failures might simply reflect limitations of the original reward models.

The PIE-Bench results also provide useful evidence that the search-time failure phenomenon extends beyond human–object interaction editing, which showing robustness. My small remaining concern is whether the benefits of factor-guided patching generalize as well, although the new experiments appear to establish transfer of the failure phenomenon.

Overall, the response has resolved most of my initial concerns. I now lean slightly toward acceptance and have revised my rating to Weak Accept.

AuthorsOpenReview ↗

Response to Reviewer 298B

To reviewer 298B: We sincerely thank the reviewer for the positive feedback and the raised rating. We are glad that the frontier-judge and PIE-Bench results resolve the main concerns. Regarding the generalization of factor-guided patching, the current evidence is summarized below.

  1. Across scorers and factors: Patching is applied to three scorers with different diagnosed factors, namely SP for InternVL-Reg (D), NP for EditReward (A) and HF for EditScore (D). In all three cases, targeted updates raise OHQ by 1.6 to 2.1 points under rerun search on the held-out patch tasks, compared with 0.3 to 0.4 for equal-budget random updates (Table 4).

  2. Beyond the original factor set: Patching relies on the link between low sensitivity to a factor and its enrichment after search. This link also holds for the five additional categories identified in the residual audit (Table 25, Appendix F.5) and when whole reward families are held out from fitting (Table R4). Since the procedure only requires a factor set with calibration pairs, the same targeting can be applied with factors suited to other editing domains.

Thanks again for the constructive discussion, which has helped improve the paper.

Reviewer_EJ17OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces EditStress-Bench to investigate the reliability of image-editing reward models under sustained search pressure. Experiments cover twelve configurations across six reward model families. Using shared candidate pools for Best-of-N and iterative optimization, the study compares static human alignment against output quality after search. With independent human quality ratings and annotations across five failure categories, the authors analyze failure modes amplified by search and propose targeted mitigation based on factor sensitivity.

Critical strengths

The evaluation shifts from static scoring to actual selection behavior. Reward models ultimately govern outputs after search, rather than merely correlation over random samples. The shared candidate pool paired with a human oracle disentangles two distinct failure modes: the generator failing to produce high-quality candidates, and the reward model misranking valid candidates. This distinction carries direct practical utility. This work establishes an evidentiary loop of diagnosis, intervention, and re-search. Mitigation is not limited to rescoring existing candidates; search is rerun independently across 900 tasks with five seeds per task. Targeted updates deliver OHQ gains of 1.6–2.1 points, compared with 0.3–0.4 points from random updates. The authors also report residual distribution shift and refrain from framing local fixes as a complete solution.

Critical weaknesses

Cross-family generalization of the diagnostic model and the unique merit of the mitigation strategy remain insufficiently validated. Under LOCO (leave-one-configuration-out) validation, other configurations from the same reward family may still appear in training. The four additional test configurations are mostly variants of existing families. Consequently, strong predictive correlation does not guarantee generalization to unseen reward families. Moreover, targeted mitigation is benchmarked primarily against random sampling; it has not been shown to outperform straightforward baselines such as prioritizing frequent failure modes, low-sensitivity factors, or hard examples.

Questions for authors
  1. Could the authors provide leave-one-family-out prediction results
  2. Under identical annotation and training budgets, does factor-based diagnosis outperform simple baselines such as directly selecting the most frequent failure modes or standard hard-example mining?
  3. How stable are quality gaps in selection performance across different configurations? Can the authors report paired OHQ difference intervals for key configurations instead of relying solely on rank shifts
2 comments
AuthorsOpenReview ↗

Responses to Reviewer EJ17

To reviewer EJ17: Thanks for the constructive comments. Responses to specific comments are given below, and the corresponding revisions are highlighted in blue in the revised paper.

  1. W1 & Q1: Following the suggestion, we have added leave-one-family-out (LOFO) prediction, which removes both settings of a family before fitting. The results are presented below and in Table R4 (Sec. 5.1.2 and 5.2.1, Appendix D.6) of the revised paper.
         Searcher            PredictorLOCO MAELOFO MAELOFO ρsLOFO Hit
Best-of-NFactor prior0.15650.16490.7205/10
Best-of-NSensitivity profile (ours)0.07660.07770.8907/10
Iterative refinementFactor prior0.14420.15330.7706/10
Iterative refinementSensitivity profile (ours)0.07780.08940.8406/10

When the whole family is held out, the error of the sensitivity profile stays well below that of the factor prior under both searchers. The diagnostic therefore transfers to reward families that are absent from fitting. The four additional configurations in Appendix D.5 were designed to test changes in calibration and aggregation within a family, and LOFO now provides the cross-family test.

  1. W2 & Q2: We would like to clarify that prioritizing the low-sensitivity factor is the targeting rule of factor-guided patching itself (Sec. 5.1.3). We have added the other suggested baselines under identical annotation and training budgets. All arms draw 512 new and 512 replay pairs from the same labeled pool and share trainable modules and optimizer steps. They are evaluated on a 300-task subset of the patch holdout with a fixed seed (Table R6, Sec. 5.1.3 and 5.2.2, Appendix E.6).
Scorer (diagnosed factor)                 ArmΔOHQΔ diagnosed-factor failure (pp)Targeted − arm [95% CI]
EditScore (D), HFRandom+0.3−1.3+1.8 [1.0, 2.5]
EditScore (D), HFFactor-isolated random+1.1−3.3+1.0 [0.3, 1.7]
EditScore (D), HFHard-example mining+1.8−9.7+0.3 [−0.5, 1.0]
EditScore (D), HFTargeted (ours)+2.1−11.7n/a
InternVL-Reg (D), SPRandom+0.4−1.7+1.5 [0.7, 2.3]
InternVL-Reg (D), SPMost frequent factor+1.7−2.0+0.2 [−0.6, 1.0]
InternVL-Reg (D), SPTargeted (ours)+1.9−11.3n/a

Factor-guided targeting clearly outperforms both random arms and performs on par with hard-example mining and the most-frequent-factor arm. The comparison with hard-example mining is informative. Among its training pairs, 71.7% contain an HF defect, compared with 29.1% of random pairs, so loss-based mining concentrates on the factor that the profile diagnoses before training. Beyond naming this target in advance, the profile also predicts enrichment for reward families held out from fitting (W1 above), which loss-based mining does not provide.

  1. Q3: We have added paired OHQ differences with 95% task-cluster bootstrap intervals for the key configurations (Table R2, Sec. 4.2 and 4.3, Appendix B.6) and for budget changes (Table R3, Sec. 4.2).
      Comparison at N = 64     Best-of-NIterative refinement
EditScore (D) − EditReward (A)−7.0 [−9.1, −4.8]−3.0 [−4.8, −1.1]
InternVL-Reg, D − A−5.5 [−7.1, −3.9]−3.9 [−5.7, −2.1]
EditReward, D − A−2.1 [−4.1, 0.0]−6.4 [−8.3, −4.3]
EditScore, D − A−6.7 [−8.4, −5.0]−0.4 [−2.1, 1.3]
VIEScore, D − A−3.2 [−4.9, −1.5]+0.6 [−1.1, 2.3]
GPT-4o judge, D − A−5.3 [−7.0, −3.5]−0.1 [−1.7, 1.5]

The static leader EditScore (D) trails the search leader EditReward (A) under both searchers, and four of five D − A gaps exclude zero under Best-of-N. Between N = 16 and N = 64 on nested pools, EditScore (D), VIEScore (D) and GPT-4o judge (D) lose 1.3 to 2.7 points with intervals excluding zero. Six other source-aware configurations gain 3.3 to 6.2 points (Table R3). The D − A gaps and budget declines also replicate on 100 general editing tasks from PIE-Bench (Table R8). The quality gaps behind the rank shifts are therefore stable at the configuration level.

We hope these responses address the reviewer's concerns, and we would be glad to discuss any remaining questions.

AuthorsOpenReview ↗

Kind Reminder of the Discussion Period

Dear Reviewer EJ17,

Thank you for your time and effort in reviewing our paper. Your comments are valuable for improving its quality. As the author-reviewer discussion period will end in less than a day, we gently request that you read our responses and check whether your concerns have been addressed. In brief, the revision adds leave-one-family-out prediction (Table R4), hard-example mining and most-frequent-factor arms under identical budgets (Table R6), and paired OHQ intervals for key configurations and budgets (Tables R2 and R3). We would be glad to clarify any remaining questions before the discussion closes.

Thanks again for your time and help in improving the quality of this paper.

Best regards,

The Authors

Reviewer_g2x7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the reliability of image editing reward models under optimization pressure. The authors introduce EditStress-Bench, which evaluates 12 reward configurations using Best-of-N selection and iterative refinement. The experiments reveal a discrepancy between static reward evaluation and search-time performance. It further proposes factor-guided reward patching, which identifies factor-specific weaknesses and uses targeted preference training to improve reward models. Experiments show that targeted updates outperform random updates in terms of human-rated quality.

Critical strengths
  1. Practically relevant research problem. The paper investigates reward model reliability under actual search procedures rather than relying solely on static evaluation. The experiments cover multiple reward configurations, search strategies, and control settings.

  2. Actionable failure analysis. Beyond identifying reward model failures, the authors introduce factor-level diagnostics and targeted reward patching, demonstrating improvements in human-rated quality.

Critical weaknesses
  1. Concerns about the central motivation and evaluation methodology.

The paper uses cross-task reward–human quality correlation as its static evaluation metric, while Best-of-N selection requires ranking candidates within the same task. These metrics measure different capabilities. Therefore, the observed ranking reversal may reflect a mismatch between static evaluation and within-task ranking rather than optimization-induced failures.

I suggest evaluating within-task pairwise ranking accuracy or group-wise correlation to determine whether the ranking reversal persists. Furthermore, the joint confidence interval for the primary 12-configuration result includes zero (Table 6), raising concerns about the robustness of the main conclusion.

  1. Insufficient baselines and ablation studies.

The factor sensitivity prediction lacks comparisons with simple baselines. For instance, always predicting structural plausibility as the most enriched failure factor achieves the same 7/10 hit rate as the proposed method in the Best-of-N setting (Tables 2–3).

Additionally, targeted patching uses factor-isolated training pairs, whereas random patching does not impose the same constraints. Thus, the improvements may partly arise from cleaner supervision rather than the proposed factor-selection strategy. A random-factor baseline with identical factor-isolation constraints would help clarify the contribution.

  1. Presentation concerns.

The paper is not particularly easy to follow, and the figures lack clarity. Figures 2 and 3 contain excessively small text, crowded layouts, and insufficient visual hierarchy, making the experimental workflow and results difficult to interpret. The figures would benefit from substantial redesign and simplification.

Questions for authors
  1. Would the reported ranking reversal persist when using within-task pairwise ranking accuracy or group-wise correlation as the static evaluation metric?

  2. How do the authors interpret the ranking reversal given that the joint confidence interval includes zero and family-level aggregation yields near-zero rank transfer?

5 comments
AuthorsOpenReview ↗

Responses to Reviewer g2x7 (Part I)

To reviewer g2x7: Thanks for the constructive comments. Responses to specific comments are given below, and the corresponding revisions are highlighted in blue in the revised paper.

  1. W1 & Q1: Following the suggestion, we have tested whether the loss of rank transfer persists under within-task metrics. Three within-task static metrics are computed on human-rated candidates that were not chosen by any reward, namely the shared initial edit and the uniform random draws from the Best-of-N pools. The results under Best-of-N are presented below and in Table R1 (Sec. 4.2, definitions in Appendix B.6) of the revised paper.
Static metricρ(4)ρ(16)ρ(64)Joint 95% CI of ρ(64)Joint 95% CI of ρ(64) − ρ(4)
Cross-task SRCC (original)0.6780.238−0.280[−0.448, 0.063][−1.105, −0.252]
Within-task accuracy0.5940.140−0.315[−0.399, −0.042][−1.000, −0.448]
Above-median accuracy0.4550.3640.385[0.042, 0.490][−0.392, 0.175]
Group-wise SRCC0.5590.182−0.364[−0.524, −0.154][−1.105, −0.594]

The experimental results demonstrate that the loss of transfer is not an artifact of the cross-task metric. Under within-task accuracy and group-wise SRCC, transfer at N = 64 is negative with joint intervals excluding zero. Above-median accuracy compares candidates in the high-quality region that search reaches. Its positive transfer is consistent with our central claim that agreement over typical candidates does not predict selection among the high-scoring candidates favored by search (Sec. 1). Under iterative refinement, the decline from N = 4 to N = 64 is also significant for the other three metrics (Table R1).

  1. W1 & Q2: The joint intervals in Table 6 were reported in the initial version, and Sec. 4.2 accordingly describes the main result as a marked loss of positive rank agreement. Four pieces of evidence support this result.

(1) First, the decline itself is robust. The joint interval of ρ(64) − ρ(4) is [−1.105, −0.252] (Table R1).

(2) Second, the source-aware pool keeps negative joint intervals at N = 64 under both searchers, with −0.552 [−0.782, −0.115] for Best-of-N and −0.733 [−0.855, −0.164] for iterative refinement (Table 6).

(3) Third, family aggregation leaves only six units, yet transfer still falls from 0.771 at N = 4 to 0.029 at N = 64 (Table 7, Sec. 4.3). The positive agreement at small budgets is lost even at the family level.

(4) Finally and most importantly, paired OHQ differences do not depend on rank statistics. At N = 64, the static leader EditScore (D) selects outputs 7.0 points below EditReward (A) under Best-of-N, with a 95% interval of [−9.1, −4.8]. Within every source-aware family, the default setting has the higher static SRCC. Yet under Best-of-N it selects outputs 2.1 to 6.7 points below the alternate setting, with four of five intervals excluding zero (Table R2, Sec. 4.2 and 4.3).

For these reasons, all four analyses show that static agreement loses its predictive value for selected quality under search pressure.

AuthorsOpenReview ↗

Responses to Reviewer g2x7 (Part II)

  1. W2 (1): A constant predictor naming SP indeed reaches the same 7/10 hit rate under Best-of-N, as now stated in Sec. 5.2.1. The hit rate only checks the most enriched factor. The diagnostic predicts the full enrichment profile, which a constant predictor cannot provide. We have added a constant predictor and a factor prior, the mean enrichment over training configurations, as baselines in Table R4 (Sec. 5.1.2, Appendix D.6).
 Predictor (Best-of-N)LOCO MAELOCO ρsLOCO HitLOFO MAELOFO ρsLOFO Hit
Constant factor (SP)n/an/a7/10n/an/a7/10
Factor prior0.15650.8007/100.16490.7205/10
Sensitivity profile (ours)0.07660.8807/100.07770.8907/10

Sensitivity profiling halves the prediction error of the factor prior and gives the best factor ordering under both holdout schemes. The profile also selects scorer-specific patch targets, namely SP for InternVL-Reg (D), NP for EditReward (A) and HF for EditScore (D) in Table 4. A constant SP rule would select a different target for two of these three scorers, and Table R6 shows that the choice of factor changes the gain.

  1. W2 (2): Following the suggestion, we have added the factor-isolated random baseline (Table R6, Sec. 5.1.3 and 5.2.2, Appendix E.6). This arm keeps the single-defect constraint of targeted pairs but draws the defect factor at random. All arms use the same labeled pair pool, 512 new and 512 replay pairs, and the same trainable modules. Evaluation uses a 300-task subset of the patch holdout with a fixed seed. The results on EditScore (D) are shown below.
                 ArmΔOHQ [95% CI]Δ HF severe failure (pp)Targeted − arm [95% CI]
Random+0.3 [−0.6, 1.3]−1.3+1.8 [1.0, 2.5]
Factor-isolated random+1.1 [0.2, 1.9]−3.3+1.0 [0.3, 1.7]
Targeted (ours)+2.1 [1.3, 2.9]−11.7n/a

Isolation alone gains 1.1 OHQ points, so cleaner supervision does contribute. Selecting the diagnosed factor adds a further 1.0 point with an interval excluding zero, and HF failures fall by 11.7 percentage points only in this arm. The gain of factor-guided patching therefore depends on factor selection, beyond cleaner supervision.

  1. W3: We have redesigned Figures 2 and 3 to improve readability. Figure 2 now presents the protocol in three panels read from left to right: the shared baseline, budgeted search with Best-of-N and iterative refinement, and the independent human evaluation. The original Figure 3 is split into Figure 3 (factor audit and mean sensitivity) and Figure 4 (search-time enrichment and the sensitivity-enrichment association), each with its own caption. The minimum font size is increased from about 3 pt to about 5 pt. In addition, the D/A settings are now defined in Sec. 4.1. A Composition column in Table 1 lists the scoring setup of every configuration, so the main comparisons can be read without consulting the appendix. If the reviewer has further suggestions, we would be happy to refine the figures accordingly.
AuthorsOpenReview ↗

Kind Reminder of the Discussion Period

Dear Reviewer g2x7,

Thank you for your time and effort in reviewing our paper. Your comments are valuable for improving its quality. As the author-reviewer discussion period will end in about a day and a half, we gently request that you read our responses and check whether your concerns have been addressed. In brief, the revision adds within-task static metrics and paired OHQ intervals for the static-search mismatch (Tables R1 and R2), constant-predictor and factor-isolated random baselines (Tables R4 and R6), and redesigned Figures 2 to 4. We would be glad to clarify any remaining questions before the discussion closes.

Thanks again for your time and help in improving the quality of this paper.

Best regards, The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Our Responses & Paper Revisions

We sincerely thank all the reviewers for their valuable and constructive reviews. We appreciate that the reviewers found the search-time evaluation practically relevant (Reviewers g2x7, EJ17). The reviewers also recognized that the shared candidate pool and human oracle separate generator failures from reward misranking (Reviewer EJ17), and valued the separation of holistic quality assessment from defect annotation (Reviewer 298B). The actionable loop of diagnosis, intervention and renewed search was acknowledged as well (Reviewers g2x7, EJ17).

We have made our best effort to address the concerns and revise the paper accordingly. The major modifications are summarized as follows.

  1. Robustness of the static-search mismatch (Reviewers g2x7, EJ17): Table R1 (Sec. 4.2, Appendix B.6) adds within-task accuracy, above-median accuracy and group-wise SRCC as static metrics. Under within-task accuracy and group-wise SRCC, Best-of-N rank transfer at N = 64 is −0.315 and −0.364, with joint intervals excluding zero. Tables R2 and R3 (Appendix B.6) report paired OHQ differences between configurations and between budgets. At N = 64, the static leader selects outputs 7.0 points below the search leader under Best-of-N [−9.1, −4.8].

  2. Stronger reward models (Reviewer 298B): Table R5 (Sec. 4.3, Appendix B.7) adds GPT-6 Astra and Claude Opus 5.5 as judges. Their default settings rank first and second in static agreement but fall to 11th and 12th of sixteen configurations at N = 64.

  3. General editing beyond human-object interaction (Reviewer 298B): Tables R7 and R8 (Sec. 4.3, Appendix B.8) add 100 PIE-Bench tasks. The D/A gaps and budget declines replicate for four families, with all eight paired intervals excluding zero.

  4. Baselines for diagnosis and patching (Reviewers g2x7, EJ17): Table R4 (Sec. 5.1.2 and 5.2.1, Appendix D.6) adds a constant predictor, a factor prior and leave-one-family-out prediction. Under family holdout, the sensitivity profile reaches MAE 0.0777 against 0.1649 for the factor prior. Table R6 (Sec. 5.1.3 and 5.2.2, Appendix E.6) adds factor-isolated random, hard-example mining and most-frequent-factor arms under identical budgets. For EditScore (D), targeting the diagnosed factor adds 1.0 OHQ point over factor-isolated random selection [0.3, 1.7].

  5. Clarity (Reviewers g2x7, 298B): The D/A settings are defined in Sec. 4.1, and Table 1 gains a Composition column. The introduction adds references for search-time uses of rewards. Figure 2 is redesigned, and the original Figure 3 is split into Figures 3 and 4 with larger text. The conclusion (Sec. 6) summarizes the new evidence.

The major revised contents in the manuscript are highlighted in blue. Point-for-point responses to specific comments are given in the following reviewer-specific responses. We welcome any further discussions and will address any remaining concerns.

AuthorsOpenReview ↗

Summary of the Discussion for the AC

Dear Area Chair,

We thank the AC and all reviewers for their time and engagement. As the discussion period closes, we briefly summarize its outcome. The full list of revisions is given in our earlier general response.

  1. Outcome of the discussion

(1) Reviewer 298B stated that the response resolved most initial concerns, citing the frontier-judge and PIE-Bench results, and raised the rating to Weak Accept.

(2) Reviewer g2x7 acknowledged the new within-task metrics, stronger baselines and improved presentation, noting that the within-task metrics strengthen the evidence for ranking degradation under search.

(3) The three questions of Reviewer EJ17 are answered with new results. The diagnosis holds when whole reward families are held out (Table R4), targeted patching outperforms both random arms under identical budgets (Table R6), and the quality gaps behind the rank shifts are stable in paired comparisons (Tables R2 and R3).

  1. Follow-up points and our responses

(1) The reversal is tied to factor-level blind spots that search amplifies (g2x7). It persists under within-task metrics that share the objective of Best-of-N (Table R1). Failure enrichment is measured against random selection from the same pool, so the candidate distribution is controlled. Factor sensitivity explains 77.0% of its remaining variation, and this link strengthens as the search budget grows (Table 17).

(2) Factor-guided patching names the repair target before any search (g2x7, EJ17). Its single-round OHQ is similar to hard-example mining and the most-frequent-factor arm. Hard-example mining converges on the same factor that the profile names in advance, and the diagnosis transfers to reward families held out from fitting (Tables R4 and R6).

(3) Patching gains hold across scorers and accumulate over rounds (298B, g2x7). The three patched scorers come from three families and have different target factors, and each improves over equal-budget random updates (Tables 3 and 4).

  1. Core contributions

(1) EditStress-Bench evaluates rewards by what they select under search. Selected outputs are scored by human raters, with shared candidate pools, a human oracle and independent defect annotation.

(2) Static agreement does not predict search-time quality. This finding holds under within-task metrics, for the frontier judges GPT-6 Astra and Claude Opus 5.5, and on 100 general editing tasks from PIE-Bench.

(3) Calibration sensitivity predicts search-time failures and guides their repair. It predicts which failures search selects, including for reward families held out from fitting, and guides patches that improve human-rated quality.

We hope this summary is helpful for the final assessment.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.