acceptodds
Under review as a conference paper at ICLR 2027

Spotting Is Not Inspecting: InspectBench for Coverage and Restraint

Abstract

Spotting one defect may flag an image, yet true inspection requires pinpointing them all. We tackle image-only defect inspection with vision–language models (VLMs), challenging a critic model to bound all visible defects or explicitly return an empty set for clean images. We present InspectBench, a cross-source benchmark designed to evaluate how models navigate the tension between defect coverage and false-positive restraint. The dataset spans 1,039 cross-reviewed images (321 clean controls) with 1,983 annotated regions across textual and visual defects, evaluated via a deterministic scoring protocol that penalizes duplicate hits and false alarms. Alongside the benchmark, we develop a reference inspection framework that combines content inventory, multi-scale evidence, and candidate verification, together with fine-tuned critic baselines spanning different supervision targets. Evaluating 17 model–procedure configurations across direct prediction and multi-stage inspection reveals a sobering baseline: region recall peaks below 34% at a lenient 0.1 localization threshold, leaving even the top-performing setup unable to touch 46.8% of defect regions. Furthermore, while selective regional review cuts false positives for some models, it backfires on others. These results expose a critical gap between high-level flaw detection and granular visual inspection, proving that evaluating coverage without false-alarm restraint gives a misleading sense of progress. Code, data, and benchmark tools will be released subject to licensing permissions.

Primary area:
datasets and benchmarks
Keywords:
Vision-Language Models, Generated Image Evaluation, Visual Artifact Detection, Defect Localization, Benchmarking

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WK9tOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses a practical yet underexplored problem in visual defect inspection using vision-language models (VLMs): comprehensive defect localization while maintaining restraint on defect-free images. Unlike conventional defect detection tasks that focus primarily on determining whether an image contains a defect, real-world inspection requires a model to identify all visible defects while avoiding false alarms on clean images. The paper argues that recognizing the presence of a defect is fundamentally different from conducting a comprehensive inspection. To investigate this problem, the authors introduce InspectBench, a benchmark comprising 1,039 images, including 718 defective images and 321 clean controls, with 1,983 annotated defect regions. The paper also proposes an ownership-based region-set scoring protocol to evaluate defect localization. Unlike conventional one-to-one matching, the proposed protocol allows a single prediction to cover multiple neighboring ground-truth regions while ensuring that each ground-truth region is credited to at most one prediction. It distinguishes among missed regions, weak overlaps, accepted detections, duplicate predictions, and false positives. Using InspectBench, the authors evaluate 17 model and inference-procedure configurations. The paper contributes primarily through benchmark construction, evaluation protocol design, and systematic empirical analysis, rather than through the development of a fundamentally new VLM architecture.

Critical strengths

1 Well-Motivated Task Definition with High Practical Relevance: The paper sharply identifies a blind spot in current VLM-based generated image evaluation and defect detection—namely, that image-level flagging does not equal fine-grained inspection. This holds significant practical value for image editing, AIGC quality control, and retouching.
2 Solid and Multi-Source Heterogeneous Dataset Construction: INSPECTBENCH integrates generated images (e.g., GPT-image-2, SenseNova-U1.5) with public datasets (ArtiBench, SalArt-VQA, etc.), encompassing 1,983 fine-grained defect region annotations across text and visual modalities. It underwent rigorous multi-annotator cross-review and arbitration, ensuring high data quality.
3 Meticulous and Comprehensive Evaluation Protocol: Unlike traditional simple Box mAP, the proposed region-set scoring protocol balances flawed-image scores and clean-image restraint using the weight parameter. Its lexicographic maximum mechanism cleverly resolves multi-region overlapping, ownership assignment, and geometric penalties.
4 Extensive Empirical Evaluation and Rich Comparison Dimensions: The paper evaluates 17 configurations (spanning commercial closed-source models, base open-source models, multiple SFT strategies, and multi-stage pipelines) and provides thorough bootstrap confidence intervals and ablation analyses, along with complete supplementary materials for reproducibility.

Critical weaknesses

1 Trade-off Between Performance Gains and Computational Cost: While multi-stage pipelines (e.g., Full-Seq, Full-Sched, Selective) show improvements on certain metrics, they introduce complex agentic procedures (multiple crops, global/local scouting, candidate verification), leading to exceptionally high computational and time overheads without achieving Pareto dominance across both recall and false-positive axes on all checkpoints.
2 Limitations in the Definition and Distribution of Clean Controls: As acknowledged by the authors in the limitations, clean images originate from three specific sources and are not strictly content-matched with flawed images. Part of the high clean FPR observed in some models may stem from sensitivity to unknown natural or art styles (mistaking artistic styles for defects); the diversity of styles could be further enhanced.
3 Subjective Biases in Geometric and Ownership Protocols: Hyperparameters in the scoring protocol are fixed conventions. Although justified by the authors, such hardcoded geometric penalties and ownership assignments may still occasionally conflict with human expert intuition in extreme scenarios involving dense small objects or complex text clumping.

Questions for authors

See Weakness

2 comments

Thank you for recognizing the benchmark's value and the distinction between image flagging and comprehensive inspection. We address the three limitations with analyses of existing predictions. We have uploaded a revised version of our paper, with all major changes highlighted in blue.

W1. Cost: procedure gains are checkpoint-dependent trade-offs.

We agree that multi-stage inspection is expensive and does not universally dominate Direct. Selective changes clean FPR by -33.33 percentage points for Trajectory-R2, but +11.21 for Workflow-SFT and +15.26 for Decision-SFT. This supports the paper's central distinction: adding an inspection procedure does not reliably improve coverage and restraint together.

Paired comparisons quantify the trade-off. For Workflow-SFT, Full-Sched versus Direct raises recall by 9.03 points (95% CI [6.55,11.57]), while FPR changes by +3.12 points [-1.25,7.48]. Selective versus Full-Sched lowers recall by 2.37 points [-4.25,-0.44] and raises FPR by 8.10 points [3.43,12.46]. The latter is worse on both axes for this checkpoint. These intervals use 10,000 paired image-bootstrap replicates conditional on fixed predictions (Appendix H.4).

Historical Full-Seq/Full-Sched/Selective runs require 51.70/26.39/14.97 requests per image and 11.18/3.69/1.72 batch hours (H.5). They quantify overhead without establishing an advantage at matched compute. The contribution is the benchmark and diagnosis of these operating points; component-level causal or universal efficiency gains are not established.

W2. Clean controls: high FPR also occurs on generated images.

For Qwen3.6-27B/Direct, clean FPR is 98.20% on GPT-image-2, 95.00% on SenseNova-U1.5 and 93.64% on natural controls. Thus high false-alarm rates are not confined to the natural subset. For Workflow-SFT/Direct -> Decision-SFT/Direct, the corresponding rates are 43.24% -> 9.01%, 54.00% -> 21.00%, and 10.91% -> 3.64%. Removing natural controls still gives 48.34% -> 14.69% FPR on 211 generated clean images (H.3).

This rules out natural-image recognition as the sole explanation for the reduction, but does not establish content-matched generalization. Recall changes by source too: 9.02% -> 5.64% on GPT-image-2, 30.63% -> 36.38% on U1.5, and 24.69% -> 20.52% on Public-4. We therefore interpret the result as source-dependent coverage/restraint changes. Style sensitivity and suppression on particular subsets remain possible. FPR is conditional on cross-reviewed reference labels.

W3. Geometry: sensitivity checks support the coverage gap, not a universal human-utility metric.

We confirmed that an enclosing box can take ownership from a tight prediction and sharply lower its credit. On actual outputs, quality-first ownership changes recall by at most 0.66 percentage points and S by 0.180, while preserving all 17 S ranks. Alternative class-agnostic one-to-one matching gives maximum recall 40.39% at IoU >=0.1 and 19.92% at >=0.5 (H.1). The broad coverage deficit therefore survives alternative matching.

Changing the area coefficient from 4 to 2/8 gives maximum recall 37.67%/30.26%; grouping-exponent variants give 30.96-34.54%. S-rank correlations remain >=0.998 for these variants. However, removing the area penalty raises maximum recall to 53.86%, and changing clean/flawed aggregation weights changes the winning configuration (H.2). The precise recall ceiling and scalar ranking are therefore protocol-dependent; S is one explicit utility convention alongside recall/FPR.

The strongest spatial conclusion is independent of those conventions: every configuration leaves at least 43.97% of annotated regions with no predicted overlap. Dense-object and text grouping may still disagree with expert judgment. We have corrected the wording that implied the 0.1 quality threshold was uniformly permissive.

The authors partially address my concerns. Considering the novelty and contribution of the paper, I will keep my score.

Reviewer_18nQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces INSPECTBENCH, a benchmark for image-only defect inspection. A model must return bounding boxes covering visible defects or an empty set for a clean image. The benchmark contains 1,039 images, including 321 clean controls, with 1,983 annotated defect regions across five families. Its principal methodological contribution is an ownership-based scoring protocol that permits one prediction to cover multiple defects while penalizing duplicate predictions and false alarms. The authors evaluate 17 model–procedure configurations, including direct prediction, several multistage inspection procedures, and fine-tuned critics. The main finding is that high image-level sensitivity does not imply comprehensive localization or reliable restraint. The highest reported region recall is 33.8% at a custom quality threshold of 0.1, while several sensitive models flag most clean images. Multistage inspection and fine-tuning produce different recall–false-positive trade-offs rather than consistent improvements on both dimensions.

Critical strengths
  1. The evaluation problem is meaningful and concretely motivated.

  2. Joint reporting of coverage and false alarms exposes important differences between systems.

  3. The spatial diagnostics provide evidence beyond the proposed composite metric.

  4. The paper identifies a reasonably clear contribution relative to nearby work.

Critical weaknesses
  1. The custom metric has a consequential ownership pathology, and its empirical validity is insufficiently established. Containment takes precedence over localization quality when assigning ownership. Consequently, a broad prediction can displace an almost exact prediction and effectively erase credit for an already localized defect. A concrete example is checked using the published equations: let one ground-truth region occupy 1% of the image. A prediction contained inside it and covering 99% of its area receives approximately 0.9965 credit. Adding a fully enclosing prediction with four times the ground-truth area transfers ownership to that broader box and reduces region credit to approximately 0.0056. Thus, for this single-region example, R@0.1[email protected] falls from 100% to 0%, although the accurate prediction remains present. The broad box itself has IoU 0.25. Section 7 acknowledges the ownership issue, but does not establish how often it affects actual outputs. This matters because the metric can conflate redundant predictions with failures of coverage, and its 0.1 threshold is not uniformly permissive about box scale. The synthetic conformance tests verify specified behavior, not agreement with inspection utility. A targeted remedy is to quantify ownership reversals on existing predictions and compare results under quality-first ownership or a conventional matching metric. Sensitivity to the area penalty and aggregation weights would further establish whether the conclusions depend on evaluator conventions.

  2. The clean-control mixture and training-label construction confound the interpretation of restraint. Public-4 contributes 499 flawed images and no clean images; natural images contribute 110 clean images and no flawed images. Clean controls are explicitly not content-matched to flawed images. Consequently, pooled clean FPR cannot distinguish transferable defect restraint from behavior associated with source or content. This concern is especially relevant to Decision-SFT: its Wikimedia/Pinterest negatives are assigned clean labels by construction, despite having been flagged by a prescreener. Those flags do not prove that the images contain defects, but discarding them without a reported human audit leaves negative-label reliability unresolved. The observed reduction in both recall and false positives is compatible with learning broader suppression. The authors acknowledge these limitations, and the fixed mixture still supports descriptive comparisons. However, stronger interpretation requires source-specific clean FPR and flawed-image recall, a generated-only analysis, and a bounded audit of the construction-assigned negatives.

  3. Annotation agreement does not establish the completeness required by the benchmark’s central objective. Acceptance requires equal region counts and union-mask IoU above 0.90. These conditions do not establish agreement on individual defect instances, and two annotators can agree while both missing a subtle defect. This is particularly consequential when the benchmark aims to measure whether a model finds all visible defects and treats predictions on clean images as false alarms. Moreover, 390 of 1,429 candidates—27.3%—are discarded, and acceptance varies substantially across sources and workload density. This creates a selection concern: the retained benchmark may underrepresent ambiguous or difficult inspection cases. The direction and magnitude of that effect remain unknown. A fresh, independent annotation audit on a stratified subset should estimate missed defects and instance-level agreement. Blind adjudication of sampled model false positives would also test whether the clean labels support the intended interpretation.

  4. The procedure experiments establish operating-point differences, but provide limited evidence about what improves inspection. Comparisons within a checkpoint are useful, but Direct and multistage procedures differ in inference budget, image views, candidate generation, and verification. Selective additionally changes routing and verification shortcuts. The results therefore do not isolate whether inventory construction, additional visual evidence, verification, or simply more computation produces the observed changes. The manuscript appropriately describes these comparisons as descriptive, so this is a limitation of the methodological contribution rather than a contradiction. Nevertheless, the proposed framework would be more informative with measured calls/tokens or latency and one simple comparator given a similar inference budget—for example, repeated direct inspection with consolidation. Recall at comparable false-positive rates would help distinguish improved inspection from increased willingness to report defects.

  5. Important reproducibility and evaluation-separation details remain unresolved in the supplied manuscript. Appendix C provides some training information, but the PDF does not give complete inference prompts, decoding settings, image-resolution policies, or training configurations for all checkpoints. Balanced+ and Pool+ are described mainly by scale and balance, and the trajectory checkpoints are also incompletely specified. The paper states that Decision-SFT training and validation images are disjoint, but does not clearly document disjointness from INSPECTBENCH or the duplicate-checking procedure. Shared generator families do not establish leakage; the issue is missing evidence of evaluation separation. The anonymous repository could not be accessed during this review, so I cannot determine whether it resolves these gaps. Exact configurations, split manifests, and actual per-image predictions would make the central results substantially more auditable.

Questions for authors
  1. How frequently does containment-first ownership assign a region to a substantially worse prediction than another available candidate? How do recall and configuration rankings change under quality-first ownership or conventional matching?
  2. What evidence validates the scorer against human judgments of useful inspection? In particular, are conclusions stable under reasonable changes to the area penalty, grouping tolerance, and clean/flawed aggregation weight?
  3. Do Decision-SFT’s restraint gains persist separately on generated clean images and natural clean images? Please provide source-specific recall/FPR and an audit of a sample of construction-assigned training negatives.
  4. How complete and reproducible are the annotations under independent reinspection? What fraction of sampled model false positives are judged to be genuine defects by reviewers blinded to the model identity?
  5. What are the actual inference costs of the multistage procedures, and does their benefit persist against a comparator with a similar budget? A focused comparison on an existing checkpoint would be sufficient.
  6. Can you provide benchmark-disjointness checks, full evaluation configurations, and paired uncertainty estimates for the main comparisons? Because configurations use the same images, paired bootstrap intervals for score differences are more appropriate than using a typical individual-score interval half-width as a comparison rule.
1 comment

Thank you for the concrete counterexample and targeted suggestions. We reanalysed all 17 saved prediction sets and reproduced the submitted scores. We have uploaded a revised version of our paper, with all major changes highlighted in blue.

W1/Q1. Ownership: limited observed impact, but a real pathology. Your example is correct: credit falls from 0.9965 to 0.0056 although the tight box remains. Per configuration, 3-50 of 1,983 regions choose an owner with lower pre-group pair quality than an alternative; 1-18 have a gap >=0.1. Quality-first election with group recomputation changes recall by at most 0.66 percentage points and S by 0.180; all 17 S ranks remain unchanged. This includes opposing effects: Full-Seq rescues 20 regions above threshold but loses 14. It bounds the observed impact without proving monotonicity.

Class-agnostic one-to-one IoU matching yields maximum recall 40.39% at IoU >=0.1 and 19.92% at >=0.5. Every configuration leaves >=43.97% of reference regions with no predicted overlap. Thus incomplete coverage persists independently of the ownership rule (Appendix H.1).

Q2. Validity: exact scores depend on conventions. The 0.1 cutoff is area-penalized quality, not IoU or uniformly lenient overlap. Changing area coefficient 4 to 2/8 yields maximum recall 37.67%/30.26%; grouping exponents 0/0.25/0.75 yield 30.96-34.54%. These variants preserve S ranks closely (Spearman >=0.998). Removing the area penalty raises maximum recall to 53.86%; changing clean/flawed weights changes the winner. Therefore the 33.8% ceiling is protocol-specific. We report recall/FPR and no-overlap rates alongside S. (H.2).

W2/Q3. Sources: gains persist on generated controls. For Workflow-SFT/Direct -> Decision-SFT/Direct, FPR is 43.24% -> 9.01% on GPT-image-2, 54.00% -> 21.00% on U1.5, and 10.91% -> 3.64% on natural controls. Removing natural controls gives 48.34% -> 14.69% FPR; recall on all flawed images falls from 24.36% to 23.25%. Per-source recall is 9.02% -> 5.64%, 30.63% -> 36.38%, and 24.69% -> 20.52% on GPT-image-2, U1.5 and Public-4. Restricting both classes to GPT-image-2/U1.5 gives recall 23.92% -> 26.84%. Natural-image recognition alone cannot explain the reduction, but content imbalance and broader suppression remain possible. (H.3; C.3).

W3/Q4. Annotation: reference coverage, not exhaustive truth. Equal counts and union-mask IoU >0.90 are acceptance criteria, not instance-agreement or completeness estimates. Excluding 390/1,429 candidates (27.3%) may select against ambiguous cases. We explicitly limit coverage/FPR to retained, cross-reviewed reference labels. This leaves annotation completeness open, while omissions of already annotated regions remain directly measurable (Appendix H, opening paragraph).

W4/Q5. Compute: descriptive comparisons. Archived Full-Seq/Full-Sched/Selective runs used 51.70/26.39/14.97 requests per image and 11.18/3.69/1.72 batch hours at common 64/32 request/image concurrency. They quantify expense but establish neither a matched-budget advantage over repeated Direct nor recall at matched FPR (H.5).

W5/Q6. Reproducibility and paired uncertainty. The prepared scoring supplement contains normalized annotations, all 17 per-image prediction sets, the scorer, analyses and checksums, plus available Direct settings/prompt code. Complete historical inference and training configurations for every checkpoint remain unavailable.

All 4,000 Decision-SFT train, 800 validation and 1,039 benchmark files were checked: zero cross-split path/SHA-256 matches; zero train/validation-to-benchmark candidates under a 63-bit perceptual-hash screen at Hamming distance <=4. This supports separation for the audited data, not other checkpoints, pretraining or all transformed duplicates (H.6).

We replace the individual-CI-half-width rule with 10,000 paired, clean/flawed-stratified image-bootstrap replicates. S differences (95% CI): Workflow Full-Sched minus Direct +3.39 [1.82,4.99]; Full-Sched minus Full-Seq +0.35 [-0.96,1.63]; Workflow Selective minus Direct +0.90 [-0.77,2.57]; Trajectory-R2 Selective minus Direct +3.65 [2.20,5.11]; Decision Selective minus Direct +0.98 [-0.74,2.76]. Only the first and fourth intervals exclude zero; these fixed-prediction intervals neither capture training variability nor establish equivalence (H.4).

Reviewer_tf1tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces INSPECTBENCH, an image-only benchmark that jointly evaluates comprehensive defect localization and false-positive restraint.

Critical strengths
  1. INSPECTBENCH jointly evaluates defect coverage and false-positive restraint, addressing an important limitation of recall only evaluation.

  2. The benchmark includes cross reviewed multi-defect images and clean controls, supported by a deterministic scoring protocol and extensive baseline evaluation.

Critical weaknesses
  1. SDG[1] already supports variable-length defect grounding and clean-image outputs. Although the introduction motivates the need to balance defect coverage and false positives, it does not clearly explain what qualitatively new problem or insight is enabled by the image-only setting and the proposed coverage–restraint protocol.

  2. I am concerned that the clean FPR may be affected by source bias. Since all natural images are used as clean controls, models may exploit source-specific cues to output an empty set instead of genuinely determining that the images contain no defects.

  3. The inspection procedures are not matched in model calls or inference cost, so the gains of multi-stage inspection may result from additional computation rather than the procedure itself.

  4. Figure 2 contains prominent black borders on both sides, which reduces readability and should be corrected with a clean vector-based rendering.

[1] Zhang H, Yu H, Zhang Y, et al. Where, What, Why, and Importance: Structured Defect Grounding for Text-to-Image Feedback[J]. arXiv preprint arXiv:2606.06113, 2026.

Questions for authors

See the Weakness.

1 comment

Thank you for recognizing the value of joint coverage and restraint evaluation. We have uploaded a revised version of our paper, with all major changes highlighted in blue.

W1. Contribution relative to SDG.

We agree that variable-length defect sets and clean outputs are already supported by SDG. Our contribution is the evaluation scope and empirical diagnosis, rather than a new output schema. Image-only inspection asks which defects are identifiable from pixels without the generation prompt: broken glyph structure can be judged, whereas a readable word differing from an intended string requires that string. SDG also addresses prompt-conditioned misalignment. InspectBench focuses on the former setting, including images received without their prompts.

The benchmark tests comprehensive inspection on 455 multi-region flawed images out of 718, together with 321 clean controls. Group-aware credit accommodates predictions covering neighboring defects; separate diagnostics expose missed regions, weak localization and false alarms. We do not claim grouping is universally preferable to one-to-one matching.

The substantive finding is that image-level detection can coexist with severe coverage gaps and poor restraint: five configurations flag at least 97% of flawed images but also 80-96% of clean images. Every configuration leaves at least 43.97% of annotated regions without any predicted overlap, independently of ownership, area penalties or composite-score weights. Thus the inspection deficit extends beyond the proposed scalar metric. These results explain why image flagging or recall alone is insufficient (Section 6; Appendix H.1).

W2. Source bias in clean FPR.

The concern is valid, but natural-image recognition alone cannot explain Decision-SFT's FPR reduction. Source-specific results are below; each cell reports Workflow-SFT/Direct -> Decision-SFT/Direct.

SourceClean FPRRegion recall
GPT-image-243.24% -> 9.01%9.02% -> 5.64%
SenseNova-U1.554.00% -> 21.00%30.63% -> 36.38%
Public-4No clean images24.69% -> 20.52%
Natural10.91% -> 3.64%No flawed images

Removing natural controls gives FPR 48.34% -> 14.69% on 211 generated controls. Restricting both clean and flawed images to GPT-image-2/U1.5 gives recall 23.92% -> 26.84%, with the same FPR reduction. Both generated sources contain both labels, so identifying a source cannot by itself identify the label. Nevertheless, source matching is not content matching, and these aggregate results do not rule out suppression on particular subsets. The supported conclusion is source-dependent coverage/restraint changes, not universal improvement in defect understanding (Appendix H.3).

W3. Inference cost and attribution.

We agree that additional compute is a confound. Historical Full-Seq, Full-Sched and Selective runs used 51.70, 26.39 and 14.97 requests/image, with batch wall times of 11.18, 3.69 and 1.72 hours. These are archived 1,054-image runs at common request/image concurrency limits of 64/32, not synchronized timings of the final 1,039-image evaluation (Appendix H.5).

The justified result is a change in operating point. For Workflow-SFT, Full-Sched versus Direct raises recall by 9.03 percentage points (paired 95% CI [6.55,11.57]), while FPR changes by +3.12 points [-1.25,7.48]. Selective versus Full-Sched loses 2.37 recall points [-4.25,-0.44] and adds 8.10 FPR points [3.43,12.46] (Appendix H.4). These fixed-prediction comparisons show that procedure choice affects coverage and restraint differently. They do not isolate a benefit per unit compute or superiority over an untested repeated-Direct baseline. The paper presents benchmark diagnostics and descriptive trade-offs, with no claim of compute-matched procedural superiority.

W4. Figure 2.

We agree that the black side borders impair readability. We will crop them while preserving the original diagram and caption.

Comments on the paper

1 comment
Area_Chair_c4VLOpenReview ↗

[Open Discussion] Thank you for your review and please check authors' rebuttal

Dear All Reviewers,

We have received mixed opinions on this paper. Thank you all for your thoughtful reviews and constructive feedback. Currently, it is still in the open discussion phase, so reviewers are welcome to have further discussions as well as the authors.

Could you please review the authors' rebuttal and share your updated opinions on whether your concerns have been adequately addressed? Please also let us know if you have any remaining questions or concerns that require further clarification.

Dear Reviewer WK9t,

You provided a positive assessment of this paper. Could you please confirm whether the authors' rebuttal has adequately addressed your remaining concerns, such as inference costs, clean-control biases, and the evaluation protocol?

Dear Reviewers tf1t and 18nQ,

You both expressed reservations about this paper. Could you please check whether the authors have provided sufficient evidence and clarification to address your concerns, including the benchmark's novelty, potential source biases, and the fairness of inference-cost comparisons?

We would also appreciate your continued discussion with the authors on any substantive concerns that remain unresolved.

Thank you all again for your time and valuable contributions to the review process.

Best regards,
AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.