acceptodds
Under review as a conference paper at ICLR 2027

Edit Discovery in Human–AI Interactions

Abstract

Human–AI interactions are becoming a rich source of data for improving AI systems: while many model responses are already satisfactory, others expose failures that could become valuable supervision if identified and corrected. We study this setting through the notion of an editor: a more capable evaluation process that can inspect a response and either confirm it or modify it, but whose invocation is costly. The editor may be a human expert, a stronger AI system, or the same model equipped with additional computation, context, tools, or information. The key challenge is to decide which interactions to send to the editor before knowing whether an edit will actually be made, since invoking the editor itself consumes limited resources such as expert time, inference cost, latency, or tool-use overhead. We formulate this problem as Edit Discovery Optimization (EDO), which seeks to minimize edit opportunities missed among skipped interactions while controlling unnecessary confirmations and respecting an editor-cost budget. We characterize the population-optimal policy and show that it takes the form of a cost-adjusted threshold on the posterior probability of an edit. Building on this structure, we develop EDIT (Edit Discovery via Iterative Thresholding), an online algorithm for the partial-feedback setting in which edit labels are revealed only for selected interactions. EDIT combines an adaptive edit score, a confirmation-control threshold, and a cost-aware selection rule; it satisfies the editor budget at every round and controls the average confirmation rate up to a vanishing finite-horizon term, without distributional assumptions on the interaction stream. Across four reasoning and factual question-answering benchmarks, EDIT achieves a better trade-off between editor cost, Type-I error, and Type-II error than direct LLM-based batch selection.

Primary area:
foundation or frontier models, including LLMs
Keywords:
human–AI interaction, selective human review, edit discovery, online learning, cost-sensitive data selection, large language models

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_7Z2VOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper explores a mechanism for finding correction opportunities, i.e., predicting when more expensive (human, stronger llm) reviews are needed to supervise LLM outputs (e.g., active learning). They define EDO which chooses a selection policy that minimizes false omission rate (related to Type II error) within the constraints of some fixed edit budget and false discovery rate (related to Type I error) thresholds. The paper explores detection of likely edits, not necessarily of examples with the greatest downstream value (which motivates the work but is not demonstrated as far as I can tell). They propose an online method, EDIT, that approximates this structure using an edit score, a feedback-updated confirmation threshold, and a cost multiplier, which comes with deterministic cost guarantees and They conduct experiments on four QA benchmarks with Llama 3.3-70B across 4 edit score approaches and report improved discovery over prompted LLM selection, especially with an online linear probe learning over frozen representations. The finding that a 22.7M-parameter online probe strongly outperforms the 70B model's self-assessment is potentially interesting.

Critical strengths

(1) Intuition for EDO objective is clear and EDIT offers guarantees: high edit probability makes review more likely, global thresholds exclude low-impact reviews, and more expensive items require stronger evidence of a recommended edit. Using interaction-specific costs on generated responses makes the formulation relevant to real review situations. This problem definition/formulation and operationalization of resource constraints is an interesting contribution even if several mathematical components have close precedents.

Critical weaknesses

(1) Experimental conditions seem to favor proposed method. Resampling the responses on a model that can learn/adapt (EDIT-REP) means that it can possibly memorize previously labeled items (e.g., 2,500 draws from MATH-500's 500 items). There is no clear justification for this choice, which biases the main method claims. Additionally, choosing epsilon so the bound matches LLM-SELECT's error and thus does not itself ensure matched actual cost and pooled confirmation precision (and makes it a weaker baseline, and its prompt prohibits solving questions, likely weakening its ability to identify incorrect answers). It isn't clear that the methods generalize across LLMs, and Llama seems to avoid weirdness shown int Fig. 6/Tab 1 with Qwen shows that epsilon = 0 for PopQA (meaning a policy could move towards selecting nothing). Since eps ≈ 0.5 and BtB_t is half the batch in many settings, the alpha controller may not contribute much, and the gains may come mostly from the probe. Basically, results establish superiority over one, weak prompted selector rather than superiority over plausible/comparable alternatives to the proposed approach.

(2) Experiments/results do not seem to match claims. "Edits" are identified incorrect answers on benchmarks with reference keys, so the editor is effectively a grader and the task becomes online error detection on closed-form QA. While sort of mentioned in conclusion, the motivations (subjective/partial corrections, ambiguity or style, customer/user intent) are not tested. The sole human annotator sees the reference answer, so it validates grading, not editing. Main experiments use unit costs, and the downstream value of discovered edits is not explored. Thus the construction doesn't distinguish between minor substantive correction from valuable failures and it doesn't seem to capture redundancy among discovered edits. With only correctness on. benchmark QA, it is unclear whether EDIT's advantages hold where real-world edit labels are noisy, subjective, and non-binary, which is precisely where the problem is hardest and most needed.

Questions for authors

(1) If EDIT-REP is able to learn and adapt, why is the sampling done with replacement? What justifies the resampling that includes identical prompt-response pairs (risking an online probe possibly memorizing previously labeled items)? Can you provide direct evidence of EDIT-REP without this advantage? What are the exact response-pool sizes, duplicate frequencies, and first-occurrence results? Are the 200 audited examples excluded from the online pools? What ablations can you provide to show whether the controller contributes, or whether EDIT-REP simply benefits from being the only method that learns from feedback? Can you rerun without repetition and with multiple streams and CIs for all methods?

(2) Why are costs defined as equal/unit? Even length-adjusted costs may not account for the amount of work taken by an expert to verify a complex scientific multiple choice results compared to a long but more easily Google-able PopQA result. How would costs be estimated in practical settings where observed LLM length may be less related to the cost of verifying the task?

(3) How sensitive are the EDIT methods to the specific EDO described? E.g, how would the results be different if there were a bound on Type-II error, recall, or regret relative to the oracle policy (as these, for a given downstream need, may be important to cost)? Can epsilon be defined without invoking LLM-SELECT? How would such epsilon definitions affect the performance?

(4) Can you evaluate on at least one dataset with genuine, non-answer-key-based edits, e.g., code-review/post-editing datasets, user-correction in public chat logs, or a human-editing study without reference answers?

(5) With the same EDIT-REP score and learning protocol, how much improvement comes from the adaptive αt\alpha_t update versus simpler fixed-threshold or budget-only selection?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 7Z2V

We thank the reviewer for their time and helpful comments. We have expanded the experiments to address the concerns below.

1. Repeated interactions and adaptive selection

We acknowledge that repetitions can ease online learning. Given our distribution-free guarantees, we initially sampled with replacement to reduce the cost of long-horizon runs. We now report first-occurrence results (D.11, Table 9). More directly, we ran 2,500 distinct interactions without replacement on MATH-500 and PopQA, with both Llama-3.3-70B-Instruct and Qwen3-32B, averaging five orderings (Figs. 16–17). Importantly, EDIT-REP retains favorable Type-I–Type-II trade-offs without repeated examples. RAGTruth test streams are also nonrepeated (D.14, Figs. 21–22).

We added Top-K, Fixed Threshold, Online Conformal Selection, Conformal Alignment, stream active learning, and Thompson sampling (D.9, Figs. 12–13). All score-based methods share frozen representations, online logistic probe, initialization, and budget. Fixed Threshold ablates adaptive αt\alpha_t, retaining the learned scorer but fixing the threshold; Top-K tests score ranking under the same budget. These isolate the benefit of adaptive selection beyond scoring.

2. LLM-Select prompts and operating-point fairness

In response to the prompt-sensitivity question, we tested reasoning, verification, and risk-ranked LLM-Select prompts (D.12, Fig. 19; prompts in D.4.2). Type-I and Type-II errors are relatively insensitive to prompts despite cost differences; none consistently closes the gap to EDIT-REP.

ε\varepsilon can be specified independently of LLM-Select. It is a user-defined tolerance for unnecessary confirmations, chosen according to the application. We used LLM-Select only to choose a comparable operating point in the original experiments. EDIT's theoretical Type-I upper bound was approximately matched to LLM-Select's observed Type-I error. Neither EDIT nor its guarantees require LLM-Select. To demonstrate this, Fig. 18 sweeps ε\varepsilon independently and reports the achieved Type-I error. We also report Type-I–Type-II frontiers (D.9, Figs. 12–13), matched realized-cost comparisons (D.10, Fig. 14), and oracle-relative missed edits (Fig. 15).

3. Generalization across response models

To address model generalization, we also evaluate Mistral Small 3.2 24B (D.5, Fig. 7 and tables). EDIT-REP attains edit recall of 0.821, 0.648, 0.534, and 0.632 on MATH-500, MMLU-Pro, PopQA, and GPQA, versus 0.446, 0.492, 0.377, and 0.248 for LLM-Select. Improvements thus extend beyond the original response-model families.

4. Human relevance, edit labels, and review costs

Regarding the validity of correctness-based edit labels, we conducted a 200-example human audit (D.2, Table 4). As illustrated in Fig. 3, the annotator inspected the candidate response, reference answer, and solution steps, and judged whether the whole response (not only the final answer) required an edit. Agreement with benchmark-derived labels was 96.5%.

Beyond benchmark correctness, we added RAGTruth [1] (D.14, Figs. 21–22), using human-annotated hallucination spans in evidence-grounded QA, summarization, and data-to-text responses. On 450 held-out, nonrepeated responses, both EDIT-REP and EDIT-LLM achieve lower Type-II error than LLM-Select and EDIT-RAND with train-selected parameters (Fig. 21); EDIT-REP also shows favorable Type-I–Type-II frontier comparisons against additional selection baselines (Fig. 22).

For the cost question, we use unit costs to isolate selection from cost estimation and test nonuniform length-based costs as well (D.6, Figs. 8–9). We agree that length alone may not reflect verification difficulty. In deployment, costs could be predicted from timed expert reviews using task difficulty, domain, and verification requirements, following prior work on annotation-cost estimation [2,3]. These estimates can be refined as more reviews are collected and incorporated directly into EDIT.

5. Alternative constraints and robustness

A Type-II or recall constraint, as the reviewer suggests, would define a different optimization problem. EDO minimizes missed edits subject to control of unnecessary confirmations among reviewed items and a per-round budget; Theorem 2 guarantees average roundwise Type-I control up to a vanishing finite-horizon term, not Type-II control. On arbitrary streams, unobserved labels for skipped items preclude a nontrivial Type-II guarantee without additional assumptions. Oracle-relative missed edits (D.10, Fig. 15), matched costs (Fig. 14), and the independent ε\varepsilon sweep (D.11, Fig. 18) show these trade-offs. We test distribution shift (D.7 and D.13, Figs. 10 and 20). Please let us know if further clarification on the choice of constraint would be helpful.

We hope these results address the reviewer's concerns and would appreciate reconsideration of the assessment and score.

References are provided in a follow-up response.

AuthorsOpenReview ↗

Reference for our response to Reviewer 7Z2V

[1] Niu et al., RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models, ACL, 2024.

[2] Arora, Nyberg, and Rosé, Estimating Annotation Cost for Active Learning in a Multi-Annotator Environment, NAACL Workshop on Active Learning for NLP, 2009.

[3] Wei et al., Cost-aware Active Learning for Named Entity Recognition in Clinical Text, Journal of the American Medical Informatics Association, 2019.

Thank you for your response.

Repetition and memorization:

thank you for directly addressing my question and for the evidence from First-occurrence evaluation in Table 9 and Nonrepeated streams in Figures 16–17 showing EDIT-REP can identify problems before receiving labels for those exact interactions. As I understand it, the nonrepeated experiment changes the model (D.11 says EDIT-REP uses Qwen3-1.7B representations and probe learning rate 4.0, rather than the main MiniLM setup. That says EDIT-REP configuration can work without repetition but not necessarily that removing repetitions keeps the original lightweight setup advantage. With the first-occurrence stuff, we see an improvement in missed-edit behavior with substantially lower precision (Table 9, PopQA)

  • Could you clarify the "2,500 distinct interactions without replacement on MATH-500" which only has 500 items?
  • Could you explain how the observed pool is constructed, how duplicate observations are handles, and how that corresponds with the held-out audit mentioned?

Baselines and controllers

Thank you for directly responding to the request with Apx D.9. Sharing representations and the online probe is a substantial fairness improvement with the additional comparisons. They resolve my concerns with weak baselines.

However, I don't think they “isolate” the adaptive controller’s benefit or quantify the size/robustness of its effects, because

  1. selective feedback, different selection policies acquire different training labels and therefore develop different probes and thus measure the total effect of the combined selection-and-learning policy,
  2. best-frontier-point reporting (across hyperparamter searches) seems to obscure this (and also I'm not clear why we exclude the first 20 rounds for these comparisons if we're comparing deployment conditions),
  3. RAGTruth configurations are selected on the same test streams and the EDIT-REP curve is labeled “larger search.” I interpret this as a implying a exploratory frontier, not an unbiased held-out evidence over the baselines.

Questions:

  • Could you provide a non-exploratory statistical test that addresses the confounds in the combined effect and share the measured effect size and robustness of the controller?
  • could you provide rationale for and the comparability of dropping the first 20 rounds?

𝜀

Thank you for the clarification and comparisons in Figs 12-18. They clarify its role and measurement. This doesn't yet resolve my concern of 𝜀=0 affecting the adaptive update rule. The exact controlled metric and low-tolerance behavior seem to need more careful interpretation. Theorem 2 uses an unweighted average of roundwise confirmation fractions, not the pooled fraction across all reviewed items making your statement "tolerance for unnecessary confirmations" a necessary descriptor (instead of a guarantee on aggregate review precision, which I think would be more helpful and intuitive)

Relevance and construct validity

Thank you for the RAGTruth update and softens my "only answer-key-based grading is tested" concern. RAGTruth seems to be a materially stronger response. The annotator given agreement, however, doesn't directly answer my core construct-validity concern around what is represented by editing. The experiment converts those annotations into a binary label about the presence of any qualifying hallucination in the text span. It is still a test of evidence-grounded factual-correction identification, not actual post-editing or the value of collected corrections. To be clearer, my concern about construct validity arose becuase of the interpretation and tests that would demonstrate its performance with respect to correction severity, downstream utility, subjective editor disagreement, actual editing time or correction quality, etc.

So far, the new experiments improve empirical credibility. As I understand them, they do not make the approach or its utility conceptually more valid (e.g., based in actual edits) nor do they fully demonstrate the mathematical and practical superiority of components with close precedents. For the latter, I could settle for an unambiguous statistical test that positively and specifically attributes controller gains over the baselines based on the most realistic and comparable deployment condition, or, alternatively if there are null results, an explanation for the mechanism giving rise to observed successes and conditional statements and qualifications for its use. I think we're closer to what I would expect and hope one more update could hopefully show that.

Reviewer_aNHUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formulates Edit Discovery Optimization (EDO) and proposes EDIT, an online method for selecting AI responses for human correction under limited budgets. The method combines learned edit scores, adaptive thresholding, and cost-aware selection. Experiments on four QA benchmarks demonstrate improvements over direct LLM-based selection.

Critical strengths

The problem is practically relevant, and the proposed framework provides theoretical guarantees on the confirmation rate and per-round budget. The experiments cover multiple benchmarks and models.

Critical weaknesses
  1. Insufficient novelty relative to prior work.

Budgeted human review and selective escalation have been extensively studied. Closely related work includes confidence-based check-set selection [1], selective LLM evaluation [2], and online selection with limited feedback [3]. The paper does not sufficiently distinguish its contributions from these approaches.

  1. Weak and incomplete baselines.

The main baseline is direct LLM-based batch selection, while the remaining comparisons are variants of EDIT itself. Crucial baselines are missing, particularly Top-K/Top-B selection using the same learned edit scores, static thresholding, and existing online calibration methods.

[1] dela Cruz et al., Evaluating Large Language Models for Confidence-based Check Set Selection, ACL Findings 2025. https://aclanthology.org/2025.findings-acl.836/

[2] Jung et al., Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreement, ICLR 2025. https://arxiv.org/abs/2407.18370

[3] Gollapudi et al., Efficient Online Conformal Selection with Limited Feedback, 2026. https://arxiv.org/abs/2605.14953

Questions for authors
  1. How does EDIT compare against Top-B selection using the identical learned scorer, feedback, and budget?

  2. What is the specific technical novelty over existing online selection and adaptive thresholding approaches?

3 comments
AuthorsOpenReview ↗

Response to Reviewer aNHU

We thank the reviewer for their time and helpful comments. We have expanded the related work and experiments to address the concerns below.

1. Novelty relative to prior work

We appreciate the reviewer highlighting the recent works [1]–[3], which were missing from our original submission. We have now added and discussed them explicitly. They address related selection problems but differ from EDO in objectives, feedback, or guarantees. Due to the main-paper page limit, this discussion currently appears in Appendix A (Extended Related Work); if accepted, we will bring it into the camera-ready main text.

[1] dela Cruz et al. study how LLM confidence estimates and direct batch selection identify predictions worth checking. Their methods neither learn from selectively revealed edit labels nor control the fraction of unnecessary confirmations. EDIT learns an edit score online and adjusts selection under a hard per-round budget. These approaches are complementary: Theorem 2 applies to any score in [0,1][0,1], so confidence estimates from [1] could serve as EDIT's score. Top-K and Fixed Threshold further compare selection rules using the same scorer.

[2] Jung et al. calibrate when to trust an LLM judge's pairwise preference, controlling disagreement among judgments not escalated to humans. EDO instead controls unnecessary confirmations among items sent for review. A guarantee on the trusted set neither bounds wasted effort on the reviewed set nor enforces a review-cost budget. EDIT also requires no separate labeled calibration set, and its online guarantee applies to arbitrary realized interaction streams. We additionally evaluate Conformal Alignment [4].

[3] Gollapudi et al. study online subset selection with limited feedback, minimizing cost while seeking a set containing at least one successful option. EDO instead penalizes each missed edit and unnecessary confirmation: reviewing one useful item alongside several unnecessary ones may satisfy [3]'s success condition but waste the editor's budget. We now include their Online Conformal Selection method in our experiments.

Technical novelty. Our main technical contribution is a cost-aware online selection framework that addresses missed edits, unnecessary confirmations, and limited review budgets under selectively observed feedback. Unlike the reviewed approaches, EDO explicitly minimizes missed edits while controlling the fraction of unnecessary reviews among selected items and respecting heterogeneous costs. Theorem 1 characterizes the population-optimal cost-aware policy, motivating EDIT's rule η~t(z)>αt+βtc(z)\widetilde\eta_t(z)>\alpha_t+\beta_t c(z) rather than a fixed threshold or Top-K ranking. In the online setting, edit labels are observed only for selected interactions. EDIT couples learning from this censored feedback with an adaptive threshold αt\alpha_t and budget-feasible selection. Theorem 2 then establishes a hard per-round budget and finite-horizon, distribution-free control of average roundwise Type-I error for any realized stream, without exchangeability assumptions or a separate calibration set. The first result characterizes efficient selection; the second makes online control possible despite selective feedback.

2. Additional baselines and experimental comparisons

We added Online Conformal Selection [3], Conformal Alignment [4], stream active learning [5], Thompson sampling [6], Top-K, and Fixed Threshold (Appendix D.9, Figs. 12–13). All score-based methods use identical frozen representations, online logistic probes, initialization, and per-round budgets; their selection rules determine the observed feedback. This directly addresses the requested Top-B comparison with the same scorer and budget. EDIT-REP generally achieves lower Type-II error at comparable Type-I error across four benchmarks and two response models. Fixed Threshold isolates the benefit of adapting αt\alpha_t, while Top-K tests whether score ranking alone is sufficient.

We further compare methods at matched realized editor cost (D.10, Fig. 14) and against a same-cost edit-aware oracle in missed edits per round (Fig. 15). This checks that gains are not merely due to spending more budget. First-occurrence and fully nonrepeated streams (D.11, Table 9; Figs. 16–17) show that favorable trade-offs persist without repeated examples; Fig. 18 evaluates achieved Type-I error across requested tolerances. Stronger LLM-Select prompts (D.12, Fig. 19) do not consistently close the gap, and shift experiments (D.13, Fig. 20) test robustness. We also add a third response model, Mistral Small 3.2 24B (D.5, Fig. 7), and RAGTruth human-annotated hallucination detection (D.14, Figs. 21–22).

We hope these clarifications and additional experiments address the reviewer's concerns about novelty and baseline comparisons. In light of these revisions, we would be grateful if the reviewer could reconsider their assessment and score.

References are provided in a follow-up response.

AuthorsOpenReview ↗

References for our response to Reviewer aNHU

[1] dela Cruz et al., Evaluating Large Language Models for Confidence-based Check Set Selection, ACL Findings, 2025.

[2] Jung et al., Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreement, ICLR, 2025.

[3] Gollapudi et al., Efficient Online Conformal Selection with Limited Feedback, 2026.

[4] Gui, Jin, and Ren, Conformal Alignment, NeurIPS, 2024.

[5] Žliobaitė et al., Active Learning with Drifting Streaming Data, 2014.

[6] Chapelle and Li, An Empirical Evaluation of Thompson Sampling, NeurIPS, 2011.

Dear Reviewer,

We would like to kindly follow up on our response and revised manuscript, where we have clarified our technical contributions and added several baseline comparisons to address your concerns. We would greatly appreciate it if you could revisit our revisions and reconsider your assessment.

Thank you very much for your time and consideration.

Comments on the paper

11 comments
AuthorsOpenReview ↗

Response to Reviewer LZt6

We thank the reviewer for their time and helpful comments. We address the concerns below.

1. Population and online error guarantees

Regarding the reviewer's question about population versus online definitions: in the population setting (Eq. 1), Type-I error is Pr⁡(A=0∣Sπ=1)\Pr(A=0\mid S_\pi=1), the confirmation rate among selected interactions. In the online setting (Eq. 4), interactions arrive in successive batches, each with a separate review budget. We therefore define Type-I error as the average of the per-round confirmation rates:

Err1^=1T∑t=1T∑iπt,i(1−At,i)1∨∑iπt,i.\widehat{\text{Err1}}=\frac1T\sum_{t=1}^T\frac{\sum_i\pi_{t,i}(1-A_{t,i})}{1\vee\sum_i\pi_{t,i}}.

Writing mt=∑iπt,im_t=\sum_i\pi_{t,i}, this averages the confirmation rate of each round equally, whereas a pooled rate weights each round by mtm_t. The quantities need not coincide. The roundwise average is a natural extension for a separately budgeted and updated online policy; analogous time-averaged error criteria appear in online conformal prediction: [1] distinguishes distributional coverage from empirical coverage over a sequence, building on [2]. In verification, [3] likewise uses population analysis to motivate online error control.

Regarding the question about selecting nothing, this gives Type-I error zero, both online (by definition) and in the population formulation under its 0/0=00/0=0 convention. However, it misses every edit. Its population Type-II error becomes Pr⁡(A=1)\Pr(A=1), while its online Type-II error is the average edit prevalence among arriving batches. Thus Type-I control alone is not useful; our objective is to minimize Type-II error subject to confirmation and budget constraints. This trade-off between the errors appears in the operating-point comparisons (Appendix D.3, Figs. 4–5) and Type-I–Type-II frontiers (Appendix D.9, Figs. 12–13).

To address the question of why Theorem 2 bounds Type-I rather than Type-II error: Theorem 2 guarantees budget feasibility and average roundwise Type-I control up to a vanishing finite-horizon term. A nontrivial Type-II bound cannot generally hold without assumptions on the score or stream. Labels of skipped examples are unobserved, so no policy can reliably distinguish edits from confirmations among them in an arbitrary stream. A similar distinction between guaranteed error control and an efficiency objective appears in [1,2], which establish online coverage properties rather than optimal prediction-set size. More closely, [3] controls online acceptance and rejection errors without an oracle-optimality guarantee on the frequency of costly strong verification. Likewise, Theorem 1 gives our population optimum given the edit posterior, while Type-II performance online is evaluated empirically, including against edit-aware oracles (Appendix D.10, Fig. 15). We would be happy to clarify this distinction further if helpful.

2. Nonrepeated streams, stronger baselines, and matched costs

We acknowledge the concern that repeated examples may make online learning easier. We added first-occurrence results from the original streams (Appendix D.11, Table 9) and streams of 2,500 distinct interactions without replacement on MATH and PopQA with Llama-3.3-70B-Instruct and Qwen3-32B (Appendix D.11, Figs. 16–17). Results average five orderings. EDIT-REP retains favorable Type-I–Type-II trade-offs even when examples are never repeated. RAGTruth also tests 450 held-out, nonrepeated responses across QA, summarization, and data-to-text generation (Appendix D.14, Figs. 21–22). These evaluations test discovery without repeated exposure to identical interactions.

We have added Top-K, Fixed Threshold, Online Conformal Selection, Conformal Alignment, stream active learning, and Thompson sampling (Appendix D.9, Figs. 12–13). All score-based methods in these frontiers share the same frozen representations, online logistic probe, initialization, and per-round budget. Their feedback histories naturally differ with their selected items. This directly tests the role of the selection rule against Top-K and fixed-threshold policies using the same scoring architecture. Because equal budgets need not mean equal expenditure, Appendix D.10 compares Type-II at matched realized reviews per round (Fig. 14) and oracle-relative missed edits (Fig. 15).

We further report stronger LLM-Select prompting (Appendix D.12, Fig. 19), response-model generalization to Qwen3-32B and Mistral Small 3.2 24B (Appendix D.5, Figs. 6–7), nonuniform costs (Appendix D.6, Figs. 8–9), distribution shifts (Appendix D.7 and D.13, Figs. 10 and 20), and representation-size ablations (Appendix D.8, Fig. 11). Together, these results provide more direct evidence on generalization to unseen interactions, the choice of selection mechanism, and performance at comparable expenditure.

We hope these clarifications and additional results address the reviewer's concerns and would appreciate reconsideration of the assessment and score.

References are provided in a follow-up response.

AuthorsOpenReview ↗

References for our response to Reviewer LZt6

[1] Ramalingam, Kiyani, and Roth, The Relationship Between No-Regret Learning and Online Conformal Prediction, ICML, 2025.

[2] Gibbs and Candès, Adaptive Conformal Inference Under Distribution Shift, NeurIPS, 2021.

[3] Kiyani et al., When to Trust the Cheap Check: Weak and Strong Verification for Reasoning, ICML, 2026.

AuthorsOpenReview ↗

Additional Clarifications, Part 3

We thank the reviewer for the concrete experimental proposal. We have conducted the requested experiment, following the protocol and evaluation criteria outlined in your comment.

1. Disjoint data, validation selection, and full-horizon evaluation

"Freeze a probe trained on disjoint data; select configurations on separate validation streams; evaluate all methods over the full task set with identical budget and accounting for feedback."

We use shared frozen scores from a probe trained on disjoint data, with calibration and policy configuration selection performed separately from evaluation. Configurations are chosen on validation streams for the prespecified target ε=0.40\varepsilon=0.40. The four QA tasks use Llama-70B responses and Qwen3-1.7B representations; RAGTruth uses Qwen3-4B representations and its held-out test split. We evaluate all records and all rounds over 30 paired stream orderings. Every method has the same ceiling of five reviews per round, and we report realized review cost and coverage, rather than treating the common ceiling as equal expenditure. Scores remain frozen throughout evaluation, eliminating policy-dependent probe learning while retaining the actual feedback-selection and review-cost differences. We also use validation-calibrated cutoffs rather than a conservative test-time threshold.

2. Review quality, discovery, and expenditure

"Report paired Type-II improvements jointly with Type-I, along with the cost, pooled precision, and coverage."

The table reports means (percent except cost, in reviews per round). Type-I is round-averaged; pooled precision is computed across reviewed items. Paired intervals and robustness checks follow in our second comment.

DatasetMethodType-I ↓Type-II ↓CostPooled precision ↑Coverage
MATH-500EDIT41.2312.783.3052.2432.95
Online Conformal Selection49.7511.654.0849.2240.77
Conformal Alignment38.4819.242.3048.7022.99
Stream Active Learning55.6412.894.5744.2045.71
Thompson Sampling56.0410.045.0043.9650.00
Top-K56.0410.045.0043.9650.00
Fixed Threshold45.5416.562.9250.6329.18
MMLU-ProEDIT32.1228.264.7466.1547.39
Online Conformal Selection36.1031.424.9363.8449.26
Conformal Alignment33.4631.494.6666.0146.59
Stream Active Learning44.9541.104.7054.7447.03
Thompson Sampling36.4131.215.0063.5950.00
Top-K36.4131.215.0063.5950.00
Fixed Threshold33.3831.884.5466.6345.45
PopQAEDIT33.0444.854.2867.1243.71
Online Conformal Selection36.4047.314.9163.1850.12
Conformal Alignment31.4448.084.1265.2642.03
Stream Active Learning44.6955.024.7555.1948.47
Thompson Sampling37.1247.324.9262.8750.20
Top-K36.8547.044.9263.1550.20
Fixed Threshold33.5247.544.0565.2241.33
GPQAEDIT27.1643.232.2052.2222.11
Online Conformal Selection50.7243.254.8848.6749.06
Conformal Alignment27.1645.632.1447.6121.52
Stream Active Learning53.5945.654.8546.2248.69
Thompson Sampling51.9043.815.0048.1050.22
Top-K51.2943.215.0048.7150.22
Fixed Threshold20.0646.170.8044.417.99
RAGTruthEDIT44.8616.794.3660.0743.59
Online Conformal Selection35.0819.913.8759.5438.74
Conformal Alignment47.0218.594.9953.0049.92
Stream Active Learning64.7836.054.3635.5143.64
Thompson Sampling47.0218.585.0052.9850.00
Top-K47.0218.585.0052.9850.00
Fixed Threshold47.0218.585.0052.9850.00

Practical trade-off. Compared with Fixed Threshold, EDIT reduces Type-II on all five datasets by 3.78, 3.62, 2.69, 2.94, and 1.79 pp, while also reducing Type-I on four. GPQA illustrates the trade-off: its conservative fixed threshold reviews fewer items and achieves lower Type-I, but misses more edits. Compared with Top-K, EDIT delivers higher pooled precision, lower Type-I, and fewer reviews on all five datasets. It also lowers Type-II on MMLU-Pro, PopQA, and RAGTruth, and nearly matches Top-K on GPQA at much lower cost. On MATH, EDIT accepts slightly higher Type-II in exchange for markedly lower Type-I and cost. Overall, these results demonstrate the benefit of adaptive αt\alpha_t in balancing edit discovery, unnecessary confirmations, and review expenditure rather than imposing a fixed cutoff or review count.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.