acceptodds
Under review as a conference paper at ICLR 2027

Counting Evidence, Not Feedback: Provenance-Aware Agent Self-Improvement

Abstract

Self-improving language-model agents can reduce manual workflow design, but deciding which updates to retain remains difficult when repeated interpretations of the same observations are counted as additional evidence. We introduce Provenance-Adjusted Evidence (PAE), a fixed-budget acceptance rule that traces feedback to its originating observations, removes replayed records, and pools judgments sharing the same sources. PAE assigns influence according to source overlap independently of feedback volume, reducing to the mean paired task gain with complete direct scores and adjusting overlapping joint assessments when per-task scores are incomplete. Across GEPA, TextGrad, and MIPROv2-No-Demos with Qwen3-8B, PAE improves five-run mean accuracy and code pass@1 by 0.72–1.33 percentage points on GSM8K, MATH-500, MBPP+, and IFBench under matched budget ceilings. PAE provides a common acceptance framework for direct and joint evaluations, allowing richer interpretations while keeping their influence tied to the underlying observations.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language models, Agent self-improvement

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_KuH2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Provenance-Adjusted Evidence (PAE) for selecting updates in self-improving LLM systems. PAE tracks the observations behind feedback, removes duplicated evidence, and adjusts aggregation. With complete per-task scores, it leads to averaging paired task gains. Experiments show that repeated interpretations of the same observations can alter update decisions, and replacing native selectors with PAE improves performance across several math, coding, and instruction-following benchmarks.

Critical strengths

Clear identification of an evaluation issue. The paper clearly distinguishes repeated feedback from independent evidence and demonstrates that repeatedly interpreting the same observations can alter update-selection decisions.

Careful experimental design. The paper evaluates multiple optimizers under controlled budgets, making the selector comparisons relatively clean.

Clear presentation and figures. The paper is well organized, and the figures and tables clearly illustrate the motivation, method, and main experimental results.

Critical weaknesses

The main experiments do not validate the most important part of PAE. With complete per-task scores, PAE reduces exactly to simple root averaging. Therefore, the reported downstream gains mainly show the benefit of replacing the native selector with a root-mean acceptance rule, rather than the benefit of provenance-aware overlap correction.

Limited validation on complex tasks. The evaluation is dominated by relatively short-form math, coding, and instruction-following benchmarks. The paper lacks validation on more complex settings, such as long-horizon agentic tasks or harder competition-level math and coding, where overlapping feedback provenance may be more realistic and consequential.

The gains may partly come from a more permissive acceptance policy. PAE accepts substantially more updates than the paired-test and PACE baselines while also showing higher false/harmful commit rates. Without comparisons at matched commit or risk levels, it is difficult to isolate the benefit of the proposed evidence aggregation rule.

Questions for authors
  1. Since PAE reduces to root averaging in the main experiments, which results specifically demonstrate the benefit of provenance-aware overlap correction beyond a simpler root-mean selector?

  2. Can the authors provide results on more complex tasks, such as long-horizon agentic benchmarks or harder competition-level math and coding, where feedback provenance and repeated evidence are likely to play a larger role?

  3. Can the authors compare PAE with the baselines at matched commit rates or matched false/harmful-commit rates to determine whether the gains come from better evidence aggregation rather than a more permissive acceptance policy?

4 comments
AuthorsOpenReview ↗

Response to Reviewer KuH2 (1/2)

Thank you for taking the time to review our work and for your constructive suggestions on task complexity and acceptance quality. Below, we respond to each point with the relevant experiments and controls.

W1. The principal experiments primarily validate root averaging.

Response.{\color{green}\mathbf{Response.}} PAE accounts for the source observations shared by multiple judgments. With complete singleton feedback, this gives exact root averaging. Incomplete feedback activates the overlap adjustment. In a controlled interface with 50% singleton coverage, fixing acceptance at 30/100 yields:

TaskAggregatorBeneficial / zero / harmfulHCR (%)Net gain per proposal (pp)
MBPPEqual-group23/3/413.330.466
MBPPPAE25/2/310.000.505
TelecomEqual-group21/3/620.000.633
TelecomPAE23/3/413.330.767

The corresponding five-seed optimization experiment gives:

TaskEqual-group final score (%)PAE final score (%)Gain (pp)
MBPP70.9071.69+0.79
Telecom28.0030.67+2.67

PAE selects more beneficial and fewer harmful updates at the same acceptance count, then achieves higher final scores in closed-loop optimization. Together, these comparisons demonstrate the contribution of cross-group overlap adjustment beyond identical-ancestry grouping.


W2. Validation on complex tasks is limited.

Response.{\color{green}\mathbf{Response.}} Your question concerns challenging math and coding problems as well as multi-turn agentic tasks. Table 2 reports results on the following harder benchmarks:

Original Table 2 benchmarkAudit tasksGEPA (%)GEPA + PAE (%)
MATH-50050070.5271.48
AIME 20253022.6724.67
LiveCodeBench17524.6925.83

AIME and LiveCodeBench are transfer evaluations. The τ²-bench [1] telecom experiment extends the evaluation to multi-turn tool use:

Joint-feedback gateAudit success (%)Median agent+user messages [Q1, Q3]Median tool executions [Q1, Q3]
Dedup26.6725 [18, 35]12 [8, 18]
Equal-group28.0025 [18, 34]12 [8, 17]
PAE30.6726 [19, 35]12 [8, 18]

This experiment uses five seeds, a 30-task held-out subset, and six update rounds. With PAE, the median episode contains 26 agent–user messages and 12 tool executions. Each complete episode forms one root. The higher success rate therefore extends the evidence to tasks requiring sustained interaction and tool use.

[1] τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment.


W3. Could the gains come from a more permissive acceptance policy?

Response.{\color{green}\mathbf{Response.}} At exactly 30 accepted candidates out of 100, PAE selects more beneficial updates and achieves a higher net gain in both direct-score banks. This controls the acceptance volume you identify:

Direct-score bankMethodBeneficial / zero / harmfulHCR (%)Net gain per proposal (pp)
GSM8KFixed-n ranking21/3/620.000.341
GSM8KPACE ranking21/3/620.000.341
GSM8KPAE = Root mean22/3/516.670.368
MBPP+Fixed-n ranking20/3/723.330.357
MBPP+PACE ranking20/4/620.000.365
MBPP+PAE = Root mean22/3/516.670.429

Fixed-n ranks by one-sided test support. PACE uses support from the shared full root sequence at λ=0.5\lambda=0.5. The tables compare ranking quality at matched acceptance counts.

The theory in Section 4.5 considers both accepting worse candidates and rejecting better ones. PACE [1] controls per-candidate false-acceptance probability under its conditional null. Here, HCR measures the harmful fraction among accepted candidates. PACE's α=0.05\alpha=0.05 guarantee applies to its original test, and the matched-count comparison evaluates a different operating point. Its zero-commit result in Table 11 has HCR = NA. The matched-count results directly support a selection-quality contribution.

[1] PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents.

AuthorsOpenReview ↗

Response to Reviewer KuH2 (2/2)

Q1. Which results demonstrate overlap benefits beyond a simple root-mean selector?

Response.{\color{green}\mathbf{Response.}} Section 4.3 proves exact equality between PAE and root mean under complete singleton feedback. The coverage experiment examines the contribution of joint judgments as singleton availability varies. On the same 100 MBPP candidate pairs:

Singleton coverageEqual-group HCR (%)PAE HCR (%)Equal-group net gain (pp)PAE net gain (pp)
0%21.0518.420.4840.542
25%20.5115.790.5110.566
50%18.4213.160.5210.582
75%15.7911.110.5500.587
100%13.5111.110.5560.587

At 100% coverage, PAE assigns zero weight to joint groups and equals root mean. Equal-group continues to weight joint groups. At 50%, using only available singletons gives U=0.429U=0.429 pp, compared with PAE's 0.5820.582 pp:

50%-coverage selectorAccepted /100Beneficial / harmful acceptsNet gain per proposal (pp)
Available-singleton mean3022 / 50.429
Equal-group3827 / 70.521
PAE3830 / 50.582

PAE uses the available joint evidence to improve recall and net gain over singleton-only averaging. At the same 38 accepted candidates, it also selects more beneficial and fewer harmful updates than Equal-group. These results identify the benefit of overlap adjustment under incomplete coverage.


Q2. Can you provide results on long-horizon agentic tasks?

Response.{\color{green}\mathbf{Response.}} The telecom experiment directly addresses your request. Qwen3-8B serves as both the task agent and user simulator, with greedy decoding shared across methods. One complete episode is one root. The 114 tasks are split into 12 proposal, 24 calibration, 48 acceptance, and 30 audit tasks. With K=8K=8, the acceptance split supports six fresh blocks.

Feedback interfaceGateSeed 0–4 successes, each /30Mean success (%)
Controlled jointDedup8, 7, 9, 8, 826.67
Controlled jointEqual-group8, 8, 9, 8, 928.00
Controlled jointPAE9, 9, 10, 9, 930.67
Official complete scoreS1 control selector9, 8, 10, 9, 930.00
Official complete scorePAE = Root mean9, 9, 10, 9, 1031.33

PAE achieves four additional successes over Equal-group across the five runs, which reuse the same 30 audit tasks. Under complete official scoring, PAE also achieves a higher mean success rate than the S1 outer-selector control. These results describe the stated 30-task audit subset and Qwen3-8B simulator configuration. They demonstrate improvements in multi-turn tool tasks under both feedback interfaces.


Q3. What happens at matched commit rates or matched false/harmful-commit rates?

Response.{\color{green}\mathbf{Response.}} Both controls are available. First, at the same 30/100 commits on identical joint-feedback candidates:

TaskEqual-group beneficial / harmfulPAE beneficial / harmfulEqual-group / PAE HCR (%)
MBPP23 / 425 / 313.33 / 10.00
Telecom21 / 623 / 420.00 / 13.33

Second, at a common 10% empirical HCR, threshold-mixture curves yield:

Joint-feedback taskGateBeneficial recall (%)FCR (%)Net gain per proposal (pp)
MBPPEqual-group37.7819.470.361
MBPPPAE57.7816.430.536
TelecomEqual-group22.5019.000.330
TelecomPAE42.5019.470.630

The 10% value is the empirical HCR operating point on these test curves. A separate calibration experiment selects and freezes thresholds before test evaluation:

TaskGateTest accepts /100Test HCR (%)Test beneficial recall (%)
MBPPEqual-group219.5237.78
MBPPPAE319.6857.78
TelecomEqual-group119.0922.50
TelecomPAE219.5242.50

Calibration yields closely matched observed HCRs, with PAE achieving 20 percentage points higher beneficial recall in both domains. The exact-count and exact-HCR comparisons reach the same conclusion: PAE improves the observed selection tradeoff at controlled operating points.

Thank you for providing these additional experiments and details. I keep my original score.

Thank you for taking the time to review our additional experiments and clarifications. We greatly appreciate your positive assessment and support for our work.

Reviewer_pdmdOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies acceptance decisions in self-improving LLM agents when multiple feedback records trace back to the same task observations. It proposes Provenance-Adjusted Evidence (PAE), which deduplicates records, aggregates identical source ancestries, and weights overlapping evidence groups through a minimum-norm source-exposure objective. The paper provides an independent-root error-exponent result and evaluates the rule in controlled feedback-reuse studies and optimizer wrappers.

Critical strengths

The paper presents clear and technically sound formalization of the distinction between new observations and repeated interpretations.

The theoretical claim is appropriately scoped and supported.

Critical weaknesses

The e2e performance evidence does not isolate the distinctive provenance-overlap mechanism, and it is mildly confounded by unequal realized evaluation costs. The main system experiments use the primary direct-score protocol, for which Section 4.2 and Appendix A define exactly one paired task-score difference per root. Under complete singleton coverage, Section 4.3 proves that PAE reduces exactly to root averaging, and Appendix B stated that PAE and root-mean acceptance share the same row in this regime. Thus, the gains in Tables 1, 2, 8 & 9 establish that replacing the native selector with fresh-block root-mean acceptance can help, but they do not establish that the overlap correction in Equations 3 to 5 is responsible for the improvement or is practically beneficial in a native joint-feedback setting.

Moreover, the primary comparison in Table 10 gives PAE 4–8 more task calls per run than the corresponding native selector, for example 700 versus 696 calls on GSM8K and 700 versus 692 on MATH-500. These are small differences under shared ceilings, but they mean the claim in Section 5.2 that performance improves by “changing the acceptance rule” is not cleanly separated from evaluating slightly more evidence.

Questions for authors

N/A

3 comments
AuthorsOpenReview ↗

Response to Reviewer pdmd (1/1)

Thank you for your careful reading and valuable feedback on the theory, mechanism, and evaluation costs. We appreciate the time you devoted to the review and address both concerns below.

W1. The main end-to-end results do not isolate provenance-overlap adjustment.

Response.{\color{green}\mathbf{Response.}} As you note, Tables 1, 2, 8, and 9 evaluate the complete-score specialization. The joint-feedback experiment directly tests overlap adjustment. PAE's central idea is to assign influence according to shared source observations. Its weights minimize source exposure independently of judgment values:

\qquad Q_{gk}=\frac{\mathbf{1}\\{k\in A_g\\}}{\sqrt{|A_g|}}.$$ Section 4.3 and Proposition 1 show that this minimizes worst-case support sensitivity to additive source perturbations. Entropy breaks ties among minimizers. The matrix $Q$ records shared ancestry directly. The empirical comparison fixes 50% singleton coverage and retains all supplied singletons. Methods share the same candidate pairs and judgment records: | Task | Aggregator | Accepted /100 | Beneficial / harmful accepts | Net gain per proposal (pp) | Separate closed-loop score (%) | | --- | --- | --- | --- | --- | --- | | MBPP | Equal-group | 38 | 27 / 7 | 0.521 | 70.90 | | MBPP | PAE | 38 | **30 / 5** | **0.582** | **71.69** | | Telecom | Equal-group | 35 | 23 / 8 | 0.633 | 28.00 | | Telecom | PAE | 34 | **25 / 6** | **0.767** | **30.67** | PAE improves selection quality and achieves five-seed closed-loop gains of **0.79/2.67 pp over Equal-group**. These results connect the source-exposure principle to better updates and higher final performance in a **controlled joint-feedback interface on real tasks**. --- > #### **W2. The extra 4–8 task calls confound attribution to the acceptance rule.** ${\color{green}\mathbf{Response.}}$ Your cost concern is addressed through two separate budget controls. First, with a common GEPA proposer, every selector uses **60 proposal-side plus 640 acceptance-side task calls, 20 proposer calls, and zero judge calls per run**. S1 labels the outer-selector control in this experiment. All calls perform useful evaluations. Five-seed final scores are: | Task | S1 control (%) | PAE (%) | Calibrated PAE (%) | PAE gain (pp) | Calibrated PAE gain (pp) | | --- | --- | --- | --- | --- | --- | | GSM8K | 89.46 | 90.14 | 89.90 | +0.68 | +0.44 | | MATH-500 | 70.68 | 71.48 | 71.20 | +0.80 | +0.52 | | MBPP+ | 71.59 | 72.43 | 72.06 | +0.85 | +0.48 | | IFBench | 38.60 | 39.73 | 39.33 | +1.13 | +0.73 | Differences use unrounded seed-level means. Calibrated PAE targets S1's acceptance rate using separate calibration data. Realized commits differ by only 1/100 proposals. Second, a common input-plus-output token ceiling controls token use. A predefined stopping rule selects the last completed gate checkpoint within that ceiling: | Task | Common token ceiling (M) | S1 control (%) | PAE (%) | Calibrated PAE (%) | | --- | --- | --- | --- | --- | | GSM8K | 2.49 | 89.46 | **90.08** | 89.90 | | MATH-500 | 3.06 | 70.68 | **71.40** | 71.20 | | MBPP+ | 3.17 | 71.59 | **72.33** | 72.06 | | IFBench | 2.12 | 38.60 | **39.60** | 39.33 | PAE completes 19–20 rounds under these ceilings. S1 and calibrated PAE complete 20. **The gains over S1 persist under both equal-call and common-token budgets**, including after calibration to a similar commit rate. Reported costs cover optimization, with shared calibration and retrospective audit accounted for separately. These comparisons support the root-mean selector's benefit relative to S1 under matched budgets.

Thx authors for their additional experiments and matched-budget controls; it did address my concerns about the contribution of overlap adjustment and the unequal realized evaluation costs. I maintain my weak-accept recommendation with Increased Confidence. I encourage the authors to incorporate these results into the paper.

Thank you for your careful assessment of the additional experiments and matched-budget controls. We greatly appreciate your continued support and increased confidence in our work.

Reviewer_TA3AOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how repeated interpretations of the same task observations affect an agent's decision to retain a proposed update. The authors introduce Provenance-Adjusted Evidence (PAE), which traces feedback to its originating observations, removes replayed records, and pools judgments sharing the same sources to adjust their evidential influence. With complete direct scores, PAE reduces exactly to averaging paired task gains. For joint assessments with incomplete per-task feedback, PAE adjusts their contributions based on source overlap. Controlled experiments show that even with candidates and task observations fixed, varying the number and allocation of interpretations can change acceptance decisions. By replacing the native selectors of several optimization methods with PAE, the paper demonstrates modest improvements in final task performance across multiple benchmarks.

Critical strengths
  1. Identifies a specific problem of evidence reuse in update selection. The controlled study in Section 3 shows that varying the number and allocation of interpretations changes acceptance decisions for 15.0% of candidate pairs, even with task observations fixed. This highlights the potential impact of evidence reuse in agent self-improvement.

  2. Extensive experimental evaluation. The paper provides a fairly comprehensive empirical evaluation across three optimizers and four benchmarks, with disjoint task splits, matched budget ceilings, and repeated runs.

Critical weaknesses
  1. Lack of end-to-end validation of the overlapping joint-feedback mechanism. The main end-to-end experiments use complete direct task scores, where PAE reduces exactly to averaging paired task gains. Therefore, the reported performance gains do not demonstrate the additional contribution of source-overlap adjustment. Although Tables 12 and 14 examine aspects of provenance adjustment in controlled settings, it remains unclear whether this mechanism improves acceptance decisions and end-to-end performance on real tasks when per-task scores are incomplete.

  2. The source of the performance gains remains unclear. Table 7 shows that PAE accepts more updates than the paired-test baselines, achieving higher beneficial-update recall but also incurring nonzero harmful-commit rates. This raises the question of how much of the performance gains in Table 1 comes from more accurately selecting beneficial updates versus simply accepting more updates. It would be helpful to report the number of accepted updates for PAE and each optimizer's native selector, and compare acceptance quality at similar acceptance rates.

Questions for authors
  1. How do these mechanisms (invocation deduplication, grouping by identical ancestry, and source-overlap adjustment) individually contribute to improving acceptance decisions?

  2. Is it because of accurately identifying beneficial updates, or is it just due to a higher update acceptance rate that improves the performance of PAE?

2 comments
AuthorsOpenReview ↗

Response to Reviewer TA3A (1/2)

We appreciate the time you spent examining our method and the detailed suggestions you provided. Below, we address your questions about end-to-end validation and the source of the performance gains.

W1. The overlapping joint-feedback mechanism needs end-to-end validation.

Response.{\color{green}\mathbf{Response.}} The joint-feedback experiment directly tests source-overlap adjustment on real tasks. In this controlled interface, 50% of roots have singleton comparisons and overlapping packets contain 2–4 roots. All supplied singletons remain available. Five-seed closed-loop results are:

TaskGateCommits / proposalsHarmful / acceptedFinal audit score (%)
MBPPDedup40/1009/4070.37
MBPPEqual-group37/1007/3770.90
MBPPPAE36/1005/3671.69
τ² telecomDedup12/304/1226.67
τ² telecomEqual-group11/303/1128.00
τ² telecomPAE12/302/1230.67

Each MBPP run uses 700 task, 20 proposer, and 640 judge calls. Each telecom run uses 114 episodes, six proposer calls, and 96 judge calls. An episode includes multiple model calls. PAE improves over Equal-group by 0.79 pp on MBPP and 2.67 pp on telecom, with the MBPP gain appearing in all five seeds. The final scores and harmful-commit counts support the practical value of overlap adjustment in this controlled joint interface.


W2. How many updates do the native selectors accept, and does the gain survive similar acceptance rates?

Response.{\color{green}\mathbf{Response.}} For the optimizer selectors in Table 1, the outer-commit counts below sum five runs of 20 proposals each. Table 7 reports the separate paired-test gate experiment.

OptimizerTaskNative commits /100PAE commits /100Native / PAE HCR (%)
GEPAGSM8K243220.83 / 12.50
GEPAMATH-500283621.43 / 16.67
GEPAMBPP+253420.00 / 14.71
GEPAIFBench273522.22 / 17.14
TextGradGSM8K223022.73 / 16.67
TextGradMATH-500253324.00 / 18.18
TextGradMBPP+233121.74 / 16.13
TextGradIFBench253324.00 / 18.18
MIPROv2-No-DemosGSM8K233121.74 / 16.13
MIPROv2-No-DemosMATH-500253324.00 / 18.18
MIPROv2-No-DemosMBPP+253220.00 / 15.62
MIPROv2-No-DemosIFBench263323.08 / 18.18

Each commit is a deployed replacement of the incumbent. PAE has a lower observed HCR in all twelve optimizer–task comparisons. Each run follows its own update trajectory, so the matched-rate experiment below further examines the contribution of acceptance volume.

This experiment uses a common GEPA proposer and equal evaluation quotas. S1 labels its outer-selector control. A PAE threshold fitted on 256 held-out calibration pairs targets S1's commit rate. Each run uses 700 task calls and 20 proposer calls:

TaskS1 / calibrated-PAE commits per100S1 / calibrated-PAE final score (%)Paired mean gain (pp)
GSM8K24 / 2589.46 / 89.90+0.44
MATH-50028 / 2970.68 / 71.20+0.52
MBPP+25 / 2671.59 / 72.06+0.48
IFBench27 / 2838.60 / 39.33+0.73

With a commit-count difference of just one per 100 proposals, calibrated PAE retains a 0.44–0.73 pp gain over S1. This supports a selection-quality contribution at closely matched acceptance rates. Under complete feedback, a selector using the identical zero-threshold root mean remains exactly equivalent to PAE.


Q1. What do deduplication, identical-ancestry grouping, and overlap adjustment each contribute?

Response.{\color{green}\mathbf{Response.}} PAE ties evidential influence to source observations. The ablation below measures how this principle affects selection as replayed records, repeated interpretations, and cross-group overlap are accounted for. All rows share the same 100 MBPP candidate pairs and joint-feedback records:

AggregationAcceptedBeneficial / zero / harmfulHCR (%)Net gain per proposal, UU (pp)
Raw artifact mean4527/6/1226.670.468
Invocation deduplication4227/5/1023.810.481
Equal ancestry-group weights3827/4/718.420.521
PAE3830/3/513.160.582

The final comparison isolates overlap adjustment at the same acceptance count: PAE selects three more beneficial and two fewer harmful updates. Table 12 provides a complementary intervention. Concentrated cached copies cause 11% raw flips on MBPP, reduced to zero by deduplication. Concentrated new interpretations produce 20% flips after deduplication and 10% after grouping. New interpretations can refine a group's mean. Cached replay leaves PAE unchanged.

AuthorsOpenReview ↗

Response to Reviewer TA3A (2/2)

Q2. Is PAE selecting better updates or merely accepting more?

Response.{\color{green}\mathbf{Response.}} At exactly 30/100 accepted candidates from the same bank, PAE selects higher-quality updates in both domains:

Joint-feedback taskGateBeneficial / zero / harmfulHCR (%)Mean accepted gain (pp)
MBPPDedup21/3/620.001.323
MBPPEqual-group23/3/413.331.552
MBPPPAE25/2/310.001.684
TelecomDedup19/4/723.331.667
TelecomEqual-group21/3/620.002.111
TelecomPAE23/3/413.332.556

PAE selects more beneficial and fewer harmful updates without increasing their number. The same direction holds at 20% and 40% acceptance:

TaskAccepted /100Equal-group / PAE HCR (%)Equal-group / PAE UU (pp)
MBPP2010.00 / 5.000.336 / 0.370
MBPP4020.00 / 15.000.532 / 0.608
Telecom2020.00 / 10.000.400 / 0.600
Telecom4025.00 / 20.000.667 / 0.833

The observed direction is consistent across all three acceptance rates: PAE selects fewer harmful updates and achieves a higher net gain per proposal than Equal-group.

Reviewer_XsPSOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper looks at how agents decide whether to keep a proposed update when several feedback reports reuse the same observations. PAE tracks where each judgment comes from, removes replayed records, and adjusts the weights of judgments with overlapping sources. With a direct score for every task, the rule becomes ordinary averaging of the paired task gains. The more general weighting is used when only some direct scores are available. The authors show that changing the feedback layout changes acceptance decisions for 36 of 240 candidate pairs without adding new task observations. They also prove an optimal worst-case error exponent for the complete-score case under independent tasks. In the main experiments, replacing the selectors in three optimizers improves scores by 0.72–1.33 percentage points across four benchmarks. These experiments use complete direct scores, so the gains are from root averaging.

Critical strengths
  1. The experiment with fixed candidates and observations is a good motivation for the paper. Changing only the feedback layout is enough to change some acceptance decisions. The follow-up analysis helps explain how much of this comes from reweighting the same tasks. I found this more informative than a comparison of final benchmark scores alone.
  2. Replacing the selector gives small but consistent gains across the tested optimizers and benchmarks. Similar gains on the additional models make the result more convincing. Although this mainly supports task averaging, it is still a useful result: the acceptance step can affect performance even when the proposal method stays the same.
Critical weaknesses
  1. I am not yet convinced that the general overlap-weighting rule improves update selection. The main performance results use complete task scores, where PAE is just root averaging. The other experiments show how the weights change, but do not show that these changes produce better updates. This leaves the main new component less well tested than the simpler rule.
  2. Errors in the provenance records change a noticeable number of decisions: 11 of 128 with shuffled connections and 16 of 128 with omitted connections. The paper does not establish what this would mean for final performance in a longer run. The judge comparisons also show that PAE accepts more beneficial updates at the cost of more harmful ones than some alternatives. I would want to understand that tradeoff better before relying on it as an acceptance rule.
Questions for authors
  1. Do you have results where direct task scores are incomplete and the overlap weighting leads to better updates than deduplicated averaging or equal weighting of source groups?
  2. Which errors in the provenance records can be caught by the validity checks? When an error passes those checks, do you know whether the resulting decision changes are harmful?
  3. How does PAE compare with the other aggregation methods when they accept the same proportion of beneficial updates? The current comparison makes it hard to separate better selection from a greater willingness to accept updates.
3 comments
AuthorsOpenReview ↗

Response to Reviewer XsPS (1/2)

Thank you for the time and care you put into reviewing our paper and for your thoughtful comments. Below, we address your questions about overlap weighting, provenance reliability, and update quality.

W1. Does overlap weighting improve actual updates beyond root averaging?

Response.{\color{green}\mathbf{Response.}} PAE assigns influence according to the observations behind each judgment. This addresses a specific problem: multiple reports can give the same source disproportionate influence. Section 4.3 recovers root averaging when all singleton scores are available. The incomplete-feedback experiment directly tests the benefit of accounting for shared sources.

The experiment uses a predeclared, controlled joint-feedback interface on MBPP and τ² telecom. Singleton coverage is 50%, and all available singletons are retained. With the same GEPA proposer and per-round evaluation quotas, the five-seed closed-loop results are:

TaskGateCommits / proposalsHarmful commitsFinal audit score (%)
MBPPDedup40/1009/4070.37
MBPPEqual-group37/1007/3770.90
MBPPPAE36/1005/3671.69
τ² telecomDedup12/304/1226.67
τ² telecomEqual-group11/303/1128.00
τ² telecomPAE12/302/1230.67

PAE improves over Equal-group by 0.79 pp on MBPP and 2.67 pp on telecom. These gains demonstrate the downstream benefit of overlap adjustment in the controlled incomplete-feedback setting. Telecom uses a 30-task audit subset and six optimization rounds. MBPP uses 378 audit tasks and 20 rounds.


W2. What are the downstream consequences of provenance errors and the beneficial–harmful update tradeoff?

Response.{\color{green}\mathbf{Response.}} Table 14 reports 11/128 and 16/128 decision flips under provenance corruption. To measure the downstream consequences, the five-seed MBPP joint-feedback experiment corrupts 10% of recorded parent edges and evaluates both final performance and harmful commits:

Parent recordsEqual-group final score (%)PAE final score (%)PAE harmful / accepted
Clean70.9071.695/36
Reconnected70.5871.227/36
Omitted70.3270.958/36

PAE retains a higher final score than Equal-group under both corruptions. Its losses of 0.48/0.74 pp relative to clean input quantify the sensitivity to parent-record accuracy. Faithful parent logging and graph validation serve complementary roles.

For the update tradeoff, fixing acceptance at 30 of the same 100 candidates separates selection quality from willingness to commit:

Joint-feedback taskGateBeneficial / zero / harmfulHCR (%)Net gain per proposal, UU (pp)Negative loss per proposal (pp)
MBPPEqual-group23/3/413.330.4660.037
MBPPPAE25/2/310.000.5050.024
TelecomEqual-group21/3/620.000.6330.233
TelecomPAE23/3/413.330.7670.167

Here U=N−1∑jajΔ^jU=N^{-1}\sum_j a_j\widehat\Delta_j measures independent audit gain per proposal on the shared candidate bank. At the same acceptance count, PAE selects more beneficial updates, incurs a smaller loss from harmful updates, and achieves a higher net gain in both domains.


Q1. Does PAE outperform deduplication or equal source-group weighting with incomplete task scores?

Response.{\color{green}\mathbf{Response.}} Yes. In the controlled joint-feedback experiment, each method receives the same candidates, outputs, and judge responses. Each domain has 100 candidate pairs. Singleton coverage is 8/16 roots on MBPP and 4/8 on telecom. All 100 decisions in each domain assign nonzero weight to joint groups.

TaskAggregatorAcceptedBeneficial / zero / harmfulBeneficial recall (%)UU (pp)
MBPPDedup4227/5/1060.000.481
MBPPEqual-group3827/4/760.000.521
MBPPAvailable-singleton mean3022/3/548.890.429
MBPPPAE3830/3/566.670.582
TelecomDedup3923/5/1157.500.500
TelecomEqual-group3523/4/857.500.633
TelecomAvailable-singleton mean2718/3/645.000.500
TelecomPAE3425/3/662.500.767

On MBPP, PAE and Equal-group each accept 38 candidates. PAE selects three more beneficial and two fewer harmful updates. The available-singleton baseline averages the observed singleton scores, excluding unavailable scores. PAE's higher recall and net gain show the value of also using the overlapping joint judgments.

AuthorsOpenReview ↗

Response to Reviewer XsPS (2/2)

Q2. Which provenance errors are detectable, and are undetected changes harmful?

Response.{\color{green}\mathbf{Response.}} The external logger records actual evidence parents outside the candidate's control. The checks in Appendix A.1 validate computation identity and graph consistency. Each structural fault below was detected in all 20 injected cases and caused Defer:

FaultDetected / injected decisions
Same invocation ID, conflicting payload20/20
Cyclic DAG20/20
Unresolved parent20/20
Empty terminal ancestry20/20
Root outside the current block20/20

The second test measures incorrect parent records that preserve graph validity. Every altered edge remains resolvable, acyclic, and within the current block. Changing only these records on the same 100 MBPP candidate pairs produces:

PAE inputDetected / injected decisionsCorrect→wrong / wrong→correctNet additional harmful acceptsΔU\Delta U (pp)
10% reconnected edges0/1004/1+2−0.024
10% omitted edges0/1006/1+3−0.050
20% omitted edges0/10012/2+5−0.159

These results quantify both the direction and cost of the injected errors. Correct parent recording is essential even when graph-validity checks pass. For structural faults, Defer consistently prevents acceptance from invalid records. Across each 20-case set, it prevents one harmful acceptance and also retains the incumbent in six cases with beneficial candidates.


Q3. How do methods compare at the same beneficial-update recall?

Response.{\color{green}\mathbf{Response.}} At exactly 50% beneficial recall, mixtures of adjacent thresholds give the following harmful-commit rates. Each comparison uses the same candidate bank and input representation. Rates are computed from the mixtures' expected counts.

Joint-feedback aggregatorMBPP HCR (%)Telecom HCR (%)
Dedup20.9022.58
Equal-group13.5620.69
PAE7.5512.00

The corresponding comparison on the balanced singleton judge data from Table 3 is:

Judge inputAggregatorGSM8K HCR (%)MBPP+ HCR (%)
ContinuousPAE = Mean10.5314.55
ContinuousCARE-SVD3.7711.32
BinaryPAE = Majority13.5619.67
BinaryDawid–Skene13.5619.67
BinaryIsing, class-independent10.5316.95
BinaryIsing, class-dependent10.5316.95
BinaryCARE-SVD10.5316.95
BinaryCARE-Tensor3.6414.04

At the same beneficial recall, PAE achieves lower HCR than Dedup and Equal-group in both joint-feedback domains. CARE achieves lower HCR in the balanced singleton setting. CARE [1] models latent confounders, and Ising aggregation [2] models statistical dependencies among judgments. PAE uses recorded ancestry to control the influence of shared observations. The joint-feedback results demonstrate the value of this source accounting where overlap adjustment is active. Both tables report empirical threshold-curve comparisons.

[1] CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation.

[2] Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models.

AuthorsOpenReview ↗

Follow-up on your questions

Thank you again for your helpful questions, which motivated our controlled incomplete-feedback experiments showing closed-loop gains of 0.79 and 2.67 percentage points over Equal-group on MBPP and τ² telecom, respectively. We also quantified the impact of provenance errors and found lower harmful-commit rates than Dedup and Equal-group at matched beneficial recall in both joint-feedback settings.

We would be happy to discuss any remaining questions and provide further clarification or evidence while the discussion is still open. If these results address your concerns, we would be grateful if you would consider updating your rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.