acceptodds
Under review as a conference paper at ICLR 2027

CATO: Contrastive Attribution for Targeted Optimization of Multi-Component LLM Agents

Abstract

Multi-component LLM agents expose several editable components, such as system prompts, tool descriptions, and experience banks. Yet when an agent fails, its binary task outcome does not indicate which component to edit. Single-component optimizers cannot reach defects elsewhere, and unguided joint search can spend its budget on unrelated components. We formulate this component-selection problem as structural attribution, which assigns a failure to the agent's editable components rather than to trajectory steps. A component is an update target of a failure when a repair of the failure needs to edit it. We propose CATO, which contrasts failures with successful runs of the same tasks under a moderately different configuration, scores how strongly this evidence implicates each component, and writes a textual gradient describing the fix. CATO then proposes more single-component edits for higher-scoring components and keeps the best configurations on a held-out validation set, so multi-component repairs build up one edit at a time. In controlled fault injection, CATO identifies the faulty component in of cases, points more often than trajectory-only diagnosis, and its edits repair of these failures while preserving of solved tasks. Across AppWorld, WebArena, and BBH with three backbones, CATO outperforms all six baselines in every setting, leads the strongest by up to points, and uses fewer training tokens than editing every component on every failure.

Primary area:
optimization
Keywords:
LLM Agents, Multi-Component Optimization, Structural Attribution, Automated Agent Optimization, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_BUUoOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses component selection when optimizing LLM agents with editable system prompts, tool descriptions, and experience banks. It formulates this problem as multi-label structural attribution: identifying components that offer feasible intervention points for repairing a failure. CATO contrasts failed and successful executions of the same task under different configurations, produces component scores and textual gradients, and allocates candidate edits according to the accumulated evidence. Each candidate changes one component and is selected using a separate validation set. Experiments cover AppWorld, WebArena, and ten BBH subtasks with three backbones. The paper reports consistent improvements over six optimization baselines, alongside controlled fault-injection experiments and ablations examining the attribution and editing pipeline.

Critical strengths
  1. The controlled experiments connect attribution quality to actual repair utility. The fault-injection protocol changes all three components and verifies that the two neutral edits preserve success both individually and jointly. This makes the evaluation more informative than simply identifying the only changed component. CATO achieves 95.7% injected-fault identification accuracy versus 52.8% for trajectory-only diagnosis. Under shared candidate-generation and validation procedures, its repairs also achieve higher failure repair and success retention. This provides substantive evidence that better component selection improves downstream optimization.

  2. The end-to-end evaluation supports the practical value of targeted editing. CATO obtains the highest mean score across the reported backbone–metric combinations, including AppWorld applications unseen during optimization. On GLM-5.2, Test-Normal scenario goal completion reaches 89.29%, compared with 83.33% for the strongest competing optimizer. The ablations also show substantial degradation when contrastive attribution, gradient consolidation, or single-component editing is removed. Resource accounting includes rollout and validation costs, demonstrating that the gains are compatible with lower token consumption than several strong baselines.

Critical weaknesses
  1. The attribution evaluation covers a narrower setting than the general multi-label formulation. The 108 injected-fault cases come from 12 faulty configurations, with one designated functional fault per configuration and a successful reference execution available. This establishes strong performance at ranking a known injected repair target. It provides less direct evidence about naturally occurring failures with several plausible repair locations or interacting defects. Moreover, the reported accuracy uses the unique highest positive score, whereas optimization routes every component whose score exceeds η = 0.6. Consequently, the headline accuracy does not directly establish the precision and recall of the assignments actually used during optimization.

  2. The reliability of the smaller comparative gains is insufficiently quantified. Table 1 reports means over three independent runs without corresponding dispersion or confidence intervals. This matters particularly for the BBH improvements of approximately 0.76–1.17 percentage points and the smaller WebArena margins. The larger AppWorld improvements are more compelling, but the claim of consistent superiority would be stronger with uncertainty estimates for paired differences. Figure 6’s variation across population members does not substitute for variability across independent optimization runs.

Questions for authors
  1. Under the actual assignment rule of scores greater than 0.6, what are the precision and recall for identifying the injected component, and how often are additional components assigned? Can you provide uncertainty estimates that account for cases sharing the same faulty configuration?

  2. During ordinary optimization, what fraction of attributed failures have a successful same-task reference, and how does that fraction change across rounds and benchmarks? How much of the improvement comes from contrastive attribution versus the failure-only fallback?

  3. Please provide per-run results or confidence intervals for the differences against the strongest baselines, especially on WebArena and BBH. Are the smaller gains consistent across all three independent runs?

2 comments
AuthorsOpenReview ↗

Response to Reviewer BUUo (Part 1/2)

We thank the reviewer for the precise and constructive review, and for noting that the fault-injection protocol connects attribution quality to downstream repair utility and that the end-to-end results and resource accounting support targeted editing. We answer each question below with new evaluations and run-level statistics.

W1 / Q1: Under the actual assignment rule of scores greater than 0.6, what are the precision and recall for identifying the injected component, how often are additional components assigned, and what are the uncertainty estimates accounting for shared faulty configurations?

A1: Under the optimization assignment rule A(r)={d:sd(r)>0.6}A^{(r)} = \{d : s_d^{(r)} > 0.6\} (Eq. 3; whereas Table 8 reports strict top-1 accuracy), CATO reaches 93.5% recall and 91.8% precision across all 108 cases (three repetitions) and assigns an additional non-faulty component in only 8.3% of cases. The 95% CIs come from a cluster bootstrap that resamples the 12 faulty configurations (3 Prompt, 5 Tools, 4 Experience):

MethodRecall (%)Precision (%)Additional component assigned (%)Top-1 accuracy (%)
Trajectory-only50.9 [37.0, 61.6]63.0 [53.2, 68.7]26.9 [20.8, 34.2]52.8 [38.3, 63.9]
CATO93.5 [91.7, 95.0]91.8 [90.1, 93.3]8.3 [6.8, 10.0]95.7 [92.8, 98.0]

Recall is the fraction of cases in which the faulty component is assigned, and precision is the fraction of assigned components that are faulty.

On your W1 point about naturally occurring failures with several plausible repair locations and interacting defects, we ran two further studies:

  • Natural failures under equal budgets. On 30 natural AppWorld failures with no injected fault (3 single-component candidates per component; 90 per rank), editing CATO's top-ranked component repairs 91.11% (82/90) of candidates and retains 96.30% of solved tasks, vs. 78.89% and 94.33% for Rank 2 and 62.22% and 89.27% for Rank 3. Secondary components can still repair many cases, because natural failures stem from missing constraints rather than planted contradictions.
  • Two-component interacting defects. On 30 compound cases where reverting either fault alone leaves the task failing, CATO reaches 93.3% 2-round fault recall and 94.9% precision and repairs 84.4% of cases within two rounds of sequential single-component edits, vs. 70.6%, 83.0%, and 62.2% for Trajectory-only.

We will summarize these evaluations in Sec. 4.3 and add the sd>0.6s_d > 0.6 metrics, configuration-clustered CIs, and both studies to App. C.1.

AuthorsOpenReview ↗

Response to Reviewer BUUo (Part 2/2)

Q2: During ordinary optimization, what fraction of attributed failures have a successful same-task reference across rounds and benchmarks, and how much of the improvement comes from contrastive attribution versus the failure-only fallback?

A2: On GLM-5.2, 40.9–52.3% of analyzed failures have a successful same-task reference (contrast pairs) across all 6 rounds, and 60.0–68.2% in Rounds 2–6. Round 1 starts from a single seed (∣P1∣=1|\mathcal{P}_1|=1), so no reference exists and its share is 0%. Entries are contrast pairs / analyzed failures (Table 10):

Benchmark (GLM-5.2)Round 1Rounds 2–3Rounds 4–6Rounds 2–6All rounds
AppWorld0/77/9 (77.8%)2/6 (33.3%)9/15 (60.0%)9/22 (40.9%)
WebArena0/1015/19 (78.9%)7/14 (50.0%)22/33 (66.7%)22/43 (51.2%)
BBH (10 subtasks)0/5276/97 (78.4%)40/73 (54.8%)116/170 (68.2%)116/222 (52.3%)

The share peaks in Rounds 2–3, when most population updates occur, and tapers in Rounds 4–6 once configurations have largely converged and the remaining failures are mostly shared hard tasks. From Round 2 onward, tasks where both the center configuration and its neighbor succeed are skipped, and failure-only fill-ins are added only when a round has fewer than Nmin⁡=8N_{\min} = 8 contrast pairs (App. A.2).

Contribution of contrastive attribution. Because in-run fill-ins are harder tasks where both configurations failed, we compare the two diagnostic modes under matched conditions:

  • Same 108 failures, matched candidate generation and validation (Table 9): contrastive diagnosis achieves 89.8% repair and 85.3% retention, vs. 71.2% and 70.9% for the failure-only fallback.
  • End-to-end optimization (Table 2): replacing contrastive attribution in Rounds 2–6 with the failure-only fallback (Trajectory-only) lowers Test-Normal TGC by 7.14 points (93.45% to 86.31%) and SGC by 12.50 points (89.29% to 76.79%); disabling contrast-pair priority lowers them by 4.76 and 8.93 points.

We will add this breakdown to Table 10 (App. C.2) and clarify the batch filtering in App. A.2.

W2 / Q3: What are the confidence intervals for the differences against the strongest baselines, especially on WebArena and BBH, are the smaller gains consistent across runs, and what does the shaded band in Figure 6 show?

A3: All six paired 95% CIs on WebArena and BBH exclude zero, and CATO achieves the highest 3-run mean in all 18 settings in Table 1 (with all 18 paired 95% CIs over the strongest competing optimizer excluding zero). Because all methods are evaluated on the same held-out test tasks, we compute CIs via a task-level paired bootstrap pooling the three runs (scenario-level for SGC). 3-run mean ± SD:

SettingStrongest baselineCATOΔ [95% CI]
WebArena SR, GLM-5.252.12 ± 1.21 (ACE)55.76 ± 1.60+3.64 [+1.62, +5.86]
WebArena SR, Qwen3.7-Plus52.73 ± 1.60 (ACE)55.15 ± 1.05+2.42 [+0.81, +4.24]
WebArena SR, DeepSeek-V4-Flash50.91 ± 1.82 (ACE)52.73 ± 1.21+1.82 [+0.61, +3.23]
BBH Macro SR, GLM-5.292.76 ± 0.61 (GEPA)93.93 ± 0.38+1.17 [+0.60, +1.87]
BBH Macro SR, Qwen3.7-Plus93.03 ± 0.34 (GEPA)94.11 ± 0.26+1.08 [+0.53, +1.68]
BBH Macro SR, DeepSeek-V4-Flash91.50 ± 0.35 (GEPA)92.26 ± 0.29+0.76 [+0.33, +1.25]

On AppWorld with GLM-5.2, the margins over GEPA are +2.97 TGC [+1.19, +4.96] and +5.96 SGC [+1.79, +10.12] on Test-Normal and +4.08 SGC [+1.68, +6.71] on Test-Challenge.

BBH. Each run is the macro-average across 10 independently optimized subtasks, which keeps CATO's 3-run SD between ±0.26% and ±0.38%. The BBH margins are smaller because 46.6–63.3% of the attribution score mass falls on the system prompt (Table 10), which GEPA already optimizes directly above 91%, whereas AppWorld failures span all three components.

Shaded band in Figure 6. We agree that the shaded band in Figure 6a does not measure variability across independent runs: it shows the standard deviation across the Npop=5N_{\mathrm{pop}}=5 population members within a single 10-round run. We will state this in the Figure 6 caption (App. D.6), summarize the run-level consistency in Sec. 4.2 and 4.4, and add the full 3-run SD and paired 95% CI tables to App. D.

We thank you again for your thorough reading and constructive suggestions.

Reviewer_9s8qOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper defines the problem of "structural attribution" and proposes CATO, which compares successful and failed trajectories generated by different agent configurations for the same task; it then leverages the differences between these configurations, along with their execution outcomes, to evaluate which components (such as system prompts, tool descriptions, or experience banks) require modification. It integrates component-specific textual gradients and allocates a budget for generating modification candidates based on calculated attribution scores. Each candidate modifies only one component, and the system retains configurations that show superior validation performance.

Critical strengths

1. Optimization problem is clearly defined. The primary strength of this paper lies in its explicit formulation of the "component selection" problem, an aspect of optimizing multi-component LLM agents that has been relatively under-explored in existing literature. Identifying which step in a failed trajectory went wrong is a distinct challenge from determining which persistent component of an agent should be modified to rectify that failure. The paper formalizes this distinction through the concept of "structural attribution" and integrates it into the actual optimization process.

2. Extensive and Detailed Ablation Studies The paper reports consistent performance improvements across three benchmarks and three different backbone models, benchmarking their approach against various existing optimisation methods, including GEPA and OPTIMAS. Notably, the finding that "contrastive attribution" achieves significantly higher accuracy than "trajectory-only diagnosis" in controlled fault-injection scenarios underscores the efficacy of an approach that compares successful and failed execution trajectories.

Critical weaknesses

1. Limited validation of the validity of structural attribution

Although the core contribution of this paper is structural attribution, the experiments used to directly verify attribution accuracy focus exclusively on single-component fault injection. The study does not sufficiently demonstrate the performance of the attribution method in scenarios where faults in multiple components interact or where various remediation paths are possible. While the paper defines structural attribution as a multi-label problem, the primary accuracy evaluation is based on whether the injected (faulty) component receives a higher score than other components. So, it remains unclear how much the reported 95.7% accuracy reflects the method's reliability in a general multi-label structural attribution context. Additional validation is required for environments involving multi-component faults and complex component interactions.


2. Lack of clear separation between the unique contributions of contrastive attribution and performance gains relative to computational cost

The proposed method integrates several components: successful reference trajectories, configuration differences, LLM-based diagnosis, attribution scores, textual gradient consolidation, and validation-based candidate selection. While the provided ablation studies demonstrate the utility of these individual elements, they do not fully disentangle the specific contribution of using contrastive references from the effects of component-specific attribution.

For instance, a robust comparative baseline, one that utilizes both successful and failed trajectories but omits the targeted selection process based on component scores, would help to more clearly isolate the contribution of contrastive reasoning.

Additionally, although the candidate budget is aligned across the baselines, the actual number of training tokens and API calls varies significantly between the different methods. Providing a performance comparison based on an equivalent total computational cost would allow for a more compelling assessment of the claims regarding the efficiency of the proposed method.

Questions for authors

Q1. Multi-component failures: The paper defines structural attribution as a multi-label problem, but the attribution experiments mainly consider single-component faults. I'm curious how CATO would perform when multiple components contribute to the same failure, particularly when fixing one component alone is not enough. Have the authors tested such cases? It would be useful to see whether CATO can identify the relevant components and whether its sequential editing strategy can successfully resolve these failures.

Q2. Contribution of contrastive attribution: The ablation results suggest that contrastive attribution is important, but I'm not entirely convinced that the benefit of component-level attribution has been separated from simply having access to a successful reference trajectory. For example, what happens if the optimizer is given the same successful and failed trajectories and configuration differences, but does not use component-level attribution scores to guide editing?

Q3. Fairness of the computational comparison The authors align the candidate budgets across methods, but the actual token usage and number of API calls differ quite a bit. Would CATO still outperform GEPA and OPTIMAS under the same total training-token budget? A performance-versus-cost comparison would make the efficiency claims more convincing.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 9s8q (Part 1/2)

We thank the reviewer for the insightful and constructive review, and for finding the component-selection problem clearly formalized and the multi-benchmark evaluation and controlled fault-injection results convincing. We answer each question below with new controlled experiments and cost-checkpoint analyses.

W1 / Q1: How does CATO perform when multiple components contribute to the same failure and fixing one component alone is not enough?

A1: We tested this case directly. On two-component interacting faults, CATO assigns 93.3% of the injected faults and repairs 84.4% of cases within two rounds of sequential single-component edits, vs. 70.6% and 62.2% for Trajectory-only.

Setup. We combined faults from two different components into 30 compound cases on AppWorld (GLM-5.2) across (θp,θt)(\theta_p,\theta_t), (θt,θe)(\theta_t,\theta_e), and (θp,θe)(\theta_p,\theta_e), with a verified neutral edit on the third component, keeping only cases where reverting either fault alone leaves the task failing. Starting from each faulty configuration, we ran two rounds of each method across three independent runs; CATO uses the pre-injection configuration as the same-task reference, while Trajectory-only uses failed executions alone. Recall is the fraction of the 60 injected faults assigned across the two rounds, and precision is the fraction of assigned components that are faulty.

Method2-round recall (%)2-round precision (%)Repaired within 2 rounds (%)
Trajectory-only70.683.062.2
CATO93.394.984.4

Why sequential edits resolve these failures. In Round 1, a single-component candidate can fix at most one fault, so the compound task still fails. Because Round 1 starts from a single seed (∣P1∣=1|P_1|=1, below the population cap N=5N=5, Eq. 6) and Top-NN selection retains non-regressing candidates that tie the parent's validation accuracy, breaking ties in favor of newly created candidates (App. A.5), the Round-1 partial fix enters P2P_2 and is selected in Round 2 to repair the remaining fault.

Why not edit several components at once? In Table 2, allowing joint updates (w/o Single-component edits, App. A.5) lowers Test-Challenge SGC by 11.75 points (77.94% to 66.19%). Because Test-Challenge uses unseen applications, this drop suggests that simultaneous multi-component edits overfit more easily to the applications seen during validation.

Multi-label rule on the single-fault study. CATO scores each component independently in [0,1][0,1] and assigns every component with sd>0.6s_d > 0.6 (Eq. 3), which routes 11.3–36.0% of real failures to two or more components (Table 10). Re-evaluating the 108 controlled cases (three repetitions) under this rule rather than top-1 ranking, CATO reaches 93.5% recall and 91.8% precision and assigns an extra non-faulty component in 8.3% of cases, vs. 50.9%, 63.0%, and 26.9% for Trajectory-only.

We will summarize both evaluations in Sec. 4.3 and add the protocols, tables, and tie-breaking selection details to App. A.1 and App. C.1.

AuthorsOpenReview ↗

Response to Reviewer 9s8q (Part 2/2)

W2 / Q2: How is the benefit of component-level attribution scores separated from simply having access to a successful reference trajectory?

A2: Holding the successful and failed trajectories (τ+,τ−)(\tau^+, \tau^-), configuration differences (Δθ)(\Delta\theta), and case-level textual gradients fixed, using component-level attribution scores for routing and budget allocation raises Test-Normal TGC / SGC by 4.56 / 7.74 points over All-component assignment and 8.53 / 17.86 points over Random attribution, while cutting training tokens by 49% relative to All-component assignment.

The first three ablations below receive the same successful and failed trajectories and configuration diff (τ+,τ−,Δθ)(\tau^+, \tau^-, \Delta\theta) and produce ∇case(r)\nabla_{\mathrm{case}}^{(r)} under the same prompt as CATO; they differ only in whether and how component-level attribution scores guide routing and the candidate budget Bcand=16B_{\mathrm{cand}}=16 (App. A.4). Trajectory-only keeps CATO's score-guided routing and budget rule but removes the successful trajectory and diff (τ+,Δθ)(\tau^+, \Delta\theta). AppWorld, GLM-5.2, 3-run mean ± SD:

Variant (Table 2)Trajectories & diffComponent-score guidanceTest-N TGC (%)Test-N SGC (%)Test-C SGC (%)Tokens (M)
Random attributionτ+,τ−,Δθ\tau^+, \tau^-, \Delta\thetanone (one random component)84.92 ± 2.4171.43 ± 4.7267.63 ± 2.59127.36
All-component assignmentτ+,τ−,Δθ\tau^+, \tau^-, \Delta\thetanone (all three, equal budget)88.89 ± 1.5081.55 ± 2.7373.38 ± 1.44222.17
w/o Score-based budgetτ+,τ−,Δθ\tau^+, \tau^-, \Delta\thetasd>0.6s_d > 0.6 routing, equal budget91.67 ± 1.0385.12 ± 2.7373.62 ± 1.66224.33
Trajectory-onlyτ−\tau^- onlysd>0.6s_d > 0.6 routing, score-based budget (bg=⌈ag⌉b_g = \lceil a_g \rceil)86.31 ± 2.1576.79 ± 3.5769.78 ± 2.16132.94
Full CATOτ+,τ−,Δθ\tau^+, \tau^-, \Delta\thetasd>0.6s_d > 0.6 routing, score-based budget (bg=⌈ag⌉b_g = \lceil a_g \rceil)93.45 ± 1.1989.29 ± 3.0977.94 ± 1.81113.41

Trajectories and diff fixed, component-score guidance varied. Sending every failure to all three components spawns edits on uninvolved components and nearly doubles token cost. Filtering out low-score components via sd>0.6s_d > 0.6 gives 85.12% Test-Normal SGC, and sizing candidate demand by bg=⌈ag⌉b_g = \lceil a_g \rceil (ag=∑sd(r)a_g = \sum s_d^{(r)}) gives 89.29% with half the tokens, because unused cap is left unspent when demand falls below Bcand=16B_{\mathrm{cand}}=16 (App. A.4). Table 9 isolates component-score routing on the same 108 failures with candidate generation and validation held fixed: score-based routing repairs 89.8% of failures and retains 85.3% of solved tasks, vs. 51.4% and 68.9% for random routing.

Component-score guidance fixed, reference trajectory and diff varied. Under the same score-guided routing and budget rule, adding (τ+,Δθ)(\tau^+, \Delta\theta) raises Test-Normal TGC by 7.14 points (86.31% to 93.45%), Test-Normal SGC by 12.50 points (76.79% to 89.29%), and Test-Challenge SGC by 8.16 points (69.78% to 77.94%). We present these as two separate controlled comparisons rather than an additive sum because the two factors interact: (τ+,Δθ)(\tau^+, \Delta\theta) is what makes the component scores accurate (52.8% to 95.7% in Table 8). We will clarify both comparisons in Sec. 4.4 and App. A.4 and add the full 3-run SD table for Table 2 to App. D.

W2 / Q3: Does CATO still outperform GEPA and OPTIMAS under the same total training-token budget?

A3: Yes. CATO leads both baselines at every matched token budget. For each method and budget, we evaluate the best validation-selected configuration reached within that token budget (AppWorld, GLM-5.2, 3-run mean Test-Normal TGC / SGC %):

Training-token budgetGEPAOPTIMASCATO
40M83.53 / 77.9887.30 / 70.2490.67 / 85.12
80M89.68 / 81.5588.29 / 74.4092.46 / 88.10
Full run (141.85M / 250.46M / 113.41M)90.48 / 83.3389.88 / 76.7993.45 / 89.29

Over full runs (Table 3), CATO uses 20.0% fewer training tokens than GEPA and 54.7% fewer than OPTIMAS, because it leaves candidate budget unspent when demand falls below Bcand=16B_{\mathrm{cand}}=16 (App. A.4). It makes 14,851 API calls, 43.1% fewer than OPTIMAS (26,083) but 4.6% more than GEPA (14,201), because it validates each candidate separately. We will add this matched-budget table to App. D, with a one-sentence summary in Sec. 4.6.

We thank you again for your thoughtful feedback and support of our work.

Reviewer_t6i1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CATO optimizes the textual components of an LLM agent, including its system prompt, tool descriptions, and experience bank. It contrasts successful and failed executions of the same task, assigns component-level scores, and uses these scores to allocate candidate edits. Each candidate changes one component, and validation performance determines which configurations survive. Experiments cover AppWorld, WebArena, and ten BBH subtasks with three backbones.

Critical strengths

The paper addresses a practical problem: identifying where execution failed does not necessarily tell an optimizer which component to edit. Using same-task successful executions as diagnostic evidence is a sensible approach, and the method connects this evidence to a concrete search procedure.

The evaluation goes beyond final task accuracy. The controlled fault experiments, repair and success-retention measurements, and component-level ablations help explain the observed gains. Results on AppWorld’s unseen applications are particularly relevant, since configuration edits can easily specialize to the optimization tasks. Using the same backbone for execution and optimization also makes the comparisons easier to interpret

Critical weaknesses

The controlled attribution study is useful, but its scope is narrower than the proposed multi-label repair-target formulation. Ranking the injected faulty component first establishes identification of a known intervention target; it does not fully characterize whether other components could provide equally effective repairs. Moreover, the 108 cases come from 12 faulty configurations. More direct evidence on natural failures would strengthen the claim that the scores guide useful component-level interventions beyond the injected defects.

The random-attribution ablation provides evidence for the value of component selection. However, its description does not explicitly establish whether the successful references and case-level textual gradients are kept identical to those in CATO, or report the actual candidate counts. Clarifying these details would make the contribution of routing, relative to edit generation and budget allocation, easier to assess.

The main result tables report means without run-to-run uncertainty. This makes the smaller improvements, particularly on BBH, difficult to assess.

Questions for authors
  1. On a sample of natural failures, could you compare the repair outcomes obtained by editing each component under an equal candidate budget? Does the attribution ranking predict repair success while preserving previously solved tasks?

  2. In the random-attribution ablation, are successful references and case-level textual gradients retained exactly as in CATO? How do the actual candidate counts and validation executions compare? This clarification would help isolate the value of component selection.

  3. What fraction of failures receive a same-task successful reference during optimization, and how do repair outcomes differ between contrastive diagnosis and the failure-only fallback?

4 comments
AuthorsOpenReview ↗

Response to Reviewer t6i1 (Part 1/2)

We thank the reviewer for the constructive feedback, and for finding same-task successful executions a sensible source of diagnostic evidence and the repair, retention, and unseen-application evaluations informative. We answer each question below.

W1 / Q1: On a sample of natural failures under an equal candidate budget, how do repair outcomes compare across components, and does the attribution ranking predict repair success while preserving previously solved tasks?

A1: On 30 natural failures under an equal budget of 3 candidates per component, the attribution ranking directly predicts repair success and task retention: editing CATO's top-ranked component gives the highest repair rate (91.11%) and retention of previously solved tasks (96.30%), ahead of Rank 2 by 12.22 / 1.97 points and Rank 3 by 28.89 / 7.03 points:

Edited component (90 candidates per row)Failure repair (%)Success retention (%)
CATO Rank 1 (highest sds_d)91.11 (82/90)96.30
CATO Rank 278.89 (71/90)94.33
CATO Rank 3 (lowest sds_d)62.22 (56/90)89.27

Protocol. We sampled 30 distinct task failures from the 3-run AppWorld GLM-5.2 optimization history, kept each failed configuration and its historical attribution scores, ranked the three components by score, and independently generated 3 single-component candidates per component. Under the official AppWorld evaluator, repair is the fraction of the 90 candidates that resolve the target task, and retention is the fraction of baseline-solved tasks (among the other 29 sampled tasks) that remain solved across all candidate–task pairs.

Other components can repair, but not equally effectively. Unlike controlled fault injection (Table 9), where a planted contradiction stays active unless the injected component is edited, natural failures typically stem from missing constraints or workflow knowledge. Adding the missing constraint to another component therefore still resolves the target task in many cases, which aligns with our multi-label formulation (sd>0.6s_d > 0.6). However, other components are not equally effective: relative to Rank 3, editing the top-ranked component raises repair success by 28.89 points and lowers the regression rate on previously solved tasks from 10.73% to 3.70%.

Uncertainty across the 12 faulty configurations. For the 108 controlled cases in Sec. 4.3, a cluster bootstrap resampling the 12 configurations gives a 95% CI of [92.8%, 98.0%] for CATO's top-1 accuracy (95.7%) vs. [38.3%, 63.9%] for Trajectory-only (52.8%). We will summarize these findings in Sec. 4.3 and add the natural-failure table and configuration-clustered CIs to App. C.1.

W2 / Q2: In the random-attribution ablation, are successful references and case-level textual gradients retained exactly as in CATO, and how do the actual candidate counts and validation executions compare?

A2: Yes. Within each step, Random attribution retains successful references (τ+,Δθ\tau^+, \Delta\theta, when available) and case-level textual gradients ∇case(r)\nabla_{\mathrm{case}}^{(r)} exactly as in CATO (running the same three-step attribution call under the same prompt). Only the routing rule changes: each failure goes to one component chosen uniformly at random, without the η=0.6\eta = 0.6 filter (App. A.4). Actual candidate counts and validation executions over the 6-round AppWorld run (GLM-5.2), under the same per-round cap Bcand=16B_{\mathrm{cand}}=16:

MethodAnalyzed failuresRoutedValidated candidatesValidation executionsTokens (M)Test-N SGC (%)
CATO2216181,026113.4189.29 ± 3.09
Random attribution2424211,197127.3671.43 ± 4.72

Random attribution generates 16.7% more candidates and validation executions (57 validation tasks per candidate), yet its Test-Normal SGC is 17.86 points lower. It generates more because it routes every analyzed failure (requesting one candidate per routed failure, with 21 unique candidates validated after deduplication), whereas CATO filters out the 6 of 22 whose scores all fall at or below 0.6, and because its lower repair accuracy leaves more unresolved failures in Rounds 4–6. Because populations diverge after Round 1, Table 9 also isolates routing on the same 108 failures with candidate generation and validation held fixed: random routing repairs 51.4% of failures and retains 68.9% of solved tasks, vs. 89.8% and 85.3% for CATO. We will add these controls and counts to Sec. 4.4 and App. A.4.

AuthorsOpenReview ↗

Response to Reviewer t6i1 (Part 2/2)

Q3: What fraction of failures receive a same-task successful reference during optimization, and how do repair outcomes differ between contrastive diagnosis and the failure-only fallback?

A3: On GLM-5.2, 40.9–52.3% of analyzed failures receive a same-task successful reference (contrast pairs) across all 6 rounds, and 60.0–68.2% in Rounds 2–6. Round 1 starts from a single seed (∣P1∣=1|\mathcal{P}_1|=1), so no reference exists and all of its failures use the failure-only fallback (Table 10):

Benchmark (GLM-5.2)AnalyzedRound 1 (failure-only)R2–R6: with referenceR2–R6: failure-onlyAll-round share with reference
AppWorld2279 (60.0%)6 (40.0%)40.9%
WebArena431022 (66.7%)11 (33.3%)51.2%
BBH (10 subtasks)22252116 (68.2%)54 (31.8%)52.3%

From Round 2 onward, CATO compares the center configuration with its median-distance neighbor on the same 32-task batch; tasks where both succeed are skipped, and failure-only fill-ins (tasks where both fail) are added only when a round has fewer than Nmin⁡=8N_{\min} = 8 contrast pairs (App. A.2). Contrast pairs are concentrated in Rounds 2–3, where most population updates occur (77.8% of analyzed failures on AppWorld, 78.9% on WebArena, 78.4% on BBH); by Rounds 4–6 the population has largely converged and the remaining failures are mostly shared hard tasks.

Repair outcomes. Because in-run fill-ins are harder tasks where both configurations failed, we compare the two diagnostic modes under matched conditions:

  • Same 108 failures, matched candidate generation and validation (Table 9): contrastive diagnosis achieves 89.8% failure repair and 85.3% success retention, vs. 71.2% and 70.9% for the failure-only fallback.
  • End-to-end optimization (Table 2): using failure-only diagnosis throughout (Trajectory-only) lowers Test-Normal SGC from 89.29% to 76.79% (−12.50 points); disabling contrast-pair priority lowers it to 80.36% (−8.93 points).

We will add this breakdown to Table 10 (App. C.2) and clarify the batch filtering in App. A.2.

W3: How reliable are the smaller improvements, particularly on BBH, given run-to-run variation?

A (W3): CATO achieves the highest 3-run mean in all six WebArena and BBH settings (and all 18 settings in Table 1), and all six paired 95% CIs over the strongest baseline exclude zero. 3-run mean ± SD, with CIs from a task-level paired bootstrap pooling the three runs:

SettingStrongest baselineCATOΔ [95% CI]
WebArena SR, GLM-5.252.12 ± 1.21 (ACE)55.76 ± 1.60+3.64 [+1.62, +5.86]
WebArena SR, Qwen3.7-Plus52.73 ± 1.60 (ACE)55.15 ± 1.05+2.42 [+0.81, +4.24]
WebArena SR, DeepSeek-V4-Flash50.91 ± 1.82 (ACE)52.73 ± 1.21+1.82 [+0.61, +3.23]
BBH Macro SR, GLM-5.292.76 ± 0.61 (GEPA)93.93 ± 0.38+1.17 [+0.60, +1.87]
BBH Macro SR, Qwen3.7-Plus93.03 ± 0.34 (GEPA)94.11 ± 0.26+1.08 [+0.53, +1.68]
BBH Macro SR, DeepSeek-V4-Flash91.50 ± 0.35 (GEPA)92.26 ± 0.29+0.76 [+0.33, +1.25]

On BBH, each run is the macro-average of 10 independently optimized subtasks, which keeps CATO's 3-run SD between ±0.26% and ±0.38%. The BBH margins are smaller because 46.6–63.3% of the attribution score mass falls on the system prompt (Table 10), which GEPA already optimizes directly above 91%, whereas AppWorld failures spread across all three components. We will summarize these results in Sec. 4.2 and 4.4 and add the full 18-setting 3-run SD and paired 95% CI tables to App. D.

We thank you again for your constructive comments and helpful suggestions.

Thank you for the detailed response which has largely addressed my concerns. Therefore I will maintain my positive rating.

We sincerely thank you for taking the time to review our response and for your continued support of our work. Your suggestions helped us improve the paper.

Reviewer_cqzbOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The question that this paper asks is about which component of a multi-component LLM agent (system prompt, tool descriptions, experience bank) an optimizer should edit when a task fails. The methodology that CATO uses to answer this question is: it runs the current best configuration and a moderately different one on the same training tasks, and for tasks where only one succeeds, an LLM compares the two runs and the configuration diff, scores each component, and writes a textual fix. More candidate edits go to higher-scoring components, each candidate changes only one component, and the best candidates on a validation set are kept. In a controlled fault-injection study on AppWorld, CATO identifies the faulty component 95.7% of the time vs. 52.8% for diagnosis from the failed trajectory alone. On AppWorld, WebArena, and BBH with three backbones, it beats six optimizers, including GEPA and OPTIMAS, in all 18 settings.

Critical strengths
  1. The problem is real and the method is simple. Existing multi-component optimizers pick which module to edit by round robin fashion (GEPA) or reward-gap estimates (OPTIMAS), not by looking at why a task failed. CATO's design choices are pretty easy to follow, and each one of these design choices is ablated: same-task contrast, scoring components separately from explaining the failure, and one-component edits.

  2. The comparison is broad and the cost accounting part is nice. The paper compares against six reasonable baselines, including the two closest multi-component optimizers, with three backbones and an unseen-app split. CATO is best in all 18 cells while using fewer training tokens than most baselines (Table 3). So the gains do not come from extra compute.

Critical weaknesses

#1. The main evidence for the attribution claim (i.e. the fault-injection study) might be set up in a way that favors CATO. The successful reference is the clean pre-injection configuration, so the configuration diff contains the fault by construction.

The injected faults are also explicit contradictions (e.g., "The amount parameter is in cents"). CATO sees this diff and the successful trajectory, but the trajectory-only baseline sees neither. So 95.7% vs. 52.8% mostly shows that a diff against a clean configuration helps. It might not show that contrasting neighbors during a real optimization run localizes faults. In particular, the fault is never placed in a component shared by both sides, which is the common case during optimization.

#2. It is hard to tell how large the gains are and whether contrast is what produces them. Table 1 reports three-run means with no variance. Outside AppWorld the margins are small: +0.8 to +1.2 points on BBH and +1.8 to +3.6 on WebArena, i.e., 3 to 6 of 165 test tasks. All hyperparameters were tuned on AppWorld with GLM-5.2, the setting with the largest gain.

And also the sweeps in App. D report only test scores, on which the default is best in every sweep. Figure 6a also suggests large run-to-run variance. After the first round, which uses the same failure-only diagnosis as the Trajectory-only ablation, mean TGC is already about 90.5, higher than that ablation's final 86.31. Without error bars, it is hard to read the ablation gaps in Table 2 as the value of contrast.

Questions for authors
  1. In the fault-injection study, what attribution accuracy do you get with (a) the diff but no successful trajectory, (b) the successful trajectory but no diff, and (c) a fault in a component that is the same on both sides? What do the neutral edits look like?

  2. Can you report standard deviations or confidence intervals for Tables 1 and 2, and the validation scores behind the sweeps in Tables 11 and 12? Were the defaults fixed before looking at test scores?

  3. How do you reconcile Figure 6a (about 90.5 TGC after a failure-only first round) with the Trajectory-only ablation's 86.31?

  4. Table 10 lists only 22, 16, and 25 analyzed failures over the whole AppWorld run. How many of these were real contrast pairs rather than failure-only fill-ins?

  5. How were GEPA and OPTIMAS adapted to edit tool descriptions and experience banks, and how were their hyperparameters set?

5 comments
AuthorsOpenReview ↗

Response to Reviewer cqzb (Part 1/3)

We thank the reviewer for the careful reading of the main text and the appendix, and for finding the problem real, each design choice ablated, and the comparison and cost accounting informative. We answer each question below.

W1 / Q1: In the fault-injection study, what attribution accuracy do you get with (a) the diff but no successful trajectory, (b) the successful trajectory but no diff, and (c) a fault in a component that is the same on both sides, and what do the neutral edits look like?

A1: In the 108-case study (App. C.1), the pre-injection reference differs on all three components (1 fault + 2 verified neutral edits), so the diff (Δθ\Delta\theta) and the successful trajectory (τ+\tau^+) contribute jointly. Separating them shows that neither (a) the diff alone (73.1%) nor (b) the successful trajectory alone (76.9%) approaches Full CATO (95.7%), and in (c) CATO still outperforms Trajectory-only (68.6% vs. 49.0%) when the faulty component is the same on both sides:

108 controlled cases, GLM-5.2, strict top-1 (%)OverallPrompt (38)Tools (34)Experience (36)
Trajectory-only (Table 8)52.868.417.669.4
(a) Diff, no successful trajectory (Δθ\Delta\theta only)73.176.350.091.7
(b) Successful trajectory, no diff (τ+\tau^+ only)76.986.847.194.4
Full CATO (Table 8)95.798.293.195.4

In (b), no successful-side configuration text remains, because CATO never passes θ+\theta^+ to the model (App. F.2). The two inputs are most complementary on Tools (50.0% and 47.1% alone vs. 93.1% combined): locating a faulty tool description requires seeing both where the tool calls diverge between τ−\tau^- and τ+\tau^+ and which description clause governs that call.

(c) Fault in a component that is the same on both sides. We constructed 51 paired cases in which the successful reference has the same faulty component as the failed side (unchanged in Δθ\Delta\theta), so Δθ\Delta\theta shows diffs only on the two non-faulty components. Accuracy is lower than in the 108-case study, as expected when the diff points away from the fault, but CATO still ranks the faulty component first in 68.6% of cases vs. 49.0% for Trajectory-only, because the attribution prompt evaluates each component's repair ownership whether changed or unchanged (App. F.2, Listing 5).

Neutral edits. Neutral edits alter realistic operational rules in the same domain and are verified individually and jointly to preserve task success. In the App. C.3 Venmo task (splitting yesterday's $73 taxi and $122 dinner bills and paying Nancy $30.50), all three edits match the task's surface terms (yesterday, Venmo, payments):

  • Prompt (neutral — "yesterday" range): 00:00:00 to 23:59:59 → past 24 hours ending at the current time. Not triggered ($73 and $122 totals are already given).
  • Tools (fault — venmo.create_transaction): add The amount parameter is in cents. Triggered (passes amount=3050 instead of 30.50 and fails).
  • Experience (neutral — Venmo history with a person): both payments sent and received → only payments sent unless incoming are requested. Not triggered (creates new requests/payments without querying history).

Beyond explicit contradictions, the 108 cases span deleted operational constraints, corrupted parameter specifications, and stale or misleading experience guidelines (Table 7). On natural optimization failures with no injected fault (Table 2), removing contrastive attribution (Trajectory-only) still lowers Test-Normal TGC by 7.14 and SGC by 12.50 points. We will state in App. C.1 that the 108-case study evaluates Δθ\Delta\theta and τ+\tau^+ jointly while the 51-case study isolates trajectory contrast on unchanged components, and add these results and neutral-edit examples to App. C.1 and C.3.

AuthorsOpenReview ↗

Response to Reviewer cqzb (Part 2/3)

W2 / Q2a: What are the validation scores behind Tables 11–12, and were the defaults fixed before test evaluation?

A2a: The validation scores behind all six sweeps are reported below, and all defaults were fixed once by 3-run mean validation TGC on AppWorld with GLM-5.2 (57 validation tasks) before running any test split and transferred without re-tuning to the other backbones and benchmarks (Sec. 4.7, App. B.1; ∗^* = default):

SweepTested values3-run Val TGC (%)
Population NpopN_{\mathrm{pop}}3 / 5$^*$ / 791.23 / 94.74 / 92.98
Contrast set kk2$^*$ / 3 / 494.74 / 92.40 / 91.81
Threshold η\eta0.3 / 0.5 / 0.6$^*$ / 0.889.47 / 91.81 / 94.74 / 86.55
Budget BcandB_{\mathrm{cand}}6 / 16$^*$ / 2490.06 / 94.74 / 94.74
Contrast pairs Nmin⁡N_{\min}6 / 8$^*$ / 1090.64 / 94.74 / 92.98
Neighbor distanceNear / Middle$^*$ / Far92.98 / 94.74 / 91.23

The default achieves the highest validation TGC (94.74%, i.e., 54.0/57 tasks) in all six sweeps; Bcand=16B_{\mathrm{cand}}=16 and 24 tie, and we kept 16 for its lower training cost (113.41M vs. 118.81M tokens). DeepSeek-V4-Flash was never used for hyperparameter selection, yet its AppWorld SGC margins over the strongest baseline (+5.36 Test-Normal, +5.52 Test-Challenge) are comparable to those of GLM-5.2 (+5.96, +4.08). We will add the validation column to Tables 11–12 (App. D.1–D.2).

W2 / Q2b: What are the standard deviations and confidence intervals for Tables 1 and 2?

A2b: CATO achieves the highest 3-run mean across all 18 settings in Table 1, and all 18 paired 95% CIs over the strongest competing baseline exclude zero. Below are the six WebArena and BBH settings you flagged as small (3-run mean ± SD; task-level paired bootstrap pooling the three runs, scenario-level for SGC):

SettingStrongest baselineCATOΔ [95% CI]
WebArena SR, GLM-5.252.12 ± 1.21 (ACE)55.76 ± 1.60+3.64 [+1.62, +5.86]
WebArena SR, Qwen3.7-Plus52.73 ± 1.60 (ACE)55.15 ± 1.05+2.42 [+0.81, +4.24]
WebArena SR, DeepSeek-V4-Flash50.91 ± 1.82 (ACE)52.73 ± 1.21+1.82 [+0.61, +3.23]
BBH Macro SR, GLM-5.292.76 ± 0.61 (GEPA)93.93 ± 0.38+1.17 [+0.60, +1.87]
BBH Macro SR, Qwen3.7-Plus93.03 ± 0.34 (GEPA)94.11 ± 0.26+1.08 [+0.53, +1.68]
BBH Macro SR, DeepSeek-V4-Flash91.50 ± 0.35 (GEPA)92.26 ± 0.29+0.76 [+0.33, +1.25]

On AppWorld with GLM-5.2, the margins over GEPA are +2.97 TGC [+1.19, +4.96] and +5.96 SGC [+1.79, +10.12] on Test-Normal and +4.08 SGC [+1.68, +6.71] on Test-Challenge. On BBH, each run averages 10 independently optimized subtasks (CATO's 3-run SD: ±0.26–0.38%); the margins are smaller there because 46.6–63.3% of the attribution score mass falls on the prompt (Table 10), which GEPA already optimizes above 91%.

In Table 2 (AppWorld, GLM-5.2, 3-run mean ± SD), the drops from removing contrastive diagnosis (Trajectory-only: −7.14 TGC / −12.50 SGC) or contrast-pair priority (−4.76 / −8.93) are roughly three times the run-to-run SDs or larger:

Table 2 variantTest-Normal TGC (%)Test-Normal SGC (%)
Full CATO93.45 ± 1.1989.29 ± 3.09
Trajectory-only86.31 ± 2.1576.79 ± 3.57
w/o Contrast-pair priority88.69 ± 1.5780.36 ± 1.79
Random attribution84.92 ± 2.4171.43 ± 4.72
All-component assignment88.89 ± 1.5081.55 ± 2.73
w/o Median-distance neighbor90.87 ± 1.8285.12 ± 3.72
w/o Score-based budget91.67 ± 1.0385.12 ± 2.73
w/o Gradient consolidation89.88 ± 2.1580.36 ± 3.57
w/o Single-component edits90.28 ± 1.5084.52 ± 3.72

Full 3-run SD tables for Tables 1 and 2 and the 18-setting paired 95% CI table for Table 1 will be added to App. D.

AuthorsOpenReview ↗

Response to Reviewer cqzb (Part 3/3)

W2 / Q3: How do you reconcile Figure 6a (about 90.5 TGC after Round 1) with Trajectory-only's 86.31 in Table 2?

A3: The two numbers come from different evaluation quantities, and under the same Table 2 protocol both variants are at a similar level after Round 1 (90.08% vs. 88.49% TGC) before diverging in Rounds 2–6:

  1. Different quantities. Figure 6a plots the mean across the Npop=5N_{\mathrm{pop}}=5 population members in a single 10-round run, and its shading is the within-run spread across members rather than variability across independent runs (App. D.6). Table 2 reports the validation-selected configuration averaged over three 6-round runs. We will state this in the Figure 6 caption.
  2. Rounds 2–6. Round 1 lifts both variants from the 81.55% ReAct seed to 90.08% Test-Normal TGC for CATO (88.69–91.07% across runs) and 88.49% for Trajectory-only (86.90–89.88%). With same-task references, CATO then rises by 3.37 points to 93.45%. Without them, Trajectory-only more often edits the wrong component: candidates that pass or tie on the 57-task validation split regress on held-out test tasks (70.9% vs. 85.3% Test-Normal retention after validation selection in Table 9), so the validation-selected configuration drops by 2.18 points to 86.31%.

Q4: Among the 22, 16, and 25 analyzed AppWorld failures in Table 10, how many were real contrast pairs rather than failure-only fill-ins?

A4: Real contrast pairs account for 9/22 (40.9%), 6/16 (37.5%), and 8/25 (32.0%) of analyzed failures across all rounds on GLM-5.2, Qwen3.7-Plus, and DeepSeek-V4-Flash, and are the majority in Rounds 2–6 (53.3–60.0%). Round 1 starts from a single seed (∣P1∣=1|\mathcal{P}_1|=1), so no reference configuration exists:

BackboneAnalyzedRound 1 (failure-only)Rounds 2–6: contrast pairsRounds 2–6: fill-ins
GLM-5.22279 (60.0%)6 (40.0%)
Qwen3.7-Plus1656 (54.5%)5 (45.5%)
DeepSeek-V4-Flash25108 (53.3%)7 (46.7%)

The totals are small because AppWorld configurations already solve 85–93% of tasks, so most tasks in each 32-task batch succeed on both sides and are skipped (App. A.2). Contrast pairs are concentrated in Rounds 2–3, where most population updates occur (77.8%, 66.7%, and 55.6% of analyzed failures on the three backbones); by Rounds 4–6, configurations have largely converged and most remaining failures are hard tasks shared by both sides. We will add this breakdown to Table 10 (App. C.2).

Q5: How were GEPA and OPTIMAS adapted to edit tool descriptions and experience banks, and how were their hyperparameters set?

A5: Both baselines operate on the same three-component editable space and single-component edit operators as CATO (Table 5, App. A.5)—full rewrites for the system prompt, description updates indexed by fixed tool IDs with the underlying tools and API schemas unchanged, and entry-level add/update/delete on ID-indexed experience entries. Rather than re-tuning the baselines on AppWorld, we kept their official defaults while aligning evaluation or candidate budgets to CATO's scale: GEPA uses reflection_minibatch_size = 3 and max_merge_invocations = 5 (as in the original paper) with an evaluation budget of max_metric_calls = 1000 (matching CATO's 1,026 validation evaluations); OPTIMAS runs num_iterations = 10 on train_subset_size = 50, and we increased num_candidates per round from 3 to 5 (matching Npop=5N_{\mathrm{pop}}=5) to widen its search. We will add a hyperparameter table to App. B.3.

We hope these results address your concerns and would be grateful if you would consider updating your rating.

Comments on the paper

2 comments
Area_Chair_qAN9OpenReview ↗

Reminder: Follow-up on Author Rebuttal

I noticed that cqzb and 9s8q have not yet provided follow-up feedback. As the discussion period is approaching its end, I would appreciate it if both reviewers could review the rebuttal and revised manuscript, particularly the additional experiments on multi-component failures, attribution effectiveness, and matched-budget comparisons. Please share whether your concerns have been addressed and update your ratings if warranted.

We sincerely thank the Area Chair for managing the review process and following up during the discussion. We also thank all four reviewers for their constructive feedback. We are encouraged by the supportive assessment across the reviews and follow-up responses, and glad that our additional experiments and clarifications have addressed the main concerns. All new results and revisions have been incorporated into the updated manuscript, which has meaningfully strengthened the paper. Thank you again for your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.