acceptodds
Under review as a conference paper at ICLR 2027

MICA: Harness Self-Evolution through Intervention-Guided Co-Adaptation

Abstract

Harness self-evolution refines runtime instructions through persistent context edits, but individually beneficial edits can fail in combination because one edit can change how another is interpreted or executed. We introduce MICA (Matched Intervention-guided Co-Adaptation), which evolves the harness by jointly adapting edit structures and templates while keeping language models frozen. Matched executions guide a frozen language model to propose composition-mismatch hypotheses and minimal template repairs. With edit structure fixed, MICA compares the original template pair, each unilateral revision, and the joint revision to measure the benefit of co-adaptation. Competing structures receive the same bounded adaptation allowance before execution-based selection, followed by frozen screening and independent deployment confirmation. Across five environments, MICA achieves macro-average scores of 80.90% with GPT-5.5 and 76.80% with DeepSeek-V4-Flash, exceeding the strongest baselines by 4.82 and 1.72 percentage points, respectively. Under common budgets, MICA improves deployed scores by 3.25 percentage points on Telecom and 4.23 percentage points on AppWorld Normal over the strongest search baseline. Controlled studies demonstrate that matched evidence and paired repair each improve deployment reward, establishing harness co-adaptation as an effective approach to harness self-evolution.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, harness self-evolution, context engineering, matched interventions, co-adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_59aKOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MICA, an intervention-guided framework for agent-harness self-evolution that addresses interactions among context modifications. The central motivation is that a revision that improves one harness component in isolation may behave differently when combined with other revisions. MICA uses matched execution evidence to diagnose potential composition mismatches and applies coordinated template repair rather than treating individual edits as independent.

The approach is evaluated across five environments with GPT-5.5 and DeepSeek-V4-Flash. The paper reports macro-average task scores of 80.90% and 76.80%, respectively. In addition to comparisons with context-learning and harness-optimization baselines, the paper includes controlled mechanism studies using 20 independent optimization runs per configuration on Telecom and AppWorld Normal. These studies compare failure traces against matched interventions and sequential against paired repair.

Critical strengths
  1. The paper addresses an important limitation of component-wise optimization: locally beneficial changes may not remain beneficial when combined. This is more specific than the general objective of finding a better prompt or harness, and it provides a useful conceptual framework for understanding failures during iterative agent improvement.

  2. Table 4 uses a controlled comparison of two sources of diagnostic evidence and two repair strategies over 20 optimization seeds. Matched interventions improve outcomes relative to ordinary failure traces, while paired repair also improves performance relative to sequential repair under the tested settings. Table 5 further shows that the jointly revised template pair outperforms the stronger unilateral revision. These experiments give substantially more direct support to the proposed mechanism than headline benchmark scores alone.

Critical weaknesses
  1. The four-condition intervention analysis is well suited to evaluating two interacting edits, but more complex harnesses may exhibit dependencies involving three or more components. Such higher-order conflicts need not be detectable from pairwise comparisons. The paper does not sufficiently establish how the method handles these situations or how intervention cost scales with the number of mutually dependent components. This limits the scope of the general co-adaptation claim.

  2. The main benchmark tables report one optimization seed per method or configuration, whereas several mechanism experiments use 20 seeds. The latter materially strengthen the paper, but they cannot fully establish the stability of every headline comparison, particularly the relatively small average advantage under DeepSeek-V4-Flash. Furthermore, joint repair outperforming unilateral repair should not automatically be interpreted as evidence of a statistically significant positive interaction between the two mechanisms.

Questions for authors
  1. How would the proposed intervention mechanism identify and repair failures involving three or more interacting harness components, especially if no individual pair exhibits a clear regression? Please clarify both the algorithmic extension and its execution cost.

  2. Can the authors provide uncertainty estimates or additional optimization-seed evidence for the main five-environment comparisons? This is particularly relevant to the smaller average improvement reported for DeepSeek-V4-Flash.

  3. For the factorial mechanism study in Table 4, do the matched-intervention and paired-repair mechanisms exhibit a statistically supported interaction beyond their separate effects? Clarifying this would help distinguish the benefit of combining two individually useful components from a stronger synergy claim.

3 comments
AuthorsOpenReview ↗

Reply - Part 1

We thank the reviewer for these constructive comments.

W1 + Q1. Higher-order composition conflicts and intervention cost

The original MICA implementation is explicitly limited to two-edit programs; pairwise interventions alone cannot guarantee detection of a purely three-way conflict. Sections 3.1–3.4 and Appendices B.1/D.1 set the maximum program length to mmax⁡=2m_{\max}=2; the compiler rejects dependency chains requiring three edits. A failure that appears only when a,b,ca,b,c are all present may therefore be invisible to every pairwise comparison in the reported two-edit experiments. We agree that the general co-adaptation claim must be read within this scope.

1. Algorithmic extension: test the complete triple, not only its pairs

For three independent edits, one direct extension is to execute the eight presence/absence configurations, using a fixed legal order and separately restored, matched initial states:

S3=∅,a,b,c,ab,ac,bc,abc.\mathcal{S}_3=\\{\varnothing,a,b,c,ab,ac,bc,abc\\}.

Let RSR_S be the task-balanced reward for subset SS. The third-order contrast is

Iabc(3)=Rabc−Rab−Rac−Rbc+Ra+Rb+Rc−R∅.I^{(3)}_{abc}=R_{abc}-R_{ab}-R_{ac}-R_{bc}+R_a+R_b+R_c-R_{\varnothing}.

This contrast can reveal a non-additive three-way effect even when all pairwise contrasts are near zero. In addition to rewards, the diagnostic model would inspect aligned traces, compiler events, edit activation, and task-constraint violations to formulate an evidence-linked repair hypothesis. A triple must not be excluded solely because its individual pairs show no regression; a bounded higher-order exploration allocation would need to sample eligible triples even without a negative pairwise signal. For order-sensitive edits, selected legal orderings must be evaluated separately. For dependent edits, only legal subsets and prefixes can execute; when some of the eight cells are illegal, the full third-order contrast is not identifiable and the diagnosis must instead use legal conditional comparisons and activation evidence.

Repair is a separate operation from presence/absence diagnosis. Once a legal three-edit structure GG is fixed, a proposal revising three templates can be evaluated using the 23=82^3=8 original/revised combinations on the same fitting tasks and repeat indices. Every hybrid must preserve its interface and compile successfully. As with the two-edit method, actual measured reward—not the diagnostic model's prediction or the sign of an interaction contrast—determines version retention, followed by frozen screening and independent confirmation. Selective evaluation cannot guarantee finding every higher-order conflict unless the relevant triple is actually tested.

2. Execution cost and additional three-edit evidence

A fixed-order, three-edit presence/absence block requires up to 8 task executions per task/repeat; checking additional orders increases this cost. For comparison, the current independent-pair matched block contains 5 executions (both orders), and a dependent-pair block contains 3 (Section 3.3). For fixed-structure repair with mm revised templates, full factorial comparison requires

Nrepair(m)=2m,∣H∣,RfitN_{\mathrm{repair}}(m)=2^m\\,|H|\\,R_{\mathrm{fit}}

fresh task executions per repair hypothesis, excluding proposal generation, broader structure search, screening, and confirmation. Under the current controlled-study panels (12 Telecom tasks or 18 AppWorld Normal tasks; Rfit=2R_{\mathrm{fit}}=2), moving from two to three templates doubles the per-hypothesis repair cost: 96 to 192 rollouts on Telecom and 144 to 288 on AppWorld Normal. Exhaustive consideration of all mm-edit subsets from a library of size BB adds a (Bm)\binom{B}{m} candidate factor, before legal orders, bindings, and dependencies. This is why we do not claim unrestricted higher-order scalability.

Our additional experiments implement a bounded three-edit extension, including legal three-edit structures and a selective factorial strategy. They use the following larger per-seed caps:

EnvironmentRolloutsLM callsTokens (M)
Telecom3,381105,731438.791
AppWorld Normal4,837113,811931.277
Method under the new capsTelecomAppWorld Normal
2-edit MICA (rerun)85.00%82.34%
3-edit Greedy Sequential85.83%83.33%
3-edit Local Pair Repair87.50%85.32%
3-edit Selective Factorial90.00%88.69%

The selective factorial variant exceeds local pair repair by 2.50 and 3.37 percentage points, respectively, under the reported new caps. This demonstrates a feasible, bounded three-edit extension, not a general scaling solution or a guarantee of detecting every pure higher-order conflict. Larger edit sets will require selectively testing suspected interacting subsets, reserving exploration for triples without pairwise warning signals, and validating the resulting complete program.

AuthorsOpenReview ↗

Reply - Part 2

W2 + Q2. Optimization-seed uncertainty in the five-environment comparisons

The original main benchmark tables use one optimization seed per method/configuration. The 20-seed mechanism and efficiency experiments do not substitute for external-baseline replication across all five environments. We therefore also report a three-seed, five-environment comparison. The runs use common per-seed resource caps (not necessarily equal actual resource consumption), and scores are macro-averaged over the same five environments within each seed before calculating their across-seed mean and standard deviation:

\bar{A}_m=\frac{1}{3}\sum_{s=1}^{3}A_{m,s}.$$ | Method | GPT-5.5 mean ± SD (%) | DeepSeek-V4-Flash mean ± SD (%) | |---|---:|---:| | ACE | 73.49 ± 0.32 | 67.33 ± 0.32 | | GEPA | 76.23 ± 0.23 | 71.84 ± 0.09 | | SkillOpt | 75.85 ± 0.20 | 75.03 ± 0.28 | | RHO | 72.19 ± 0.29 | 70.95 ± 0.25 | | **MICA** | **80.78 ± 0.60** | **77.00 ± 0.39** | The comparison with the strongest baseline for each model is clearer when the difference is computed within matched seeds: | Paired macro-average difference | Seed 1 | Seed 2 | Seed 3 | Mean ± seed SD | |---|---:|---:|---:|---:| | GPT-5.5: MICA − GEPA | +4.81 pp | +3.63 pp | +5.20 pp | **+4.55 ± 0.82 pp** | | DeepSeek-V4-Flash: MICA − SkillOpt | +1.72 pp | +2.16 pp | +2.02 pp | **+1.97 ± 0.22 pp** | **All three observed seed-level advantages are positive for both models.** The DeepSeek advantage remains smaller than the GPT-5.5 advantage, but is consistently positive across the three observed seeds. This extends the original single-seed gaps of **+4.82 pp** and **+1.72 pp** (Tables 1–2) to three-seed mean gaps of **+4.55 pp** and **+1.97 pp**.
AuthorsOpenReview ↗

Reply - Part 3

Q3. Do matched interventions and paired repair exhibit a statistically supported interaction?

The additional seed-level analysis supports positive main effects of both mechanisms, but does not establish a statistically significant positive interaction.

Thank you for highlighting this distinction. We conducted a factorial interaction analysis using the complete results from 20 independent optimization seeds for each of the four configurations in Table 4.

For each seed, we computed the difference-in-differences:

I = (M+P − M+S) − (F+P − F+S)

We then applied a two-sided paired-seed t-test to the 20 interaction contrasts, using the optimization seed rather than individual task outcomes as the independent statistical unit.

StatisticTelecomAppWorld Normal
Matched-evidence main effect+4.63 pp+5.73 pp
Paired-repair main effect+2.75 pp+3.65 pp
Factorial interaction+0.25 pp+1.64 pp
95% CI for interaction[−3.87, 4.37][−1.28, 4.55]
Two-sided p-value0.9000.254

Both mechanisms have statistically positive average main effects, whereas neither interaction estimate is statistically distinguishable from zero.

MICA still achieves the highest 20-seed mean deployed score among the four configurations, reaching 84.50% on Telecom and 81.07% on AppWorld Normal. However, these results support the separate benefits of matched interventions and paired repair, rather than a stronger claim of statistically significant positive synergy.

Reviewer_SahhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MICA improves context-editing rules for frozen agents by comparing controlled executions, revising interacting templates, and evaluating joint and one-sided changes. The paper reports results across five environments.

Critical strengths

The 20-run controlled studies are a strength: they separate the effects of matched evidence and joint repair. The sequential baseline can also revise both templates under the same budget. Keeping screening and confirmation feedback out of the repair loop is another sensible design choice.

Critical weaknesses
  1. My main concern is the single optimization seed used in Tables 1–3. The 20-run studies support the design choices, but do not establish robustness against GEPA [1], SkillOpt [2], or RHO. The 1.72-point average gain with DeepSeek-V4-Flash is encouraging, but its variation across runs is unclear.

  2. The interaction-repair claim would be easier to assess with a real example. Table 5 shows average gains, but not how often joint revision beats both one-sided revisions. Figure 1 is illustrative.

Questions for authors
  1. Could you repeat the comparison with the strongest baseline on one representative environment, using comparable budgets, and report variation across optimization seeds?

  2. Could you show one real repair with the old and new templates and all four outcomes? From the existing results, how often does joint revision beat both one-sided revisions?

  3. If the existing logs allow a comparison, do gains start to level off over the three search rounds? A brief summary would be useful; I am not asking for additional runs here.

References

[1] Agrawal et al. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. ICLR 2026. https://arxiv.org/abs/2507.19457

[2] Yang et al. SkillOpt: Executive Strategy for Self-Evolving Agent Skills. 2026. https://arxiv.org/abs/2605.23904

4 comments
AuthorsOpenReview ↗

Reply - Part 1

We greatly appreciate your recognition of the paper's strengths and your insightful criticism of its weaknesses.

Regarding your criticisms of the paper's weaknesses and questions, we respectfully note that:

W1 + Q1. Variation across optimization seeds

The additional three-seed comparison preserves MICA's macro-average advantage: +4.55 pp with GPT-5.5 and +1.97 pp with DeepSeek-V4-Flash. The original 20-run internal studies do not substitute for this external-baseline replication.

For the five environments, scores are macro-averaged within each seed, then summarized across three seeds:

\bar A_m=\frac{1}{3}\sum_{s=1}^{3}A_{m,s},\qquad \mathrm{SD}(A_m)=\sqrt{\frac{\sum_s(A_{m,s}-\bar A_m)^2}{2}}.$$ | Method | GPT-5.5 mean ± SD (%) | DeepSeek-V4-Flash mean ± SD (%) | |---|---:|---:| | ACE | 73.49 ± 0.32 | 67.33 ± 0.32 | | GEPA | 76.23 ± 0.23 | 71.84 ± 0.09 | | SkillOpt | 75.85 ± 0.20 | 75.03 ± 0.28 | | RHO | 72.19 ± 0.29 | 70.95 ± 0.25 | | **MICA** | **80.78 ± 0.60** | **77.00 ± 0.39** | | Comparison | Seed 1 | Seed 2 | Seed 3 | Mean difference | |---|---:|---:|---:|---:| | GPT-5.5: MICA − GEPA | +4.81 pp | +3.63 pp | +5.20 pp | **+4.55 pp** | | DeepSeek: MICA − SkillOpt | +1.72 pp | +2.16 pp | +2.02 pp | **+1.97 pp** | ## W2 + Q2. A real repair and how often joint revision wins **In the AppWorld Todoist repair, the joint revision succeeds where both unilateral revisions fail on the trigger task, and improves the fitting-panel reward by 19.44 pp over the better unilateral revision.** The structure is fixed; only the two templates change. ### 1. Exact templates | Edit | Original | Revised | |---|---|---| | $a$: evidence retention | After reading a task-list result, summarize the relevant items briefly. Omit repetitive task identifiers to keep the context concise. | Maintain an explicit ledger of every qualifying task identifier and the required update. Carry unprocessed identifiers across subsequent tool calls; never replace the ledger with a prose-only summary. | | $b$: completion rule | Once the currently listed tasks have been handled, provide a completion message and avoid unnecessary tool calls. | Report completion only after every identifier in the ledger has been updated, the changes have been checked by a fresh read, and no additional qualifying items remain; otherwise continue. | The mismatch concerns **retaining all required identifiers and deciding when processing is complete**. The revised templates instruct the agent to maintain a ledger and check completion against it; this is not a new compiler-maintained state mechanism. The two unilateral results help explain why both changes matter in this case. Preserving identifiers does not by itself prevent the original stopping rule from ending the task prematurely; strengthening the stopping rule does not recover identifiers that the context has already omitted. Joint revision makes the information retained by one edit usable by the completion criterion of the other. **This is a concrete compatibility issue between the templates**, not a claim that every repair requires simultaneous changes. ### 2. Four measured conditions | Condition | Trigger task | Reward on 18 tasks × 2 repeats | Observed behavior | |---|---|---:|---| | $P_{00}$ | Fail | 47.22% | Missing identifiers and early stopping | | $P_{10}$ | Fail | 63.89% | Ledger retained, early stopping remains | | $P_{01}$ | Fail | 66.67% | Stronger completion check, incomplete identifiers | | **$P_{11}$** | **Pass** | **86.11%** | Complete updates and fresh-read verification | $$\Delta_{\mathrm{case}}=Q_{11}-\max(Q_{10},Q_{01})=\mathbf{19.44\\,pp}.$$ This case-specific panel contrast differs from the original Table 5 AppWorld aggregate contrast of 14.44 pp. ### 3. Proposal-level frequency $$\mathrm{Win}(h)=\mathbb{1}\\!\left[Q_{11}^{(h)}>\max\\{Q_{10}^{(h)},Q_{01}^{(h)}\\}\right].$$ | Outcome relative to the better unilateral revision | Proposals | Frequency | |---|---:|---:| | **Strictly better** | **29** | **78.38%** | | Tie | 4 | 10.81% | | Worse | 4 | 10.81% | | Total | 37 | 100.00% | **The denominator is 37 repair proposals, not seeds or task executions.** Joint revision is not always preferable, which is why MICA compares all four measured versions rather than automatically accepting $P_{11}$.
AuthorsOpenReview ↗

Reply - Part 2

Q3. Do gains level off over three rounds?

Yes, the existing logs show diminishing incremental fitting gains over the three rounds. No additional optimization runs are used. For each round, the retained candidate and incoming incumbent are compared on that round's same panel:

\bar g_r=\frac{1}{20}\sum_{s=1}^{20}g_{s,r}.$$ | Environment | Round 1 gain | Round 2 gain | Round 3 gain | Positive-gain seeds (R1/R2/R3) | |---|---:|---:|---:|---:| | Telecom | 8.96 pp | 4.58 pp | **2.71 pp** | 17/20, 12/20, **8/20** | | AppWorld Normal | 10.28 pp | 6.39 pp | **2.92 pp** | 18/20, 13/20, **7/20** | Using within-round contrasts avoids treating a change in panel difficulty as a search improvement. The positive-gain counts complement the mean: a smaller average alone could conceal improvements concentrated in a few seeds, whereas the declining counts show that improvements also occur in fewer runs. The third round still has positive mean gains in both environments, so the evidence does not support claiming that further search is already useless.

Dear Reviewer,

If you have any further questions or concerns, please do not hesitate to reach out. We would be more than happy to continue the discussion with you. Thank you very much for your valuable time and effort in reviewing our work.

Warm regards, The Authors

Dear Reviewer Sahh,

Thank you again for your careful review and constructive feedback. We sincerely appreciate your time and consideration. We would like to briefly reiterate our responses to your two main concerns and highlight additional experimental evidence that may further support our conclusions.

1. Robustness across optimization seeds. As detailed in our rebuttal, the additional three-seed, five-environment experiments confirm that MICA consistently outperforms the strongest baselines under common resource caps. Specifically, MICA achieves 80.78 ± 0.60% vs. 76.23 ± 0.23% for GEPA with GPT-5.5, and 77.00 ± 0.39% vs. 75.03 ± 0.28% for SkillOpt with DeepSeek-V4-Flash. The improvements are positive across all three seeds for both models (+4.55 pp and +1.97 pp on average).

2. Effectiveness of joint template repair. Beyond the concrete AppWorld Todoist example provided in our rebuttal, where joint repair achieves 86.11%, compared with 63.89% and 66.67% for the two unilateral revisions, our proposal-level analysis shows that joint repair outperforms the better unilateral revision in 29 out of 37 cases (78.38%).

Importantly, these findings are further supported by the original paper's 20-seed controlled experiments (Tables 4–6). MICA achieves deployed scores of 84.50% on Telecom and 81.07% on AppWorld Normal in the mechanism study, demonstrating the benefits of matched evidence and paired repair. Moreover, the budget-controlled search experiments show improvements of +3.25 pp and +4.23 pp over the strongest search control, respectively. These independent experimental results provide additional evidence that MICA's improvements are not limited to a particular repair example or optimization run.

Given these clarifications and the additional empirical evidence, we would be sincerely grateful if you could kindly reconsider your assessment. We also note that the other two reviewers have provided positive assessments of our work. Of course, we fully respect your independent judgment and understand that the final evaluation is entirely at your discretion.

Thank you once again for your thoughtful feedback and valuable time.

Best regards, The Authors

Reviewer_tzsXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that, in harness self-evolution, where improving an agent’s prompt-level context while keeping the LM frozen, edits that each help on their own can fail when combined. To address this, the authors proposed MICA, that runs the activation and ordering combinations of two edits ({pi, a, b, ab, ba}) from identical initial states to collect evidence. A frozen LM then proposes joint revisions of both templates. MICA evaluates four versions (original, each one-sided revision, and the joint revision) and keeps the one with the highest measured reward. Final candidates are frozen and go through screening and a one-shot confirmation on held-out data before deployment.

Critical strengths
  • The protocol of evaluation and deployment is well-designed. Data are strictly partitioned into fitting, screening, confirmation, and evaluation sets. After candidates are frozen, confirmation is performed only once, and a candidate is deployed only if the one-sided lower confidence bound on its improvement over the base agent exceeds zero; otherwise the system falls back to the base agent. This is close to a pre-registered design, which is rare in the self-evolution literature. The cost accounting is also thorough and includes control branches, intermediate versions, and retries.

  • Cost-controlled ablation provides the effectiveness by cleanly isolating the effect of matched evidence. Every variant is charged the same evidence-collection cost, and the only difference is whether the diagnostic LM sees the aligned control branches. Matched evidence improves deployed scores over failure-trace-only reflection on AppWorld. The ranking of the four variants is consistent across both environments and 20 seeds.

Critical weaknesses
  • There is a large gap between the claimed contribution and the actual mechanism. The core mechanism behind “intervention-guided co-adaptation” does not work the way its name suggests. The “interventions” are not counterfactuals in this paper. Specifially, they are repeated runs that share only the initial state. LM sampling and the user simulator are independent across branches, and only two repeats are used.

  • The scope is much narrower than the framing suggests. The deployed artifact is at most two conditional prompt snippets (m_max = 2), drawn from 2–4 initialized primitives, which gives roughly 11 legal structures in total. MICA cannot change tools, control flow, or anything beyond appending or prepending text to message copies. The title and introduction present the work as harness self-evolution and as a solution to the composition of context edits. In practice it is closer to small-scale prompt optimization, and the composition problem is studied only for a single pair of edits. This limitation surfaces only in the method section and appendix, and there is no Limitations section discussing it.

Questions for authors

Q1. Do the gains from paired repair really come from interactions that require both templates to be revised together? In Table 5, the one-sided revisions (P10, P01) also improve on the original (P00) on average. That suggests the landscape is reachable by sequential revision.

Q2. Is the comparison against external baselines in RQ1 fair? How many rollouts, LM calls, and tokens did GEPA, ACE, SkillOpt, and RHO each use, and how does that compare with MICA?

Q3. Why is such elaborate search machinery needed in such a small space, and how does it scale? With roughly 11 structures, exhaustively evaluating all of them would take only about small portion of the fitting budget. Compared with simple baselines such as “evaluate all structures + template repair” or “select the best single edit,” what do coverage, epsilon-exploration, and racing actually contribute? And how do the authors plan to extend matched blocks and the 2×2 repair to programs with three or more edits?

5 comments
AuthorsOpenReview ↗

Reply - Part 1

We greatly appreciate your recognition of the paper's strengths and your insightful criticism of its weaknesses.

Regarding your criticisms of the paper's weaknesses, we respectfully note that:

W1. What does “intervention” mean?

We would like to clarify that MICA does not claim to implement strict counterfactual trajectory replay. It performs controlled intervention-based comparisons of different context-edit programs, rather than simply repeating the same program. As described in Section 3.3, MICA executes the unedited program, individual edits, and their legal compositions from the same initial environment state, agent state, tool definitions, and message prefix. Reward contrasts are then used to examine non-additive effects, order effects, and conditional gains between edits. Section 3.4 further holds the structure fixed and compares four template combinations to assess the additional benefit of joint revision over unilateral revision. These comparisons provide testable execution evidence for diagnosing and repairing composition mismatches, rather than deterministic causal attribution for an individual trajectory.

We acknowledge that LM sampling and subsequent user-simulator interactions are stochastic across branches. Accordingly, MICA uses sampled rewards to estimate differences in expected performance between program configurations under stochastic execution, rather than strict trajectory-level counterfactual outcomes. The two repeats mentioned by the reviewer apply to each fitting task and program version; the RQ2/RQ3 experiments contain 20 independent optimization runs, and final deployment confirmation uses five repeats. These evaluate different levels of variability and do not eliminate the sampling uncertainty of an individual fitting contrast. MICA’s core contribution is to guide the co-adaptation of context edits through controlled program-level interventions and actual execution feedback; it does not depend on strict counterfactual causal identification.

W2. Scope of harness evolution

We would like to clarify that MICA studies a context-layer realization of Harness Self-Evolution, not unrestricted modification of the entire harness. This scope is stated in the Introduction. Although the current program contains at most two edits, MICA is not limited to optimizing the wording of a single prompt. As described in Sections 3.1–3.2, MICA jointly optimizes the edit structure GG and templates Θ\Theta. Structural optimization includes edit selection, execution order, input bindings, and dependencies. These variables can affect edit activation, information flow, and the agent’s subsequent behavior. Thus, the object of study is both template content and the structural interactions and co-adaptation of executable context edits.

Focusing harness evolution on the context layer is a deliberate design choice. This layer directly affects how an agent interprets task constraints, tool feedback, and execution history, allowing its decisions and tool-use behavior to be influenced without modifying the underlying tools or runtime framework. This design also avoids the engineering overhead of changing tool code or restructuring execution flow, preserves compatibility with existing tool interfaces and frameworks, and makes candidate programs easier to validate, compare through execution, and analyze in terms of performance differences. The restricted edit space keeps structural search and joint repair manageable under a fixed optimization and deployment-validation budget.

AuthorsOpenReview ↗

Reply - Part 2

Additionally, here are our responses to your questions:

Q1. Does paired repair require simultaneous revision?

No: the results do not establish sequential unreachability. They support better finite-budget discovery by paired repair. Three quantities must be distinguished.

1. Aggregate joint advantage versus interaction

I=\bar{Q}_{11}-\bar{Q}_{10}-\bar{Q}_{01}+\bar{Q}_{00}.$$ | Original fitting results | Telecom | AppWorld Normal | |---|---:|---:| | $P_{00}$: original/original | 62.71% | 50.42% | | $P_{10}$: revised/original | 71.46% | 62.36% | | $P_{01}$: original/revised | 69.79% | 64.86% | | $P_{11}$: revised/revised | **81.67%** | **79.31%** | | Joint advantage $A$ | **10.21 pp** | **14.44 pp** | | Factorial interaction $I$ | **3.125 pp** | **2.50 pp** | **$A$ is neither $I$ nor a proposal-level win rate.** In particular, better unilateral means are compatible with a monotonic sequential path. ### 2. Observed sequential paths A strictly improving path satisfies $$(Q_{10}>Q_{00}\land Q_{11}>Q_{10})\\ \lor\\ (Q_{01}>Q_{00}\land Q_{11}>Q_{01}).$$ | Seed-level diagnostic | Telecom | AppWorld Normal | |---|---:|---:| | $P_{11}$ strictly best | 18/20 | 19/20 | | Strictly improving sequential path | **16/20** | **17/20** | | Joint-only improvement | 2/20 | 2/20 | Thus, **most observed joint gains do not require a reward-decreasing intermediate state**. These retrospective hybrid conditions also differ from the candidates independently generated by a sequential optimizer. ### 3. Actual sequential algorithms | Method | Telecom deployed | AppWorld Normal deployed | |---|---:|---:| | Sequential-1pass | 81.13% | 76.99% | | Sequential-best-first | 81.63% | 77.83% | | Sequential-coordinate | 82.13% | 78.63% | | **MICA** | **84.38%** | **81.22%** | | **MICA − coordinate** | **+2.25 pp** | **+2.59 pp** | The key distinction is **reachability versus discovery**. The four-condition diagnostic asks what happens when two already-proposed revisions are recombined. A sequential optimizer must instead generate and select revisions with limited feedback and budget; it does not receive the final joint proposal in advance. The stronger sequential baselines therefore provide a more direct test than the four-condition means alone. Their gains also show that sequential repair is effective, while the remaining gap quantifies the benefit of the tested paired strategy. ## Q2. Are the external comparisons budget-matched? **The additional comparison uses common resource caps, with actual usage reported below.** The original RQ1 matched task splits, tool permissions, and initial framework state; that alone did not establish compute matching. For method $m$, environment $e$, seed $s$, and resource $k\in\\{\text{rollouts,calls,tokens}\\}$: $$C_{m,e,s}^{(k)}\le B_e^{(k)},\qquad B_e^{(k)}\text{ is shared by all methods};\quad \textbf{equal caps}\ne\textbf{equal actual consumption}.$$ | Per-seed cap | Rollouts | LM calls | Tokens (M) | |---|---:|---:|---:| | Telecom | 2,080 | 66,000 | 270 | | ALFWorld | 3,100 | 98,000 | 465 | | AppWorld Normal | 2,940 | 70,000 | 570 | | Airline | 1,500 | 54,000 | 205 | | Retail | 3,100 | 97,000 | 505 | **Actual usage: three-seed mean of each seed's five-environment total.** Tokens include input and output; M = million. | Model | Method | Rollouts | LM calls | Tokens (M) | |---|---|---:|---:|---:| | GPT-5.5 | ACE | 11,145.33 | 309,655.00 | 1,406.45 | | GPT-5.5 | GEPA | 12,079.33 | 367,535.00 | 1,881.38 | | GPT-5.5 | SkillOpt | 11,210.33 | 355,976.67 | 1,703.57 | | GPT-5.5 | RHO | 11,209.00 | 336,088.33 | 1,696.01 | | GPT-5.5 | **MICA** | **10,979.67** | **330,035.33** | **1,653.51** | | DeepSeek-V4-Flash | ACE | 11,141.67 | 307,271.67 | 1,298.31 | | DeepSeek-V4-Flash | GEPA | 12,103.00 | 359,157.00 | 1,754.56 | | DeepSeek-V4-Flash | SkillOpt | 11,337.67 | 353,042.00 | 1,597.61 | | DeepSeek-V4-Flash | RHO | 11,318.33 | 333,718.33 | 1,572.33 | | DeepSeek-V4-Flash | **MICA** | **11,260.33** | **327,907.33** | **1,543.36** | The common caps constrain the opportunity available to each method without forcing methods to spend resources on unnecessary operations. This matters because a rollout count alone does not capture differences in trajectory length or the cost of reflection and repair calls. Reporting all three resources makes the comparison more transparent, while retaining each baseline’s own optimization decisions. The three caps apply simultaneously. Algorithms retain their own search decisions and may leave budget unused.
AuthorsOpenReview ↗

Reply - Part 3

Q3. Why search, what do its components contribute, and how does it scale?

Enumerating 11 structures is feasible; the relevant baseline is enumeration plus template repair and validation. The additional experiments address this directly.

(a) Fixed 11 structures

Both sides use the same fixed bindings/dependencies and repair permissions. The reinvestment baseline spends remaining resources on repair and validation.

MethodTelecomAppWorld Normal
Best-Single-Edit74.17%68.65%
Exhaustive-Structure76.67%71.43%
Exhaustive + Sequential78.33%74.80%
Exhaustive + Paired79.17%75.79%
Exhaustive + Paired + Reinvest80.83%77.78%
MICA-fixed-1183.33%80.75%
Gain over strongest enumeration+2.50 pp+2.97 pp

For the strongest comparison, three-run resource totals are:

EnvironmentMethodRolloutsLM callsTokens (M)
TelecomExhaustive + Paired + Reinvest5,819183,501741.726
TelecomMICA-fixed-116,126193,337782.748
AppWorldExhaustive + Paired + Reinvest8,103192,5161,539.096
AppWorldMICA-fixed-118,527203,3871,624.260

This is a common-cap comparison, not an exactly equal-consumption comparison: MICA uses somewhat more of the available budget. Its benefit is in allocating evaluations and repairs, not merely listing the legal structures.

The successive enumeration baselines also help locate the source of improvement. Adding sequential repair, paired repair, and reinvestment improves upon structure enumeration alone. Thus, the result is not that exhaustive structure selection is ineffective, but that template adaptation and budget allocation remain consequential after the small structure set has been enumerated. MICA is compared against this stronger alternative rather than only against an unrepaired enumeration baseline.

(b) Component ablations

VariantTelecomAppWorld NormalRole tested
w/o Coverage79.17%73.81%Initial evidence coverage
w/o ε-Exploration80.00%75.79%Exploration of less-tested candidates
w/o Common-Panel Racing80.83%76.39%Comparison on shared task panels
Full MICA84.17%81.15%All components

Removing these components reduces the score by 5.00/7.34, 4.17/5.36, and 3.34/4.76 pp, respectively (Telecom/AppWorld). These are contributions within the tested implementation and budget, not universal necessity claims.

ε0.00.10.20.4
Telecom80.00%82.50%84.17%80.00%
AppWorld Normal75.79%78.57%81.15%77.78%

The epsilon sweep is consistent with a budget-allocation trade-off: too little exploration can concentrate effort on an early incumbent, whereas too much can reduce the resources available for adapting and comparing promising candidates. This is an interpretation of the observed trend, not a separate proof of the mechanism. The supported conclusion is that moderate exploration helps in these experiments.

(c) Larger libraries, still two edits

Library size BBStructuresTelecom MICA / enumerationAppWorld MICA / enumeration
41183.33 / 80.83%80.75 / 77.78%
62785.83 / 78.33%82.74 / 74.40%
85187.50 / 76.67%84.72 / 72.82%

Enumeration here is the reinvestment variant. This tests library-size scaling, not higher-order composition.

Under the same bounded search setting, the advantage over enumeration increases as the library grows. This is consistent with enumeration spreading evaluation and repair effort over more candidates. Nevertheless, these experiments cover only the reported library sizes and do not establish asymptotic efficiency or the same advantage for arbitrary libraries.

(d) Three-edit extension

The added compiler supports legal three-edit structures. For a fixed structure, original/revised choices require 23=82^3=8 conditions.

New per-seed capRolloutsLM callsTokens (M)
Telecom3,381105,731438.791
AppWorld Normal4,837113,811931.277
Method under the new budgetsTelecomAppWorld Normal
2-edit MICA (rerun)85.00%82.34%
3-edit Greedy Sequential85.83%83.33%
3-edit Local Pair Repair87.50%85.32%
3-edit Selective Factorial90.00%88.69%

Three-edit repair is a demonstrated bounded extension, not a general scaling solution. Full factorial comparison costs 2m2^m conditions for a block of mm templates. For longer programs, the proposed direction is selective higher-order checks on small interacting subsets followed by whole-program validation; pairwise diagnosis alone can miss higher-order effects.

Comments on the paper

1 comment
AuthorsOpenReview ↗

ICLR 2027 — Urgent Concern: Reviewer Non-Response to Author Rebuttal

Dear Area Chair and Reviewers,

As explicitly stated in the ICLR 2027 Area Chair Guidelines, "Responding to rebuttal comments is a core part of the reviewer’s responsibilities."

We have carefully addressed the reviewers' concerns in our rebuttal and provided detailed clarifications and additional evidence. However, we have not yet received follow-up responses to our rebuttal.

As the discussion period is approaching its deadline, we would sincerely appreciate it if the reviewers could revisit our responses and let us know whether their concerns have been adequately addressed or if any further clarification is needed.

We would also be grateful if the Area Chair could help facilitate this discussion, as encouraged by the official ICLR guidelines.

We believe that constructive discussion is essential for a fair and thorough evaluation of our work, and we sincerely hope that our rebuttal and additional evidence can be fully considered in the final assessment.

Thank you very much for your time and consideration.

Warm regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.