acceptodds
Under review as a conference paper at ICLR 2027

Counterfactual Harness Intervention Learning for Long-horizon Agents

Abstract

Agent harnesses have become the operational infrastructure of modern large language model agents, coordinating context, tools, verification, and execution control to translate latent model capability into reliable long-horizon behavior. However, reliable long-horizon behavior requires harness control to adapt to task demands, execution environments, and evolving execution states, whereas current harnesses predominantly rely on hand-crafted or globally fixed policies; this mismatch manifests as unnecessary computational overhead and, in adverse cases, reduced task success. To address this limitation, we formulate the task of enabling adaptive orchestration in harness systems as a causal learning problem and propose **C**ounterfactual **H**arness **I**ntervention **L**earning for **L**ong-Horizon Agents (**CHILL-Harness**). CHILL-Harness intervenes at the orchestration layer to enable advantage-guided workflow adaptation, with the goal of improving reasoning and execution efficiency without broadly sacrificing task performance. Specifically, we develop causal intervention effect learning as the effect-estimation component of CHILL-Harness to estimate intervention-relative workflow advantage from confidence-weighted matched execution evidence and identify advantageous workflow adaptations. We further introduce advantage-realizing causal orchestration as its realization component to adaptively allocate counterfactual reasoning and realize only workflow adjustments supported by sufficient expected advantage. Finally, we incorporate a success-preserving objective and advantage-margin authorization constraints into CHILL-Harness to promote reliable adaptation. Experiments on heterogeneous long-horizon tasks spanning information seeking, software engineering, and terminal interaction show that CHILL-Harness substantially reduces token consumption and execution time while maintaining competitive task success across the evaluated benchmarks.

Primary area:
causal reasoning
Keywords:
Causal Inference, Harness, Multi-agent system

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 6 reviews, above 10% of the others.
Reviewer_pzmFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how an agent harness decides whether to keep doing what the agent proposed or to adjust the workflow in long-horizon task (e.g., thinking more, gathering evidence, or wrapping up an answer). Current harnesses mostly make these choices with fixed rules, which the authors argue wastes compute and can sometimes hurt success. There, they propose CHILL-Harness that learns these choices instead. CHILL-Harness restores an execution from a saved checkpoint, runs both the original step and an alternative, and uses the difference in outcomes to train a model that estimates how much an alternative would help. These are all done offline. It also trains small predictors that decide how much extra deliberation to spend, whether to replace the current step, and when the task is done. At run time, the harness first decides how much deliberation is worth spending, then generates and scores alternatives, and only replaces the original step when the estimated benefit is clearly large enough. The method is evaluated on GAIA, SWE-bench Verified, and Terminal-Bench 2.0, where it reports lower token use and runtime than several public harnesses while keeping task success competitive.

Critical strengths
  • The motivation makes sense. Harnesses often spend extra reasoning or verification even when it is not needed. Learning when such interventions are worth their cost is a useful direction for making long-horizon agents cheaper to run.
  • Training signals come from comparing two branches restored from the same checkpoint, which is a reasonable way to estimate whether a change helps. At run time, the method falls back to the original step unless the evidence for a change is strong, and it only ends a task when an explicit check confirms completion. Deciding how much to deliberate before generating candidates also avoids paying for alternatives that are unlikely to be used.
  • The experiments cover three quite different domains (information seeking, software repair, and terminal tasks) and compare against several well-known harnesses. Token counts include the extra model calls made by the method itself.
Critical weaknesses
  • The writing obscures a fairly simple method. The abstract and introduction lean on vague phrases such as “advantage-guided workflow adaptation,” “confidence-weighted matched execution evidence,” and “causal–operational advantage,” and after reading them it is still unclear what the method actually does. The main text also never says which base harness CHILL-Harness is built on, which is basic information needed to interpret the results. Finally, five new acronyms (CIEL, ARCO, CDE, CRE, CAE), together with do-notation and potential-outcome notation, are used to describe what is essentially a comparison between two branches run from the same checkpoint. This could be stated in one sentence and would make the paper much easier to follow.
  • It is unclear which part of the design does the work. The final decision combines a learned effect estimate, an intervention predictor, a route predictor, a replacement classifier, a hand-built operational score (progress, cost, risk, information gain, robustness, safety), and an authorization margin. The only ablation (Always-Full) does not separate these parts.
  • The experiments do not test the paper’s main claim. The paper’s central claim is that CHILL-Harness reduces cost without hurting task success (Section 3.3, RQ1). The clean way to test this is to run the same base agent, with the same model, tools, and budget, both with and without CHILL-Harness, and compare the two. Section 4.1 says such matched reference runs were done, but their results are never reported. Instead, Table 1 compares CHILL-Harness with public systems that use different models and possibly different subsets of tasks, which the authors themselves describe as only “descriptive.” The lower token use and runtime could therefore come from these differences rather than from the method, and RQ1 remains unanswered.
Questions for authors
  • Where are the matched reference results?
  • Why does more deliberation cut GAIA success by more than half in the Always-Full ablation, and what does this say about the robustness of the replacement step?
Reviewer_C8CwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes CHILL-Harness, a framework for adapting the workflows of long-horizon LLM agents, aiming to reduce reasoning and execution costs while preserving task success. Its Causal Intervention Effect Learning (CIEL) component learns the expected utility difference between candidate and reference workflows from offline paired executions starting at the same checkpoint. Its Advantage-Realizing Causal Orchestration (ARCO) component combines these estimates with learned decision signals and operational constraints to select deliberation depth before candidate generation and decide whether to retain or revise the reference workflow. The framework also includes completion verification and conservative authorization rules intended to limit unnecessary or harmful interventions.

Critical strengths

The results provide useful evidence for selective deliberation as a way to reduce agent costs. The Always-Full ablation (Table 2) compares deliberation policies within the same overall framework and reports lower token use with CHILL across all three benchmarks. On Terminal-Bench 2.0, for example, tokens per solved task decrease from 5.65M to 4.87M, while the reported success rates remain close (56.3% and 56.1%, respectively). This supports the practical value of deciding when additional deliberation is worthwhile, particularly for long-horizon agents where repeated planner calls can become expensive.

Critical weaknesses

Critical Weaknesses

1. The methodological novelty is insufficiently distinguished from established rollout-based policy learning and adaptive agent control. Under the paper’s fixed continuation protocol, the proposed workflow effect can be written as a difference between downstream action values, Γt(ω)=Qπcont(χt,ω)−Qπcont(χt,ωt0)\Gamma_t(\omega)=Q^{\pi^{\mathrm{cont}}}(\chi_t,\omega)-Q^{\pi^{\mathrm{cont}}}(\chi_t,\omega_t^0). Estimating alternative-action values through reset-based rollouts and training a predictor from the resulting preferences has direct precedent in rollout-based policy learning and cost-to-go learning [1, 2]. Related agent work also already provides cost-aware selection over heterogeneous operators, as in MaAS [3], and pre-generation reasoning allocation, as in Thinkless [4]. Furthermore, causal analysis of reasoning has precedent: Yu et al. [5] use counterfactual interventions and rollouts to estimate the causal necessity and sufficiency of reasoning steps and refine reasoning chains. Although their intervention targets and causal quantities differ from CHILL’s, the use of causal intervention analysis alone does not establish novelty. These precedents do not reproduce the complete CHILL system, whose contribution may lie in adapting and integrating these ideas for ongoing harness execution. However, the manuscript does not sufficiently explain what new estimation, supervision, or control mechanism, or substantive systems contribution, that this integration introduces.

2. The experiments do not isolate the value of the proposed intervention-effect learning. The sole reported component ablation forces Full deliberation while retaining effect learning, intervention prediction, revision and completion decisions, valuation, and authorization (§4.3, Table 2). This provides evidence for selective routing relative to indiscriminate full deliberation, but does not establish that checkpointed paired-effect supervision or the resulting effect estimates contribute to the gains. A matched comparison with a conventional value-based controller or an otherwise comparable controller without paired-effect supervision, together with held-out validation of effect predictions, would directly address this question. The comparisons between complete systems do not substitute for this control. Consequently, the current evidence supports a narrower claim about selective deliberation than the broader methodological contribution presented.

References

[1] Reinforcement Learning as Classification: Leveraging Modern Classifiers, Michail G. Lagoudakis and Ronald Parr. ICML 2003.

[2] Reinforcement and Imitation Learning via Interactive No-Regret Learning, Stéphane Ross and J. Andrew Bagnell. arXiv:1406.5979, 2014.

[3] Multi-agent Architecture Search via Agentic Supernet, Guibin Zhang et al. ICML 2025.

[4] Thinkless: LLM Learns When to Think, Gongfan Fang, Xinyin Ma, and Xinchao Wang. arXiv:2505.13379, 2025.

[5] Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning, Xiangning Yu et al. NeurIPS 2025.

Questions for authors
  1. Clarification of the claimed novelty. What specific methodological or systems contribution distinguishes CHILL from rollout-based action-value learning [1, 2] and existing causal intervention-based analysis of reasoning [5]? Please provide a concrete comparison identifying what is new in the estimation procedure, training supervision, or use of the learned effects for workflow control. If the contribution is primarily the integration of these ideas at the harness level, please explain the specific technical challenge this integration resolves and how it advances beyond the cited approaches. A clearly established methodological distinction or substantive systems contribution could change my assessment of novelty.

  2. Evidence for the contribution of intervention-effect learning. Can the authors provide a matched comparison showing whether paired-effect learning improves CHILL over an otherwise comparable controller that removes or replaces it? Please retain the remaining controller components and use the same evaluation instances and settings. The comparison should account for the influence of effect estimates on both candidate scoring and intervention-family training, and report task success alongside inference costs. Evidence of a reliable improvement in the success–cost trade-off attributable to this component, beyond what simpler adaptive control achieves, could change my assessment of the empirical support.

Reviewer_yQWHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CHILL-Harness, a framework that enables agents to be more selective in their use of "harnessing" (external planning or intervention mechanisms) during long-horizon tasks, thereby avoiding unnecessary planning and interventions. The core idea is to compare the outcomes and costs of intervening versus not intervening from the same execution state, allowing the agent to learn when an intervention is truly worthwhile. During inference, a "Controller" determines whether additional planning is required, allocates a computational budget, and assesses whether a newly generated plan is worth executing. Experiments demonstrate that this approach reduces token consumption while maintaining a high success rate.

Critical strengths
  1. This paper addresses an intriguing question: is the introduction and adjustment of a "harness" always beneficial, or should it be applied more selectively, with due consideration for the associated costs? This has practical implications for reducing redundant planning and excessive intervention in long-horizon tasks.

  2. The proposed method learns the potential benefits of an intervention through paired comparisons within the same execution state. It determines the depth of reasoning before generating candidate solutions, accounting for both the effectiveness of an intervention and the cost of evaluating it.

  3. The experiments covered agentic tasks and showed empirical effectiveness. All three areas achieved competitive success rates and good token efficiency.

Critical weaknesses
  1. Details regarding the implementation and training of the Controller are insufficient. The paper fails to adequately explain how long-form text, tool feedback, and environment states are encoded. Furthermore, it lacks information on model architecture, parameter scale, and the volume of training data, which hinders reproducibility.

  2. Counterfactual learning is highly sensitive to the quality and coverage of the candidate plans. If the generated candidates are poor, the value of certain interventions might be incorrectly underestimated. The paper does not analyze the impact of the candidate generator, search budget, or stochasticity in the replay process, nor does it account for the costs associated with constructing the offline dataset.

  3. The efficiency comparisons in the experiments are not entirely fair. Since different methods may complete varying numbers of tasks, comparing total token usage or execution time does not necessarily prove superior efficiency. Additionally, the experiments are conducted exclusively using the DeepSeek model family, leaving the cross-model generalization of the approach unaddressed.

  4. The ablation studies are insufficient to isolate the specific contribution of counterfactual learning. The comparison against an "Always-Full" (continuous full-planning) baseline merely demonstrates that selective planning is more efficient than constant full-scale planning; it does not prove that the performance gains stem specifically from learning the value of interventions, as opposed to simple budget control, filtering mechanisms, or other heuristics.

Questions for authors

Please refer to the weakness part.

Reviewer_V6tFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CHILL-Harness, a framework for adaptive orchestration in long-horizon language-model agents. The method consists of Causal Intervention Effect Learning (CIEL), which estimates workflow-relative advantages from checkpointed factual and counterfactual executions, and Advantage-Realizing Causal Orchestration (ARCO), which uses the learned signals to allocate deliberation, select workflow revisions, and authorize execution. Experiments on GAIA, SWE-bench Verified, and Terminal-Bench 2.0 report reductions in token consumption and runtime while maintaining competitive task success.

Critical strengths
  1. Using matched counterfactual executions to construct workflow-effect supervision is an interesting and potentially useful approach.
  2. The framework considers several important aspects of agent orchestration, including deliberation allocation, workflow revision, and completion verification.
Critical weaknesses
  1. Insufficient experimental details and unfair comparisons. The paper does not clearly report the training and evaluation set sizes. Table 1 appears to compare methods evaluated on different numbers of tasks. For example, estimating the task count as Total Tokens / (Tokens per Solved Task × Success Rate) gives approximately 90 tasks for CHILL-Harness versus 100 for OpenHands and Terminus-KIRA on SWE-bench Verified. It's unreasonable to compare methods evaluated on different evaluation subsets.
  2. Incomplete cost accounting. Constructing matched counterfactual training data requires expensive agent executions. However, this offline cost is entirely excluded from the efficiency comparisons and is not quantitatively reported, making it difficult to assess the overall cost-effectiveness of the approach.
  3. Unclear improvement. Beyond the fairness concerns above, the success-rate improvements over existing methods are often within 1–2 percentage points. Without matched evaluations and statistical significance analysis, it is unclear whether the proposed harness provides meaningful improvements.
  4. Insufficient component ablations. The framework contains multiple learned predictors and rule-based components, including intervention-family prediction, CDE, CRE, CAE, causal-effect estimation, candidate valuation, and conservative authorization. However, the only major ablation compares CHILL-Harness against Always-Full deliberation, leaving the contributions of most components unclear.
  5. Many components of the proposed framework appear closely related to existing agent orchestration methods. For example, the hypothesis–feedback–revision loop in Meta-Harness can also be viewed through a counterfactual intervention perspective. The paper does not sufficiently clarify what fundamentally distinguishes its causal formulation and methodology from prior work.
Questions for authors

Aforementioned in the weakness section.

Reviewer_AM6sOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CHILL-Harness is a controller for agents that work through many tool calls. Suppose a coding agent wants to edit a file. The controller can let it proceed, ask it to run a test first, or spend more time looking for another approach. The paper aims to make that choice based on the agent’s current situation.

During training, the authors save an agent’s state, try its proposed next step and an alternative from that same state, and compare the outcomes and costs of the two runs. A model learns to predict which alternative is likely to help. During evaluation, other learned components decide whether to generate alternatives, whether to use one, and whether it is time to stop. A separate verifier must also find evidence of completion before the agent stops. The paper reports competitive success rates and lower total token counts than the listed public systems on GAIA, SWE-bench Verified, and Terminal-Bench 2.0.

Critical strengths

Choosing how much planning to do before generating alternatives is a good practical idea. Suggestions cost tokens even when the controller rejects them. I appreciate that the reported CHILL token counts include planner calls, and that the paper tests what happens when the controller always uses its most extensive planning route.

Starting both training branches from the same saved state gives the effect predictor a more direct learning signal than comparing unrelated runs. The paper also acknowledges that these replay differences approximate the ideal intervention effect and that its success-preserving objective is a goal, not a guarantee. Testing the approach on three different kinds of tasks adds useful breadth.

Critical weaknesses

The headline token savings are hard to interpret across different evaluation counts. Table 1 compares total tokens, while its footnote says the public systems may cover different numbers of valid instances. Using the metric definitions in Appendix D.1, the reported figures imply roughly 90 CHILL evaluations versus 100 baseline evaluations on SWE-bench Verified, and roughly 82 CHILL evaluations versus about 129–131 for Meta-Harness and LemonHarness on Terminal-Bench 2.0. These are estimates because the paper does not report the counts.

On SWE-bench, dividing total tokens by the implied number of evaluations gives about 1.48 million tokens per evaluation for CHILL and 1.53 million for CodeSweep–SWE-agent: a difference of roughly 3–4%, rather than the reported 13.1% difference in totals. On Terminal-Bench, Meta-Harness uses about 2.63 million tokens per evaluation versus CHILL’s 2.73 million. CHILL’s tokens per solved task still look slightly better than Meta-Harness’s, but that comparison may also reflect different evaluated instances. The paper needs results on the same instances, with counts and per-instance costs.

The comparison that would best test the paper’s central claim is missing. Section 4.1 says each CHILL run has a reference run using the same model, tools, task, environment, and limits. Those results are not shown. Without them, I cannot tell whether CHILL actually saves resources while preserving success under matched conditions. It is also unclear whether the public systems were rerun with the same DeepSeek models or drawn from their published results.

The ablation shows that planning frequency matters, but does not isolate the learned effect estimate. In Table 2, forcing full planning raises cost and lowers GAIA success from 71.3% to 27.7%; 76.1% of full-planning events in that variant authorize a change. That is an important result, but it does not tell us whether the effect predictor improves the choices made under CHILL’s normal routing policy. When the controller skips planning, there is no alternative for the effect predictor to score. The paper does not report how often it skips, how often it changes the agent’s proposed action, or whether the resulting changes help. A version with the effect score removed from candidate ranking, and a controller that uses the same planning budget without the paired-replay effect model, would help separate the value of routing from the value of effect learning.

The method would also be easier to place relative to simpler alternatives. Its predicted “intervention effect” measures the expected benefit of one next step over another, learned from runs that start at the same state. A baseline that learns a single score for each action, or one that adjusts planning frequency without an effect model, would make the added value of the proposed components clearer.

Questions for authors

Could you report evaluation counts, task identities, tokens per task, and runtime per task for all methods on matched instances? Please include the reference-harness results described in Section 4.1, identify that harness, and clarify which public systems were rerun with the same DeepSeek model and settings.

How often does CHILL select SKIP, LIGHT, or FULL, and how often does it replace the agent’s proposed action? What happens when the effect score is removed from candidate ranking while keeping the routing policy and planning budget fixed? A routing-only comparison would help test the contribution of paired-replay effect learning.

What information can the completion verifier see during a SWE-bench Verified or Terminal-Bench 2.0 run? In particular, are hidden test results or benchmark success signals unavailable until after the agent has finished?

Reviewer_LUv2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CHILL-Harness learns workflow-relative utility differences from offline checkpointed execution pairs and combines intervention selection, deliberation routing, revision authorization, and evidence-based completion. Experiments on GAIA, SWE-bench Verified, and Terminal-Bench 2.0 report success and resource use, with an Always-Full ablation evaluating adaptive deliberation.

Critical strengths

Separating deliberation before candidate generation from subsequent revision authorization (§3.2; Appendix B) addresses a practical source of agent overhead. Table 2 reports 71.3% versus 27.7% GAIA success for CHILL and Always-Full, supporting selective routing within this system. Appendix A.4 also carefully distinguishes the replay estimand from the ideal causal effect, while §3.3 explicitly treats success preservation as a design target. These qualifications make the formulation more interpretable.

Critical weaknesses

The matched-reference comparisons described in §§4.1–4.2 are not reported numerically. Table 1 explicitly permits different evaluation counts, so its aggregate savings cannot establish reduced cost at preserved success on the same tasks. Additionally, Table 2's Terminal-Bench Always-Full row—259.77M total tokens, 5.65M tokens/solved, and 56.3% success—does not reconcile with Appendix D.1 for a single evaluation pool, even allowing rounding. These reporting gaps prevent a reliable assessment of the central efficiency claim.

The contribution of learned workflow effects remains insufficiently isolated. Always-Full changes routing while retaining effect estimation and the other controls (§4.3); it does not establish the benefit of paired-effect learning over operational valuation. Appendices A.4/B describe objectives and rules but omit the estimator architecture, training/replay counts, and numerical calibration settings, limiting assessment and reproduction of this central contribution.

Questions for authors
  1. Clarification of reported results: Please provide the matched-reference success/cost results and clarify evaluation counts and aggregation. How does the Terminal-Bench Always-Full row reconcile with Appendix D.1? This could resolve the primary empirical concern.
  2. Clarification of supporting evidence: What validation or controlled comparison establishes the contribution of learned workflow effects beyond routing and operational scoring? Please also specify the estimator and training/replay configuration. This determines whether the evidence supports the proposed learning component.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.