acceptodds
Under review as a conference paper at ICLR 2027

AttriOpt: Attribution-Based Credit Assignment for Evolutionary API-Calling Optimization

Abstract

Large language models (LLMs) are increasingly used to solve complex tasks by composing sequences of calls to external APIs, forming executable API workflows. However, optimizing such API workflows remains challenging because existing methods typically rely on a single evaluation signal of each workflow, providing little information about which API calls actually contribute to performance. This coarse feedback limits the effectiveness of subsequent search, especially when workflow evaluation is expensive. To address this problem, we propose AttriOpt, a statistical attribution framework for evolutionary optimization of LLM-generated API workflows. AttriOpt exploits accumulated evaluations across workflows to estimate fine-grained attribution scores for API identities, parameter configurations, and interaction patterns. These attribution signals are then incorporated into evolutionary search to effectively guide the API calling process. Experiments on mixed-integer linear programming, traveling salesman problem, and logic synthesis show that AttriOpt consistently outperforms existing API-calling methods and LLM-driven evolutionary approaches. Further ablation and attribution analyses demonstrate that these attribution signals provide more informative guidance than relying solely on a single evaluation signal for each workflow.

Primary area:
optimization
Keywords:
Large Language Model, API-Calling, optimization, Evolutionary Computing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_q2cHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AttriOpt, an attribution-guided evolutionary framework for optimizing LLM-generated API workflows. It uses augmented inverse probability weighting (AIPW) and hierarchical shrinkage to estimate the contributions of API calls, parameter configurations, and interaction patterns from historical evaluations. These attribution scores then guide workflow evolution. Experiments on MILP, TSP, and logic synthesis report improvements over existing API-calling and evolutionary methods.

Critical strengths
  1. The paper addresses an important credit assignment problem in evolutionary API workflow optimization. Extracting component-level feedback from historical evaluations offers a promising way to guide workflow refinement beyond scalar rewards.
  2. The evaluation covers three distinct optimization domains. AttriOpt achieves the best mean solving time across all four MILP benchmarks, with notable gains on Cauctions and Facility. These results support the potential value of attribution-guided evolution.
Critical weaknesses
  1. The reliability of the attribution estimates is not sufficiently established. AIPW is applied to adaptively generated workflows with correlated, non-randomly selected components. It is unclear whether the observed context adequately accounts for selection bias. The reported 78.8% agreement between attribution signs and observed improvements is encouraging but does not establish reliable component-level credit assignment.
  2. The experimental evidence has several important gaps. The TSP200 results in Tables 2, 4, and 5 use different metrics and report values whose relationship is unclear, while some standard deviations in Table 2 appear unusually large for the stated three-run protocol. The TSP500 runtime advantage may partly reflect a reduced solver search budget (RUNS=2). Standard algorithm configuration baselines such as SMAC or irace are also missing. Finally, the component ablation is limited to TSP200, where removing attribution changes tour length only from 10.69 to 10.75, without variance estimates.
Questions for authors
  1. How are the assumptions underlying AIPW justified for adaptively generated workflows? Can controlled component interventions validate the attribution estimates?
  2. Does attribution-guided evolution consistently improve performance across all three domains? How does AIPW compare with simpler attribution methods under matched budgets?
  3. The same 20 EPFL circuits are used for optimization and evaluation. Can the selected workflows be tested on held-out circuits?
  4. Can the authors clarify the TSP evaluation settings, particularly the instances, reference tour lengths, and standard deviations in Tables 2, 4, and 5? Are the solver and workflow evaluation budgets consistent across methods?
Reviewer_BxaJOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AttriOpt uses past evaluations of LLM-generated API workflows to identify useful API calls, parameter settings, and combinations. The motivation is that a single score for an entire workflow provides little guidance on which decisions actually helped. The method estimates component-level attribution scores using AIPW and hierarchical shrinkage, then uses them to guide evolutionary search. The authors evaluate it on MILP solver configuration, TSP solver configuration, and logic synthesis. They report improvements in several settings and include ablations to examine the benefit of attribution guidance.

Critical strengths

The framework makes useful use of evaluation history. It turns past workflow scores into guidance about which API calls, parameters, and combinations to retain or revise. This addresses a practical limitation of workflow-level feedback and gives the LLM more specific information for generating new candidates.

Critical weaknesses

The attribution mechanism needs stronger empirical support. The ablation covers only TSP200, where tour length decreases from 10.75 to 10.69 without uncertainty estimates. Figure 4 does not clearly explain how component effects were measured or whether independent validation data were used. Comparisons with simpler component-scoring methods under the same search conditions are also missing. These gaps leave the mechanism’s added value unclear.

Questions for authors
  1. How are the component-level performance changes in Figure 4 measured? What is held fixed during this measurement, and are the validation workflows separate from those used to estimate attribution scores?

  2. Could the authors clarify how workflows are decomposed into API, parameter, and interaction components? In particular, how are call order and repeated parameter assignments represented.

Reviewer_f2HsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AttriOpt, a statistical attribution-guided evolutionary framework for optimizing LLM-generated API workflows. The central motivation is that existing LLM-driven optimization methods typically evaluate complete workflows using a scalar reward, making it difficult to identify which individual API calls, parameter configurations, and interactions contribute to the final performance.

AttriOpt addresses this problem by maintaining an archive of previously evaluated workflows and their outcomes, decomposing workflows into fine-grained components, and estimating component-level attribution scores from historical evaluations. Specifically, it uses augmented inverse probability weighting (AIPW) to estimate the association between component presence and workflow performance while accounting for observed search context. Empirical Bayes hierarchical shrinkage is then applied to regularize estimates for less frequently observed components. Finally, the estimated attribution scores guide evolutionary operators, including elite recombination, backbone refinement, structural mutation, and parameter mutation.

Critical strengths
  1. The paper targets at a meaningful limitation of evolutionary optimization with LLMs. It identifies a practical limitation of existing evolutionary methods: the evaluation of an entire candidate workflow provides limited information about the contribution of individual components.
  2. The proposed framework integrates statistical attribution with evolutionary search. AttriOpt combines several mechanisms into a unified optimization pipeline. It uses AIPW to estimate component-level performance differences, hierarchical shrinkage to regularize noisy estimates, and attribution-guided evolutionary operators to exploit the resulting signals. The use of statistical attribution is an interesting aspect of the design.
  3. The evaluation covers multiple optimization domains. The evaluation spans three different external-tool environments: mathematical optimization, combinatorial optimization, and logic synthesis. These tasks differ in their API spaces, optimization objectives, and execution characteristics.
Critical weaknesses
  1. The statistical attribution estimates may not reliably identify the actual contributions of individual API components. It models component occurrence using a propensity model and workflow performance using conditional outcome models, then combines these estimates through AIPW. However, AIPW does not automatically eliminate confounding. Its causal interpretation depends on assumptions, including adequate adjustment for relevant confounders, sufficient overlap between component-presence states, and appropriate statistical estimation. In an evolutionary search process, component selection is strongly influenced by parent workflows, previous evaluations, and the search policy. These factors may also correlate with performance.
  2. The experimental evidence does not sufficiently isolate the benefits of statistical attribution from other aspects of the evolutionary framework. The paper compares AttriOpt against several established evolutionary baselines and reports improvements across multiple tasks. However, the central claim is that fine-grained statistical attribution improves evolutionary optimization. The existing comparisons do not fully establish that the gains arise specifically from the proposed attribution method rather than from differences in the evolutionary operators, prompting strategy, or allocation of proposals across operators.
  3. The practical cost and scalability of the attribution mechanism are insufficiently evaluated. AttriOpt maintains an archive of evaluated workflows and repeatedly estimates attribution scores for components extracted from that archive. As the search progresses, the number of evaluated workflows and unique components may grow, increasing the cost of fitting propensity and outcome models.
Questions for authors
  1. How do the authors ensure attribution reliability under adaptive evolutionary search, where component selection correlates with parent workflows, previous evaluations, and the search policy?
  2. Can the authors compare AttriOpt with simpler attribution methods under identical evolutionary operators, prompts, and search budgets to isolate the contribution of statistical attribution?
  3. How does attribution overhead scale with archive size and component vocabulary, and does it offset the optimization gains at larger search budgets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.