acceptodds
Under review as a conference paper at ICLR 2027

TRACE: Tackling Real-World Resource Assignment Problems via Agentic Heuristic Discovery

Abstract

Dynamic resource assignment, the real-time allocation of task streams to heterogeneous processing nodes, is the backbone of modern computing infrastructure. While learning-based schedulers excel in research, industrial deployments still rely on hand-written rules that operators can read, audit, and execute within tight latency budgets. LLM-based Automatic Heuristic Design (AHD) promises to automate writing such rules. However, existing AHD frameworks were developed for combinatorial problems fully specified to the LLM, and they learn only from a scalar fitness score. In real systems, the behaviour that determines a good heuristic, such as processor speeds or power consumption, is unknown a priori: the score reveals which heuristic performs better, but not why. This missing information is recorded in the system logs that every evaluation produces. Exploiting it is non-trivial: logs are massive and noisy, the relevant signals depend on the objective, and their content and format vary across hardware and software stacks, so they can neither be fed to an LLM as is nor processed by a fixed parser. We propose TRACE, which couples an evolutionary AHD loop with an agentic knowledge-extraction workflow. A Reasoner agent analyzes the log schema in light of the objective and formulates hypotheses about the system dynamics; a Coder agent writes and executes schema-specific code to test them, producing insights or executable tools for the evolved heuristics. We evaluate TRACE on a synthetic cloud benchmark and a 5G vRAN scenario built from industrial testbed measurements and operational traffic traces. TRACE consistently outperforms state-of-the-art AHD methods in resource assignment problems and yields more auditable heuristics at under 2% overhead.

Primary area:
optimization
Keywords:
automatic heuristic discovery, LLM-driven evolutionary program search, agentic workflows, dynamic scheduling, resource allocation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rKJyOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents TRACE, a framework that extends LLM-based automated heuristic design for dynamic resource assignment from scalar fitness feedback to structured knowledge extracted from system execution logs. An agentic workflow pairs a Reasoner agent, which formulates hypotheses about unknown system dynamics in light of the optimization objective, with a Coder agent, which writes and executes schema-specific code to produce textual insights or executable surrogate models (e.g., processing-time predictors); these inform an evolutionary loop that generates assignment heuristics. Evaluated on a synthetic cloud benchmark and a measurement-driven 5G vRAN scenario, TRACE outperforms FunSearch, EoH, ReEvo, and HSEvo, reduces energy use versus an industry-standard policy while maintaining zero task loss, and yields more auditable heuristics at under 2% overhead.

Critical strengths

The first strength is a clearly identified and plausible limitation of existing LLM-based heuristic design: scalar fitness cannot explain why one heuristic beats another when system dynamics are unknown, and execution logs are an obvious, largely untapped source of that explanation. The second strength is the 5G case study grounded in industrial testbed measurements and operational traces is also a stronger form of evaluation than purely synthetic benchmarks.

Critical weaknesses

The first and decisive weakness is that the absolute quality of the solutions TRACE produces is unknown, which limits the scientific value of the claimed gains. No optimality gap, lower bound, or even gap-to-oracle is reported in either scenario; all headline numbers ("36× lower cost than EoH," "65% less energy than HA-Only") are purely relative. Because the cost is dominated by the drop penalty, these numbers largely encode a threshold effect — TRACE achieves zero loss while baselines do not — rather than a measured distance from good solutions. Combined with the fact that the best privileged oracle (Oracle-Time) already achieves zero loss in the 5G case and near-zero loss with 1.30 ms delay in Scenario 1, it is impossible to tell how much headroom TRACE leaves on the table or whether a simple informed rule would match it.

The second weakness is that the baseline set cannot support the paper's own framing. The intro positions the work against "learning-based schedulers [that] excel in research," yet no DRL scheduler (Mao et al. 2016; Li et al. 2023, both cited), no classical load-balancing policy (join-shortest-queue, least-loaded), and — most critically — no "surrogate + simple rule" control is evaluated.

Questions for authors

Q1. What is the quality of TRACE's solutions in absolute terms? Specifically: (a) in the cloud scenarios, what is the gap between TRACE's cost/delay and the best achievable policy with known dynamics (e.g., an oracle that knows the true processing-time functions and optimizes the assignment), or a simulated lower bound; (b) in the 5G scenario, how close is TRACE's energy consumption to the minimum energy among all zero-loss policies? Without such a reference, the reported gains cannot be interpreted.

Q2. The most direct test of your mechanism would be a hand-written baseline that receives the same extracted knowledge (the processing-time surrogate, per-processor speed and energy statistics) and applies a simple transparent rule, e.g., assign to the feasible processor minimizing predicted waiting plus processing time, or minimizing energy subject to the deadline. How does TRACE compare against such a surrogate-plus-greedy rule? If the gap is small, what justifies the evolutionary loop and the LLM machinery? If it is large, the comparison would substantially strengthen the paper.

Q3. The paper cites DRL schedulers as the learning-based state of the art for resource assignment but does not compare against any. Why? Even accounting for differences in observability and interface, could you adapt one DRL baseline (e.g., a policy trained with full state information, or evaluated under your restricted interface) so that readers can position TRACE against the paradigm the introduction argues it should replace in deployment?

Reviewer_7ehJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes TRACE, an LLM-based Automatic Heuristic Design (AHD) framework for resource assignment problems. The key idea is to introduce an Agentic Knowledge Extraction module that extracts useful information from the system log to guide heuristic evolution in the generation loop. The experiments evaluate task-to-processor assignment in two simple simulation-based environments.

This paper is well-written, logically organized, and clearly motivated. I appreciate the authors' efforts in polishing the manuscript. However, the technical novelty is insufficient for a top-tier AI conference such as ICLR. My initial assessment would be around 1.5, but I assign a rating of 2 in recognition of the writing quality, although I do not believe the paper meets the acceptance standard.

Critical strengths

The paper is well-written, with reasonable motivation and a clear logical flow. The presentation is carefully organized and polished.

Critical weaknesses
  1. The selected baselines are not sufficiently up-to-date. In particular, the paper lacks comparisons with recent methods such as [1] and [2], as well as Genetic Programming-based Hyper-Heuristics (GPHH).

  2. Evaluations are limited to two simple simulation-based environments. The lack of real-system deployment and online decision-time measurements weakens the authors' claims regarding the dynamics and complexity of real-world applicability.

Questions for authors
  1. Could the authors compare TRACE with recent methods [1], [2], and GPHH under comparable resource assignment settings? The current baselines are insufficient to demonstrate the advantages of Agentic Knowledge Extraction over alternative approaches.

  2. I disagree with the statement that existing AHD approaches share the limitation that "the only feedback is a scalar." For example, ReEvo employs LLM reflection. MLES [1] utilizes multimodal feedback to analyze policy behavior. RASS-Evo [2] leverages regime detection and constraint-violation traces to improve routing heuristics. Therefore, recent AHD methods have already moved beyond purely scalar feedback. Could the authors clarify the differences between TRACE and these existing approaches?


[1] MLES: "Multimodal LLM-assisted evolutionary search for programmatic control policies." ICLR 2026.

[2] RASS-Evo: "LLM-Driven Regime-Adaptive Strategy Synthesis for Dynamic Network Routing." Neural Networks (2026): 109269.

2 comments

The authors have responded to your reviews. Please read their response and reply. Please specify which concerns are resolved and which remain. Please update your review and score if the assessment has changed.

AC

Thank you to the authors for the clarification. However, my concerns regarding W1 and Q1 remain unresolved. Therefore, I maintain my original rating of 2.

Reviewer_X7pFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper looks at LLM-based automatic heuristic design (AHD) for dynamic resource assignment, where things like per-node processing time, energy use and queue capacity are not known in advance. Existing AHD methods only get a scalar fitness. TRACE adds an agentic step on top of an EoH-style loop: a Reasoner (a frontier model) looks at the log schema and proposes a hypothesis, and a Coder (Qwen3-30B) writes code to test it on the collected execution logs. The output is either a short textual insight or a callable surrogate model, and it is merged into the generation prompt. On a synthetic cloud benchmark with 10 or 50 heterogeneous servers and on a 5G vRAN scenario built from real measurements, TRACE beats FunSearch, EoH, ReEvo and HSEvo. It gets zero loss and about 36x lower cost at N=50, and uses 25% less energy than EoH on 5G, with around 2% extra wall-clock time.

Critical strengths
  • The paper targets a real gap. AHD methods only see a scalar score, while real systems produce detailed logs. Letting an agent decide what to pull out of the logs and write code for that specific schema is a sensible idea, and it doesn't depend on which evolutionary backbone is used.
  • Making the learned surrogates available as tools that the evolved heuristics can call is a nice touch. It lets the final heuristic be written in terms of physical quantities.
  • The 5G case uses real testbed measurements and real traffic traces, which is more realistic than the usual AHD benchmarks.
  • The paper includes several useful controls: EoH with the same frontier model used as the Reasoner (App. E.2), different Reasoner models, per-run distributions (Fig. 4), and a cost breakdown.
Critical weaknesses
  1. The setup keeps the key information away from the baselines, and a simple non-agentic baseline is missing. The heuristic is a stateless function of the task features. It never sees queue state or any online statistics. In the cloud benchmark, processing time is exactly linear (t = x^T h_n), so TRACE's surrogate is basically a linear fit that recovers h_n, which is close to oracle knowledge. The scalar-feedback baselines can only pick this up indirectly through search, which probably explains why they end up with "hashing" rules. What I'd really like to know is whether the agentic part matters or whether any regressor fitted on the logs would do the job. I would ask for at least:

    • a fixed pipeline that fits a per-processor regressor on the random-policy log and gives predict() to EoH, with no Reasoner or Coder;
    • EoH with a fixed summary of log statistics (e.g. mean time and energy per processor) in the prompt;
    • a hand-written rule that combines predicted time with load or queue information, e.g. a least-expected-completion-time or JSQ variant using the regressor.

    If these close most of the gap, the contribution is more "give the heuristic a model of the system" than "agentic extraction", which is a lot less novel.

  2. The non-LLM baselines are weak. They are Random, two greedy "oracles" that each ignore one side of the trade-off, and two industry policies. There is no comparison to classical dispatching rules that use both expected service time and queue length, to DRL schedulers (which the introduction argues against), or to model-based optimization once the system is identified. Also, calling these policies "Oracle" suggests an upper bound, which they are not.

  3. Tables 1 and 2 report the best of 10 runs. I think the median and IQR should be the main numbers, especially for Table 2, where no run distribution is shown for 5G. On 5G, the only AHD baseline is EoH. Γ is also set differently per scenario (135 and 600), and the paper doesn't discuss whether this affects the ranking.

  4. "Real-world" is a bit of a stretch. Both scenarios are simulated. The 5G one replays measured latency and energy, but there is no live system. The logs are clean CSV files with useful columns, so the difficulties listed in the introduction (gigabyte-scale, noisy, heterogeneous logs) are never really tested. An experiment with noisier or messier logs, or with irrelevant or misleading columns, would back up the motivation.

  5. The auditability evidence is weak. It is based on LLM judges scoring one best heuristic per method, and TRACE's heuristic calls an opaque predict() (Simulatability 1.0). A small study with human operators, or at least several heuristics per method, would be more convincing.

  6. Robustness of the workflow is unclear. App. E shows performance drops sharply when the Coder fails to produce the surrogate. How often does that happen across all runs? And how often does the workflow produce wrong or misleading knowledge, like a badly fitted surrogate? The claim that TRACE falls back gracefully to EoH is stated but not measured.

Questions for authors
  1. How does TRACE compare to EoH given a fixed surrogate fitted on the initial random-policy log, without the Reasoner and Coder? To me this is the most important ablation.
  2. Why can't the heuristics see queue occupancy? Most real schedulers have it. Do the results still hold when queue state is an input?
  3. Could you report median and IQR for Tables 1 and 2, and the run distribution for the 5G case?
  4. How do the results change if processing times in the cloud benchmark are non-linear, so that a low-order regressor is no longer close to exact?
  5. What happens as logs get noisier, larger, or contain many irrelevant fields?
1 comment

The authors have responded to your reviews. Please read their response and reply. Please specify which concerns are resolved and which remain. Please update your review and score if the assessment has changed.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.