acceptodds
Under review as a conference paper at ICLR 2027

Semantic Relaxation with Large Language Models

Abstract

Recently, agentic workflows have achieved substantial advances across downstream tasks by enabling multiple Large Language Models to interact with external environments (\eg, tools, memory, and API). Nevertheless, their discrete and non-differentiable nature poses a fundamental optimization barrier in that external feedback often fails to translate into explicit update directions. In this work, we carefully design SemanticRelaxation, a domain-agnostic optimization framework that semantically bridges discrete generation with continuous optimization. Specifically, SemanticRelaxation integrates two core components: (i) Semantic Representation Learning (SRL), which learns a joint embedding space by promoting cross-modal correlation between descriptions and their corresponding artifacts while preserving semantic structure; and (ii) Barycentric Semantic Optimization (BSO), which explicitly guides semantic directions through barycentric aggregation and accumulates historical updates as trajectory momentum after parallel transport. Finally, SemanticRelaxation effectively encourages agentic workflows by grounding executable artifact refinements in retrieved semantic trajectories within an iterative optimization loop. Our strategy achieves state-of-the-art performance on benchmarks spanning Agentic Workflows, Code Generation, Molecular Editing, and Neural Architecture Search, increasing the success rate on M3Tool by up to 12.2 and reducing token consumption on HumanEval by 29.2. Comprehensive empirical analyses further support the effectiveness and validity of our framework.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Agentic workflow, Multi-agent system

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_A8a1OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SemanticRelaxation, a framework for optimizing discrete artifacts in agentic workflows by operating in a joint semantic embedding space. The framework consists of two components: Semantic Representation Learning, which aligns natural-language descriptions with corresponding artifacts using a Barlow Twins-style objective and a topological regularizer; and Barycentric Semantic Optimization, which estimates a semantic update direction from performance-weighted barycentric aggregation of evaluated candidates and accumulates historical directions as momentum. The resulting direction guides retrieval of relevant historical transitions, which LLM agents then use to refine the description and generate new candidate artifacts. The method is evaluated on benchmarks spanning Agentic Workflows, Code Generation, Molecular Editing, and Neural Architecture Search.

Critical strengths
  1. The problem motivation is genuine: translating discrete execution feedback into principled optimization directions for agentic workflows is an open and practically important challenge. The paper proposes a method that explores this direction and evaluates it across several distinct domains.
Critical weaknesses
  1. The presentation of the paper is not very clear. Although the problem is intuitively understandable, neither the problem definition nor the method is rigorously established. The notation is also inconsistent. These issues make it difficult to follow the paper and to evaluate its contribution, novelty, and limitations.

  2. In the method, the first step aligns descriptions and artifacts in a shared space, and BSO then estimates a direction to improve performance. The method achieves the best results on most benchmarks. However, the contribution of each component is not sufficiently supported by the experiments. For example, the effect of BSO should be studied more rigorously on the main benchmarks, such as by replacing it with a random direction or removing the momentum. Also, different tasks have different characteristics, and it is unclear why the method works well across them and when it would not.

Questions for authors
  1. Could the authors elaborate on the gain from the alignment design? Since the semantic direction is only used to retrieve historical transitions, and the artifacts are still generated by the LLM agents, how much of the improvement comes from the aligned embedding space itself rather than from the retrieval-augmented refinement?

  2. Compared with alternative ways of choosing the update direction, such as random directions or independent per-candidate updates, what are the experimental advantages of Barycentric Semantic Optimization?

  3. Since the tasks are very diverse, what do the authors think makes the method effective across them, and in which settings would they expect it to help less?

4 comments

We sincerely thank you for your constructive feedback and for recognizing the problem's importance. We have clarified the formulation and notation, expanded component-level experiments on the main benchmarks, and specified conditions and limitations across domains.

References in our responses refer to the revised manuscript; reviewer comments are reproduced verbatim.

W1

The presentation of the paper is not very clear. Although the problem is intuitively understandable, neither the problem definition nor the method is rigorously established. The notation is also inconsistent. These issues make it difficult to follow the paper and to evaluate its contribution, novelty, and limitations.

Response.

We revised Sections 2.1–2.4 and expanded implementation details in Appendices A, C, and D.

Section 2.1, Eq. (1) now defines the constrained discrete objective: x⋆∈arg⁡max⁡x∈X(c),V(x;c)=1U(x;c),x^\star \in \arg\max_{x\in\mathcal X(c),\\ V(x;c)=1} U(x;c), where X(c)\mathcal{X}(c) is the artifact space, VV is validity, and UU is task performance. We distinguish the artifact xtx_t, descriptive summary dtd_t, and modification instruction rr, as well as evaluated candidate comparisons Bt\mathcal{B}_t and historical transitions Mt\mathcal{M}_t.

Section 2.2 distinguishes normalized embeddings from standardized cross-correlation inputs and identifies topology affinities derived from frozen descriptions. It states that SRL learns without performance labels and remains fixed during artifact optimization. Section 2.3 distinguishes the local target, tangent momentum, and accumulated latent target. Appendix C specifies regularized whitening and singular-covariance/zero-update cases.

Section 2.4, Eq. (8) defines retrieval through the alignment of the BSO query and transported edit directions, signed by observed performance changes. Algorithm 1 and Appendix D specify eligibility, absolute-score ranking, directional labels, ties, and empty retrieval. Figure 3 and Eq. (9) connect this evidence to instructions, artifact generation under task constraints, and evaluation.

The formulation separates the objective from latent guidance: candidates are generated discretely and assessed through external evaluation.

W2

In the method, the first step aligns descriptions and artifacts in a shared space, and BSO then estimates a direction to improve performance. The method achieves the best results on most benchmarks. However, the contribution of each component is not sufficiently supported by the experiments. For example, the effect of BSO should be studied more rigorously on the main benchmarks, such as by replacing it with a random direction or removing the momentum. Also, different tasks have different characteristics, and it is unclear why the method works well across them and when it would not.

Response.

We expanded Section 3.3, Table 4, with representation, direction, momentum, retrieval, and refinement ablations on ALFWorld, SciWorld, and M3Tool. Section 3.4 and Appendix E add alignment comparisons and directional-search diagnostics.

Appendix A.1 specifies shared GPT-4o-mini generation, initialization, search seed 0, evaluation populations of 134/26/82 instances or cases, and eight attempts including initialization, failures, and cache reuse. Tables 4–5 report best observed scores within this budget on the shared populations. Below, Q1 addresses alignment, Q2 alternative directions, and Q3 applicability and limitations.

Q1

Could the authors elaborate on the gain from the alignment design? Since the semantic direction is only used to retrieve historical transitions, and the artifacts are still generated by the LLM agents, how much of the improvement comes from the aligned embedding space itself rather than from the retrieval-augmented refinement?

Response.

Although the LLM generates artifacts, the representation determines which editing evidence it receives and how that evidence relates to the intended update. We therefore compare representations while retaining the optimization and generation pipeline.

Alignment objectives — Section 3.4, Table 5; Appendix A.1.

AlignmentALFWorldSciWorldM3Tool
Normalized ℓ2\ell_20.7310.6420.707
Multi-positive contrastive0.7610.6400.683
SRL0.7840.6680.756

The alternatives share encoders, projection architecture, training split, initialization, and minibatch schedule. Representations remain fixed during BSO and refinement, connecting alignment design to optimization outcomes. Both alternative objectives omit topology regularization, as specified in Appendix A.1; Table 4 separately evaluates the effect of removing this regularizer.

Historical context controls — Section 3.3, Table 4. Score-ordered history without SRL yields 0.694/0.489/0.622, or 9.0–17.9 points below full performance. Artifact-only optimization retains BSO and retrieval over frozen features but omits the description encoder and learned projections, yielding 0.731/0.610/0.659. This evaluates the joint representation as a whole.

Uniform selection retains eligible records but removes directional ranking, yielding 0.746/0.639/0.683, a 2.9–7.3-point reduction. These conditional effects support both representation and selection within the same refinement pipeline; their contributions interact.

Appendix E, Table 10 provides complementary semantic checks. Redundancy reduction improves matching after meaningful module changes from 28.95% to 44.74%; topology preservation raises it to 50.00% and improves rephrased-query matching from 16.67% to 35.71%. Tables 4–5 additionally test optimization outcomes.

Q2

Compared with alternative ways of choosing the update direction, such as random directions or independent per-candidate updates, what are the experimental advantages of Barycentric Semantic Optimization?

Response.

Main-benchmark controls — Section 3.3, Table 4; Appendix A.1.

Update variantALFWorldSciWorldM3Tool
Score-target gradient, w/o BSO0.7390.6230.671
Independent candidate, w/o btb_t0.7390.6350.683
No momentum, w/o mtm_t0.7540.6400.683
Full BSO0.7840.6680.756

Appendix A.1 specifies the controls:

  • Independent candidate: substitute a uniformly sampled candidate's normalized embedding for the barycentric direction. This omits centering, whitening, and performance weighting in direction construction, retaining the step mapping, momentum, retrieval, generation pipeline, and budget.
  • No momentum: retain the current local BSO update but remove accumulated historical directions.
  • Score-target gradient: retain learned representations and retrieval, replacing BSO with the specified score-target gradient without whitening or momentum.

Full BSO outperforms independent candidate updates by 3.3–7.3 percentage points, the no-momentum variant by 2.8–7.3 points, and the score-target-gradient variant by 4.5–8.5 points. The independent-candidate comparison tests the full local direction construction, while the momentum control examines accumulation more specifically.

Random directions — Section 3.4, Figure 4; Appendix A.5. On 50 paired Semantle instances with shared representation, generation pipeline, initialization, and guess budget, a same-norm random tangent query reduces discovery from 56% to 42%. Reversal yields 36%, and gain-ranked retrieval yields 46%. This pure random-direction control is on Semantle; Table 4 provides candidate-based and momentum controls on the main benchmarks.

Appendix D clarifies that reversal preserves selected records at a fixed state while exchanging labels, testing interpretation rather than selection.

Appendix E, Figure 9, compares 12 frozen states under three paired seeds. Our method and BOPRO improve the parent in 3/36 trials, versus 1/36 for gain-ranked retrieval and 0/36 for uniform selection. Especially notable are our larger positive state-level mean gains compared with BOPRO's near-zero positive means, revealing differences in gain magnitude despite identical improvement counts. Overall means remain negative, so this does not imply reliable improvement at every step.

Q3

Since the tasks are very diverse, what do the authors think makes the method effective across them, and in which settings would they expect it to help less?

Response.

The tasks share an optimization interface: an editable artifact, a description, external evaluation, and prior edits that can inform subsequent changes. Task-specific scores weight candidate updates and orient retrieved evidence, while generators, constraints, and metrics remain domain-specific.

Section 2.2 explicitly states that representations are learned separately for each domain. The experiments evaluate a common procedure with domain-specific representations, rather than one embedding model transferred unchanged across tasks. Section 3.1 and Appendix A specify these settings.

We expect benefits when descriptions capture behaviorally relevant differences, feedback distinguishes candidate quality, and prior edits offer transferable guidance. Section 3.4, Figure 5, and Appendix E, Figure 12, trace this transfer through instructions and concrete workflow changes. These examples support this interpretation without establishing a universal explanation across domains.

Appendix F, “Limitations,” now discusses poor descriptions, scarce edit records, sparse or noisy feedback, and geometry that misses task-relevant relationships. Prior SRL training also adds overhead that may limit benefits for short searches. Directional alignment does not guarantee validity or improvement.

The revised discussion includes observed limits:

  • Section 3.2, Table 1: the method leads on four of six workflow benchmarks, not ALFWorld or PDDL. Macro-level guidance may insufficiently specify the concrete action sequences needed in these tasks.
  • Section 3.5; Appendix E, Tables 7–8: molecular gains vary by objective. Under loose multi-objective criteria, accuracy increases from AgentDrug's 31.3% to 60.3%, but validity decreases from 78.7% to 72.3%.
  • Appendix E, Figures 9–10: negative mean proposal gains and target-dependent failures show that aggregate search improvements can coexist with unsuccessful proposals and difficult targets.

We appreciate your comments, which prompted clearer definitions, more direct controls, and explicit applicability conditions.

Dear Reviewer A8a1,

We would like to express our sincere gratitude for the time and effort you have dedicated to reviewing our manuscript. We greatly appreciate your constructive feedback, which has helped us identify important areas for improvement in both the methodological presentation and experimental validation.

Guided by your valuable suggestions, we have carefully revised the problem formulation and notation, conducted additional ablation studies to examine the individual contributions of SRL and BSO, including alternative update directions and momentum, and expanded our discussion of the method's applicability and limitations across different tasks.

We sincerely understand how demanding this period can be. However, with the discussion period nearing its conclusion, we would be extremely grateful if you could kindly review our responses and share your thoughts on whether any concerns remain, should your schedule allow. Your feedback would mean a great deal to us and would be immensely helpful in further strengthening our work.

Thank you once again for your generous time, careful evaluation, and thoughtful consideration.

Reviewer_5P7JOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work proposes SemanticRelaxation, a framework that computes update directions in a continuous semantic representation space and then uses retrieval and LLMs to translate these directions into executable discrete edits, thereby guiding optimization decisions for discrete artifacts such as agent workflows, code, molecular structures, and neural network architectures. Its two main components are Semantic Representation Learning (SRL) and Barycentric Semantic Optimization (BSO). The resulting numerical directions guide the retrieval of historical edits, which agents use to produce concrete modifications. The method achieves improved performance across multiple tasks. The paper’s core contribution is to equip LLM-driven discrete optimization with a numerical update direction that can be estimated from execution feedback and accumulated across search iterations, and to use this direction to guide the retrieval and transfer of prior editing experience.

Critical strengths
  1. An explicit connection between persistent optimization directions and evaluated editing experience.

The paper combines a learned description–artifact representation with performance-conditioned barycentric updates and transported lineage momentum, then uses the resulting direction to retrieve evaluated edit transitions. I find this coupling more compelling than continuous-space optimization or retrieval alone: it provides a concrete mechanism for aggregating feedback across candidates and iterations and using that information to guide subsequent discrete edits. This is the main methodological consideration supporting my positive assessment.

  1. Meaningful empirical gains with improved computational efficiency.

The reported results show substantial improvements over strong task-specific baselines in various areas.

Critical weaknesses
  1. The core mechanism is not sufficiently validated, and comparisons with closely related work are needed.

BOPRO[1] adopts a similar approach and also optimizes the generation of discrete artifacts. I would like the authors to include a direct comparison with BOPRO.

[1] Dhruv Agarwal, Manoj Ghuhan Arivazhagan, Rajarshi Das, Sandesh Swamy, Sopan Khosla, and Rashmi Gangadharaiah. Searching for Optimal Solutions with LLMs via Bayesian Optimization. ICLR, 2025.

Questions for authors

Clarification requests:

The authors should clarify how the numerical update direction is used to select historical edit records. The pseudocode is not sufficiently clear or explicit; for example, how is the retrieval scoring function defined?

Suggestions:

The authors should include additional experimental comparisons with closely related methods.

5 comments

We sincerely thank you for your constructive review and for recognizing the connection between accumulated directions and evaluated editing experience. We agree that direct BOPRO comparisons and an explicit retrieval procedure were needed. We have added these comparisons and expanded the mechanism analysis.

References in our responses refer to the revised manuscript; reviewer comments are reproduced verbatim.

W1

The core mechanism is not sufficiently validated, and comparisons with closely related work are needed.

BOPRO[1] adopts a similar approach and also optimizes the generation of discrete artifacts. I would like the authors to include a direct comparison with BOPRO.

[1] Dhruv Agarwal, Manoj Ghuhan Arivazhagan, Rajarshi Das, Sandesh Swamy, Sopan Khosla, and Rashmi Gangadharaiah. Searching for Optimal Solutions with LLMs via Bayesian Optimization. ICLR, 2025.

Response.

Following your suggestion, we added BOPRO comparisons under both a common workflow-generation pipeline and its native Semantle pipeline. We also added fixed-state retrieval comparisons and component controls to examine the mechanism underlying the improvements.

1. Workflow comparison — Section 3.4, Table 5; Appendix A.1.

We adapted BOPRO-UCB to the same workflow search and Refine–Model pipeline.

MethodALFWorldSciWorldM3Tool
BOPRO-UCB, adapted0.7390.6400.683
SemanticRelaxation0.7840.6680.756
Difference, percentage points+4.5+2.8+7.3

Both use GPT-4o-mini, identical initial workflows and evaluation populations, search seed 0, and eight attempts, including initialization, failures, and cache reuse. The populations contain 134 ALFWorld instances, 26 SciWorld cases, and 82 M3Tool tasks. We report the best observed scores within this budget on the shared populations.

Appendix A.1, “BOPRO workflow adaptation,” specifies a Gaussian process over frozen all-MiniLM-L6-v2 embeddings, a Matérn-5/2 kernel, and UCB acquisition with exploration parameter 0.3. Each proposal retrieves up to three evaluated workflows by cosine similarity for the common Refine–Model pipeline.

Holding generation and candidate budgets fixed, this comparison examines how the methods construct search guidance and retrieval context. SemanticRelaxation obtains higher scores on all three benchmarks.

2. BOPRO's native pipeline — Section 3.4, Figure 4; Appendix A.5; Appendix E, Figure 10.

We added Semantle experiments following BOPRO's task setup. Appendix A.5 specifies 50 paired instances: ten targets and five initial sets. All methods use Llama-3.1-8B-Instruct, 20 shared warm-start words, and up to 1,000 additional guesses, stopping upon discovery. Invalid and repeated guesses count toward the budget.

BOPRO retains its native UCB-based pipeline and ten retrieved candidates; SemanticRelaxation uses its Refine–Model pipeline and three edit records. The workflow comparison controls generation, while this experiment preserves BOPRO's pipeline under matched initial observations and guess budgets.

Discovery reaches 56% for SemanticRelaxation versus 48% for BOPRO. Figure 4 tracks cumulative discovery, similarity, and gains over initialization. Performance-only, random-direction, and reversed-direction controls reach 46%, 42%, and 36%. They share our representation and generation pipeline while changing ranking or directional guidance.

Appendix E, “Target Discovery on Semantle,” Figure 10, shows highest or joint-highest success on six of ten targets, alongside target-dependent failures.

3. Retrieval from identical states — Appendix E, “Effect of Semantic Retrieval,” Figure 9.

To examine proposal quality without confounding it with diverging histories, we compare 12 frozen states across ten targets. Variants share parents and observations and generate ten guesses under three paired decoding seeds: 36 trials each.

VariantMean best-candidate changeImproving trials
SemanticRelaxation, K=3−0.06223/36
BOPRO, K=10−0.06953/36
Performance-only, K=3−0.07731/36
SemanticRelaxation, K=1−0.07991/36
Uniform selection, K=3−0.10100/36

SemanticRelaxation has the smallest mean decrease and more improving trials than score-only, single-record, or uniform retrieval. Especially notable are its larger positive state-level mean gains compared with BOPRO's near-zero positive means: identical improvement counts (3/36) do not capture differences in gain magnitude. Each point averages three decoding seeds for one state.

These are proposal-quality diagnostics: both methods improve in 3/36 trials, all overall means are negative, and BOPRO retains a different pipeline and retrieval capacity. They do not establish reliable one-step improvement. Parent retention prevents unsuccessful proposals from reducing best-so-far performance (Section 2.4; Appendix D).

4. Mechanism validation — Section 3.3, Table 4; Section 3.4, Figure 5; Appendices A.1 and E.

We expanded ablations under the common workflow protocol:

  • History without SRL guidance: ranking records by task scores without learned joint embeddings or directional reranking reduces performance by 9.0–17.9 percentage points.
  • Artifact-only representation: retaining BSO and retrieval over frozen features while removing the description encoder and learned projections reduces performance by 5.3–9.7 points.
  • Uniform evidence selection: retaining eligible records but removing directional ranking reduces performance by 2.9–7.3 points.
  • Direction and refinement: replacing BSO with a score-target gradient, removing momentum, and removing the separate Refine Agent yield reductions of 4.5–8.5, 2.8–7.3, and 2.6–6.1 points.

These controls examine whether history alone suffices and whether its representation, selection, and translation into instructions contribute to optimization.

We expanded Section 3.4, “Qualitative Analysis of Editing Evidence,” to trace intermediate changes. In Figure 5, retrieved evidence recommending multiple experiences becomes an instruction to blend plans; the generated module implements this and completes the mug task in eight turns, versus 47 for performance-only and failure after 50 for random guidance. Appendix E, Figure 12, traces an object-location example. These cases illustrate the evidence–instruction–artifact connection alongside controlled experiments.

Q1

The authors should clarify how the numerical update direction is used to select historical edit records. The pseudocode is not sufficiently clear or explicit; for example, how is the retrieval scoring function defined?

Response.

We revised Section 2.4, Eq. (8) and added Algorithm 1, with implementation details in Appendix D, to specify both the scoring function and how scores are used.

Scoring. For an evaluated parent–child transition ee, we define

\cos\left(u_t,\operatorname{PT}_{z_e^-\rightarrow z_t} \left(\operatorname{Log}_{z_e^-}(z_e^+)\right)\right).$$ Here, $u_t=\operatorname{Log}_{z_t}(\widehat z_{t+1})$ is the BSO query direction, and $\Delta y_e=U(x_e^+;c)-U(x_e^-;c)$ is the measured parent–child performance change. The current embedding $z_t$ anchors the transport of historical edit directions. The query direction $u_t$, derived from performance-weighted updates and momentum in **Section 2.3, Eqs. (6)–(7)**, determines their alignment. The sign of $\Delta y_e$ orients the evidence according to its measured outcome. **No separate artifact-similarity term is used.** **Selection and interpretation.** Algorithm 1 and Appendix D specify: 1. **Eligibility:** prior, protocol-compatible workflow transitions changing one permitted component, with descriptions and nonzero performance changes. Duplicate edges and undefined or zero directions are excluded. 2. **Ranking:** stable sorting by absolute score, retaining at most three records and reserving the strongest of each sign when both exist. Remaining capacity follows rank order; ties preserve memory order. 3. **Evidence:** records receive same-direction or opposite-direction labels and retain measured outcomes and feedback. Performance-change magnitude is not a ranking weight; regressions remain counterevidence against repeating an edit. 4. **Boundary cases:** undefined or zero queries produce empty retrieval; zero-score records are excluded. Empty retrieval uses remaining task and candidate context. Query reversal at a fixed state preserves absolute rankings and selected records while exchanging labels, testing interpretation rather than selection (**Appendix D**). **Figure 3 and Eq. (9)** connect this evidence to refinement instructions, artifact generation, evaluation, and memory updates. The best parent or offspring is retained, preserving the parent on ties; retrieval scores remain internal. This distinguishes retrieval of artifacts near BOPRO's surrogate-guided proposal from our comparison of evaluated edit directions at the current artifact, whose directional relationships and outcomes guide refinement. **Q2 — Suggestion** > `The authors should include additional experimental comparisons with closely related methods.` **Response.** Beyond BOPRO, **Section 3.4, Table 5**, compares normalized ($\ell_2$) and multi-positive contrastive alignment with shared encoders, projection architecture, training split, initialization, and minibatch schedule (**Appendix A.1**). They achieve 0.731/0.642/0.707 and 0.761/0.640/0.683, versus 0.784/0.668/0.756 for SemanticRelaxation, connecting representation design to optimization outcomes. We expanded **Appendix B.2, “Semantic Retrieval for LLM Optimization,”** to discuss BOPRO alongside OPRO, TextGrad, GEPA, InstructZero, LLAMBO, and MIPRO. This clarifies their relationship to our use of learned geometry and accumulated directions to select and interpret evaluated transitions. *We appreciate your suggestions, which helped make the comparisons and retrieval mechanism more explicit and better supported.*
Reviewer_QoTHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Semantic Relaxation with Large Language Models proposes a framework for optimizing discrete artifacts, including agent policies, programs, molecules, and neural architectures, using external evaluation feedback. The method first learns a shared embedding space for natural-language descriptions and their corresponding artifacts through cross-modal alignment, redundancy reduction, and neighborhood preservation. It then applies Barycentric Semantic Optimization to estimate promising search directions from performance-weighted candidate embeddings, incorporating historical updates through momentum with parallel transport. These directions guide retrieval of previously evaluated editing trajectories, which two LLM agents use to generate refinement instructions and revised artifacts. The framework therefore connects continuous semantic search with discrete generation without requiring differentiation through the generator or evaluator. Experiments report the best performance among compared methods on four of six agent benchmarks, a 12.2-percentage-point improvement on M3Tool, and improved code-generation accuracy with lower token consumption. Additional results demonstrate applicability to molecular editing and neural architecture search.

Critical strengths
  1. The method provides an effective way to connect numerical search with LLM-based refinement. SemanticRelaxation computes search directions from candidate performance and uses them to retrieve historical editing experiences, enabling the LLM to generate new candidates informed by previous edits and their outcomes. This connection between candidate comparison and experience reuse is a valuable aspect of the design.

  2. The experiments provide strong support for the practical value of the complete framework. The method achieves the best results on four of six agent benchmarks and improves both correctness and computational efficiency in code generation. The molecular editing experiments also report a trade-off between improved objective satisfaction and a lower proportion of valid molecules.

Critical weaknesses
  1. The novelty lies primarily in using numerical search directions to select editing experiences, yet this crucial step is underspecified. Representation alignment, performance-based weighting, and the aggregation of historical directions all build on established techniques, so the added value depends on how these computations improve experience selection. However, the paper does not explain how the current artifact and search direction jointly determine retrieval rankings, weakening the clarity and reproducibility of the methodological contribution.

  2. The evidence convincingly demonstrates overall system performance, but provides limited insight into why the core design works. Table 4 shows that the model can associate textual descriptions with corresponding artifacts, while the direction-reversal experiments demonstrate that different search directions lead to different outcomes. However, the paper offers little further analysis of how the computed directions help retrieve more useful experiences. Moreover, although Figure 6 is presented as an analysis of the learned semantic space, it actually uses fingerprint distances computed from molecular structures. The paper therefore lacks a coherent account of how representation learning, direction computation, and experience reuse jointly produce the observed gains.

Questions for authors

How does the search direction affect the ranking of historical editing experiences? How is it combined with similarity to the current artifact to determine retrieval results?

How should the molecular fingerprint distances used in Figure 6 be related to the semantic space learned by the method?

5 comments

Thank you for your careful review. We have clarified retrieval and expanded the evidence connecting representation, direction, and refinement. References in our responses refer to the revised manuscript; reviewer comments are reproduced verbatim.

W1

The novelty lies primarily in using numerical search directions to select editing experiences, yet this crucial step is underspecified. Representation alignment, performance-based weighting, and the aggregation of historical directions all build on established techniques, so the added value depends on how these computations improve experience selection. However, the paper does not explain how the current artifact and search direction jointly determine retrieval rankings, weakening the clarity and reproducibility of the methodological contribution.

Response.

We agree that the central retrieval step was insufficiently specified. Section 2.4, Eq. (8), Algorithm 1, and Appendix D now define selection, interpretation, and boundary cases. Figure 3 and Eq. (9) connect retrieved evidence to instructions, artifact generation, and evaluation. We acknowledge the established building blocks; our contribution lies in coupling numerical updates with evaluated editing evidence. The controls under W2 examine its value beyond historical context alone.

Q1

How does the search direction affect the ranking of historical editing experiences? How is it combined with similarity to the current artifact to determine retrieval results?

Response.

Section 2.4, Eq. (8) defines

\cos\left(u_t,\operatorname{PT}_{z_e^-\rightarrow z_t} \left(\operatorname{Log}_{z_e^-}(z_e^+)\right)\right).$$ Here, $u_t=\operatorname{Log}_{z_t}(\widehat z_{t+1})$ is the BSO query direction, and $\Delta y_e$ is the measured parent–child performance change. The current embedding $z_t$ anchors transported edit directions, while $u_t$, derived from performance-weighted updates and momentum in **Section 2.3**, determines their alignment. **No separate artifact-similarity term is used.** **Algorithm 1 and Appendix D** restrict workflow retrieval to prior, protocol-compatible transitions changing one permitted component, with edit descriptions and nonzero performance changes. Duplicate edges and undefined or zero directions are excluded. Absolute scores determine ranking; we retain at most three records, reserving the strongest of each sign when both exist. Ties preserve memory order. Records receive directional labels and retain measured outcomes. Performance-change magnitude is not a ranking weight, and regressions remain counterevidence against repeating an edit. Empty retrieval uses remaining task context. For a fixed state, reversing the query preserves absolute rankings and selected records while exchanging their labels; this tests evidence interpretation rather than selection alone.

W2

The evidence convincingly demonstrates overall system performance, but provides limited insight into why the core design works. Table 4 shows that the model can associate textual descriptions with corresponding artifacts, while the direction-reversal experiments demonstrate that different search directions lead to different outcomes. However, the paper offers little further analysis of how the computed directions help retrieve more useful experiences. Moreover, although Figure 6 is presented as an analysis of the learned semantic space, it actually uses fingerprint distances computed from molecular structures. The paper therefore lacks a coherent account of how representation learning, direction computation, and experience reuse jointly produce the observed gains.

Response.

We expanded the analysis to connect representation quality, directional guidance, evidence selection, and resulting artifact changes through the following comparisons.

1. Component controls — Section 3.3, Table 4; Appendix A.1.

Comparisons share GPT-4o-mini, initialization, evaluation populations, search seed, and eight attempts, including initialization, failures, and cache reuse. Appendix A.1 specifies 134 ALFWorld instances, 26 SciWorld cases, and 82 M3Tool tasks. Results are the best observed scores within this budget.

VariantALFWorldSciWorldM3Tool
SemanticRelaxation0.7840.6680.756
Score-ordered history, w/o SRL0.6940.4890.622
Artifact-only representation0.7310.6100.659
Uniform record selection0.7460.6390.683

Without SRL, history remains available but is ranked by task scores without learned embeddings or directional reranking. The 9.0–17.9-point reductions indicate that this use of history does not recover full performance.

Artifact-only optimization retains BSO and retrieval over frozen features, omitting the description encoder and learned projections. Its 5.3–9.7-point reductions support the learned joint representation. Uniform selection retains eligible evidence but removes directional ranking, reducing performance by 2.9–7.3 points and directly testing evidence selection.

Replacing BSO with a score-target gradient reduces performance by 4.5–8.5 points; removing momentum reduces it by 2.8–7.3 points. Direct editing without the separate Refine Agent reduces performance by 2.6–6.1 points. These controls examine direction construction, accumulation, and explicit instructions. They measure matched-budget search performance on shared populations, not independent held-out generalization.

2. Representation design — Section 3.4, Table 5; Appendices A.1 and E.

Normalized ℓ2\ell_2 and multi-positive contrastive alignment share encoders, projection architecture, training split, initialization, and minibatch schedule. They yield 0.731/0.642/0.707 and 0.761/0.640/0.683, versus SRL's 0.784/0.668/0.756.

BOPRO adapted to the same workflow search and Refine–Model pipeline yields 0.739/0.640/0.683 under matched initialization, population, and budget. This compares search-and-retrieval mechanisms, not alignment losses alone.

In Appendix E, Table 10, redundancy reduction improves matching after meaningful module changes from 28.95% to 44.74%; topology preservation raises it to 50.00% and improves rephrased-query matching from 16.67% to 35.71%. These characterize semantic relationships; Tables 4–5 test their optimization value.

3. Directional search — Section 3.4, Figure 4; Appendices A.5 and E.

Semantle uses 50 paired instances, 20 shared warm-start words, and up to 1,000 additional guesses, including invalid and repeated guesses (Appendix A.5). Our variants share the LLM, frozen SRL heads trained without performance labels, Refine–Model pipeline, and three records. Controls use gain-ranked retrieval, a same-norm random direction, or query reversal. BOPRO retains its native UCB-based pipeline and ten retrieved candidates.

Discovery rates are 56% for SemanticRelaxation, 48% for BOPRO, 46% for performance-only, 42% for random-direction, and 36% for reversed guidance. These test score-only selection, arbitrary directions, and evidence interpretation. Appendix E, Figure 10, reports highest or joint-highest success on six of ten targets, while documenting failures such as machetes.

4. Retrieval from identical states — Appendix E, “Effect of Semantic Retrieval,” Figure 9.

We compare 12 frozen states across ten targets, sharing parents and histories to separate immediate proposal quality from diverging search trajectories. Each variant generates ten guesses under three paired decoding seeds: 36 trials.

VariantMean best-candidate changeImproving trials
SemanticRelaxation, K=3−0.06223/36
BOPRO, K=10−0.06953/36
Performance-only, K=3−0.07731/36
SemanticRelaxation, K=1−0.07991/36
Uniform selection, K=3−0.10100/36

Our method has the smallest mean decrease and more improving trials than score-only, single-record, or uniform retrieval. Especially notable are the larger positive state-level mean gains achieved by SemanticRelaxation (K=3), compared with BOPRO’s near-zero positive means, showing that their identical improvement counts (3/36) do not capture the differences in the magnitude of the observed gains. BOPRO has the same improvement count, and all means are negative; this does not establish reliable one-step improvement or decisive superiority over BOPRO's different pipeline and retrieval capacity. Parent retention prevents unsuccessful proposals from reducing best-so-far performance (Section 2.4; Appendix D).

5. From evidence to execution — Section 3.4, Figure 5; Appendix E, Figure 12.

We expanded “Qualitative Analysis of Editing Evidence” to trace clues through instructions, artifact changes, and execution.

For Figure 5's mug task, a retrieved memory edit recommends multiple relevant experiences. The Refine Agent requests a blended plan; the module combines retrieved plans; the agent succeeds in eight turns. Performance-only retrieval leads mainly to revised feedback labels and success at turn 47. Random-direction guidance fails after 50 turns.

In Appendix E, Figure 12, evidence about checking multiple object locations becomes a corresponding instruction; the module completes the two-keychain task at turn 42. Alternatives mainly reformat prompts or revise feedback labels and fail to complete it. These cases expose intermediate changes alongside controlled comparisons, without establishing how often retrieved evidence produces successful refinement.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Additional Experiments

We thank the reviewers and the Area Chair for their careful evaluation and constructive feedback. We apologize that, in condensing the original manuscript to fit the page limit, we left some methodological details insufficiently explained. Your comments guided the following experimental and presentation revisions.

All references below refer to the revised manuscript unless otherwise stated. Major revisions are highlighted in blue.

A. Additional Experiments and Analyses

1. Direct comparisons with BOPRO.

Following Reviewer 5P7J’s suggestion, Table 5 adds a BOPRO-UCB workflow adaptation under shared evaluation conditions. SemanticRelaxation improves over this baseline by 4.5/2.8/7.3 percentage points on ALFWorld/SciWorld/M3Tool. On Semantle, 50 target–initialization pairs with Llama-3.1-8B-Instruct and matched initial observations and guess budgets yield discovery rates of 56% versus 48% (Figs. 4 and 10). BOPRO retains its native generation pipeline on Semantle; the workflow adaptation applies only to Table 5.

2. Component and representation comparisons.

In response to Reviewers A8a1 and QoTH, Tables 4 and 5 compare history retrieval without SRL, artifact-only representations, alternative alignment objectives, and controls for BSO, topology regularization, momentum, the Refine Agent, and retrieval ranking. The independent-update control substitutes a uniformly sampled candidate’s normalized SRL embedding for the barycentric vector at each local update. It also bypasses performance weighting and whitening, comparing full BSO direction construction with individual-candidate guidance.

3. Directional, retrieval, and qualitative analyses.

Five-way Semantle comparisons include random-direction, performance-only, and reversed-direction controls (Figs. 4 and 10). Retrieval comparisons from 12 frozen search states with three paired decoding seeds examine evidence selection and interpretation under shared parents and observation histories (Fig. 9). ALFWorld cases trace retrieved clues through refinement instructions and artifact changes to execution outcomes (Figs. 5 and 12).

B. Writing and Presentation Revisions

1. Clearer formulation, notation, and retrieval mechanism.

Addressing Reviewer A8a1’s concerns, Section 2.1 defines the constrained discrete optimization objective and distinguishes execution logs, the candidate comparison set, and transition memory. In response to Reviewers QoTH and 5P7J, Section 2.4 and Eq. (8) clarify how the current artifact anchors transported edit directions and how the BSO query determines their alignment. Algorithm 1 and Appendix D specify absolute-score ranking, sign-based selection, evidence labels, filtering, and context construction.

2. Explicit experimental protocols and scope.

Appendix A.1 identifies the modular workflow implementation adopted from AgentSquare and specifies the comparison populations, initialization, seed, candidate budget, and component controls. Tables 4 and 5 report the best observed score on the shared evaluation population within that budget. We distinguish the BOPRO workflow adaptation from its native Semantle setup.

3. Corrected interpretation of the molecular visualization.

Following Reviewer QoTH’s comment, Fig. 7 (originally Fig. 6) is explicitly described as a fingerprint-based chemical neighborhood analysis. Its caption and discussion distinguish this from the learned SRL geometry examined by the existing semantic-perturbation evaluation in Table 10 (originally Table 4). We explain their complementary roles without treating fingerprint neighborhoods as direct evidence of learned SRL geometry.

4. Expanded related work and limitations.

In response to Reviewer 5P7J, Appendix B.2 expands the discussion of BOPRO and related optimization approaches. Addressing Reviewer A8a1’s applicability concerns, the limitations explain the method’s dependence on task-relevant descriptions, useful editing evidence, and informative evaluation feedback, including conditions under which these provide insufficient guidance.

5. Revised claims and consistent presentation.

We revised relevant claims in the Abstract, Introduction, and Conclusion to reflect the clarified methodology and expanded evidence. We also updated the captions and Fig. 3 to maintain consistency with the revised method description and analyses, and corrected typographical errors throughout the manuscript.

The reviewer-specific replies provide detailed responses and supporting evidence for each concern. We appreciate the opportunity to strengthen the paper and welcome further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.